Hi guys,
I know there are threads in this forum adressing this topic already and Multicharts issued several articles on it, for those who don’t know (beginners) please see:
https://www.multicharts.com/trading-sof ... acktesting
https://www.multicharts.com/trading-sof ... ve_Trading
https://www.multicharts.com/trading-sof ... _Magnifier
https://www.multicharts.com/trading-sof ... ng_Results
But this is in my opinion on of the most important questions in Algortihmic trading: How probable is it, that backtesting results can be reproduced in Forward Testing, Live Trading. Just to make this clear: I am not addressing Auto trading (this is the next step, where other issues might occur) just chart trading and new signals being generated. MC is saying in general: The higher the timeframe used (and therefore the larger TP/SL), the more realistic it gets.
Don’t forget about the importance of precise backtesting: When I tested TF <1HR, this produced totally different results.
My data feeds are Oanda and TS at the moment. Oanda offers Bid and Ask data, so precise backtesting is possible.
1. For the moment I decided to backtest on higher Timeframes, mostly 1D, 2HR TF being the smallest, using the Bar Magnifier with Tick (Oanda) and 1-Minute Data. I also enable the Intrabar Order Generation. So with TS data I can only use 1 Min in the Bar Magnifier and Classic Backtesting only. Going for 200-600 pips in general, from your experience how realistic are those results?
With Oanda, even for those high TF, I am using Precise Backtesting and the Bar Magnifier based on Ticks. From your experience guys, is this even necessary?
2. As I did not take the precise backtesting method into consideration in the past, I was disappointed with losses trading on smaller timeframes. What is your experience on the reliability of backtesting results? Where is the threshold of timeframes, where results become unrealistic? Are we “save” backtesting in Classic mode and with 1-min Bar Magnifier only for Timeframes >4HR?
3. I found a interesting source (video) on this topic; unfortunately as I don’t have a webex account, somebody can convert it and share it here:
http://edgesense.net/blog/reliable-back ... lticharts/
Once the forward testing on these higher timeframes goes as supposed, I will dare to go back to the smaller ones and share my experiences. Please share your experiences guys, thanks!
Backtesting results VS Forward Testing
-
Sebastian Vermont
- Posts: 37
- Joined: 06 Jan 2016
- Has thanked: 6 times
- Been thanked: 5 times
Re: Backtesting results VS Forward Testing
I will offer my opinions on your questions. Hopefully others who are more experienced will chime in. To give you an idea of my background: I am still new to MultiCharts, but I have been in the game for years. I have successfully converted my strategies to MultiCharts. These strategies are currently running live.
1) Even on 15M bars I do not use Bar Magnifier. For my strategies, if both a profit and stop loss target is being hit in the same bar with any frequency then there is a serious problem with my logic. I do use Precise Backtesting and I consider it an absolute "must have" for the type of testing I do. Please note that you can use Oanda and TS data at the same time, and if you are going to be executing the orders on Oanda this is definitely what you should do. For example, load the Oanda bid chart (chart / data 1), the Oanda ask chart (chart / data 2), and the TS chart (chart / data 3). Then, in your strategy logic, do something like:
I use eSignal for my data provider and I essentially do the same thing.
2) I can't fully comment on this one, since I always use Precise Backtesting. I will add this, though: Since you can use bid and ask data alongside separate price data, why not always use Precise Backtesting? It's one less thing to worry about.
Your question focused on fills in regards to backtesting vs forward results, so I'm not sure if you are interested in any of the other considerations or not (you may already be fully aware of them and have them fully under control). I'll mention them here just in case they are of any value.
Aside from fills, the biggest thing you need to look at is called curve fitting. There are several things you can do to protect yourself from curve fitting. If this is unfamiliar territory, I recommend starting with "Trading Systems: A New Approach to System Development and Portfolio Optimisation" by Tomasini and Jaekle. By using a combination of careful thought/planning, walk forward testing, and 3D optimization you can create very robust trading strategies.
1) Even on 15M bars I do not use Bar Magnifier. For my strategies, if both a profit and stop loss target is being hit in the same bar with any frequency then there is a serious problem with my logic. I do use Precise Backtesting and I consider it an absolute "must have" for the type of testing I do. Please note that you can use Oanda and TS data at the same time, and if you are going to be executing the orders on Oanda this is definitely what you should do. For example, load the Oanda bid chart (chart / data 1), the Oanda ask chart (chart / data 2), and the TS chart (chart / data 3). Then, in your strategy logic, do something like:
Code: Select all
variables:
chartBrokerBid (1),
chartBrokerAsk (2),
chartPriceData (3),
conditionEngulfingClose (False);
conditionEngulfingClose=Close of Data(chartPriceData) > High[1] of Data(chartPriceData);
2) I can't fully comment on this one, since I always use Precise Backtesting. I will add this, though: Since you can use bid and ask data alongside separate price data, why not always use Precise Backtesting? It's one less thing to worry about.
Your question focused on fills in regards to backtesting vs forward results, so I'm not sure if you are interested in any of the other considerations or not (you may already be fully aware of them and have them fully under control). I'll mention them here just in case they are of any value.
Aside from fills, the biggest thing you need to look at is called curve fitting. There are several things you can do to protect yourself from curve fitting. If this is unfamiliar territory, I recommend starting with "Trading Systems: A New Approach to System Development and Portfolio Optimisation" by Tomasini and Jaekle. By using a combination of careful thought/planning, walk forward testing, and 3D optimization you can create very robust trading strategies.
Re: Backtesting results VS Forward Testing
1) Even on 15M bars I do not use Bar Magnifier. For my strategies, if both a profit and stop loss target is being hit in the same bar with any frequency then there is a serious problem with my logic. I do use Precise Backtesting and I consider it an absolute "must have" for the type of testing I do. Please note that you can use Oanda and TS data at the same time, and if you are going to be executing the orders on Oanda this is definitely what you should do.[/i]
Yes you are right I could not believe thats it makes such a difference in the results using Precise/Classic Backtesting. For higher Timesframes 4HR/1D with large TP/SL (200-500 pips) I still think that it is not absolutley necessary; for the TS data I have to use Classic Backtesting as said before. The same with the Bar Magnifier: For the longterm strategies the 1-Min Bar Magnifier should be enough, no need for the Tick Bar Magnifier.
2) I can't fully comment on this one, since I always use Precise Backtesting. I will add this, though: Since you can use bid and ask data alongside separate price data, why not always use Precise Backtesting? It's one less thing to worry about.
Your question focused on fills in regards to backtesting vs forward results, so I'm not sure if you are interested in any of the other considerations or not (you may already be fully aware of them and have them fully under control). I'll mention them here just in case they are of any value.
Aside from fills, the biggest thing you need to look at is called curve fitting. There are several things you can do to protect yourself from curve fitting. If this is unfamiliar territory, I recommend starting with "Trading Systems: A New Approach to System Development and Portfolio Optimisation" by Tomasini and Jaekle. By using a combination of careful thought/planning, walk forward testing, and 3D optimization you can create very robust trading strategies.
Curve fitting is nothing new to me, but I will have a look at the book. Dont forget about Perry Kaufman: Trading Systems and Methods (you probably know it already).
@all: Please share more experiences about how Backtesting results played out in Forward Testing
Re: Backtesting results VS Forward Testing
just a remark on OANDA...
...in the past I started also out with OANDA. Then they had gaps in their data. Thus the question is how reliable are backtest based on this...
...in the past I started also out with OANDA. Then they had gaps in their data. Thus the question is how reliable are backtest based on this...