Backtesting vs Live Trading

From MultiCharts
Revision as of 12:57, 18 April 2012 by Dave Masalov (talk | contribs) (Created page with "==Understanding Backtesting limitations== Backtesting (strategy calculation on historical data) is an essential tool for a certain type of strategies. However, it has some lim...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Understanding Backtesting limitations

Backtesting (strategy calculation on historical data) is an essential tool for a certain type of strategies. However, it has some limitations as every simulator has.

When you are trading live, the market always responds to your actions. Your trades and submitted orders always have impact regardless of how small your trade size is. The orders that you submit change market depth and you can see these changes in real-time. This can never be simulated in backtesting as no algorithm is able to recreate market reaction on market depth change.

Moreover, there is always broker latency (exchange latency, internet connection latency). This latency is a dynamic constantly changing value. It can be 400 milliseconds in one moment of time and 1 millisecond 10 seconds later. Consequently, it cannot be compensated in backtesting by setting a fixed latency time (e.g. 400 milliseconds).

It is important to understand that every trading strategy type needs different simulation: scalping strategies require one simulation type, position trading needs another one etc.

Example: A scalping strategy with average profit per trade = 1 pip (strategy 1) will produce less realistic results in backtesting than a position strategy with 300 pips average profit per trade (strategy 2) as 1 or 2 pip variations between backtesting and real trades will make a little impact on strategy 2 while strategy 1 backtesting performance may be completely different from live trading due to these variations.

Broker latency can be negligible for position trading while it should be taken into consideration when the position holding time is relatively small.

Example : Strategy 1 average position holding time is 30 minutes. Strategy 2 average position holding time is several seconds (scalping). In the first case latency factor is negligible, so strategy 1 backtesting results will be more or less the same as live trading performance. However, in case of strategy 2, broker latency can change live trading performance dramatically compared to backtesting.

Generally, the higher average profit per position and average position holding time are, the more accurate backtesting results are.

Backtesting vs Live Trading

Here are the potential reasons why you may see orders not filled in real-time strategy calculation or real-time strategy trading compared to Backtesting results.

Live Trading Backtesting
In real-time trading the order is filled according to market dynamics on Bid and Ask prices taking Volume into account. The order may not be filled in real-time trading because of insufficient volume at the broker. In Backtesting and real-time strategy calculation (real-time simulation or data playback) the order is filled on the main data series price without taking Volume into account. In order to calculate order execution in Backtesting at Ask and Bid prices Precise Backtesting option should be used
In real-time trading, a strategy will monitor and respond to real-time data on tick-by-tick basis. Historical data available for backtesting will, in most cases, be in the form of time-based bars (minute, daily etc.) based on a group of ticks, with only Open, High, Low, and Close prices available.

While it is not possible to get information on the actual price movement within each bar from these four values, in MultiCharts Backtesting accuracy is improved by incorporating Intra-bar Price Movement Assumptions logic.

As real-time and historical data comes from different servers (even if the same data vendor is used) there can be differences in real-time data and historical data that affect the strategy calculation results. A single tick difference between real-time and historical backfilled data can generate completely different looking charts. This in turn would impact the calculations of your strategy.
The difference between real-time and historical calculation is especially seen on certain chart types (e.g. Point and Figure) than others due to their inherent nature in bar formation (see Non-Standard Chart Types).

Backtesting Options

Backtesting provides you with a general idea of how your strategy can behave. This information is not to be solely relied on for making decisions. The features to bring your Backtesting results closer to the real-time strategy behavior are:

Note: Different backtesting and strategy calculation methods can be selected which will produce different results.

Backtesting, Optimization, and forward testing (real-time simulation) provide an insight into potential performance of your strategy in “ideal world”. This information should be analyzed to get a range of possible scenarios. It is up to trader to decide if this range is acceptable for live trading or not.