Strategy Robustness
What is Strategy Robustness?
Strategy Robustness is an indication of how sporadic the results of backtesting are. The more robust a strategy is, the more backtesting results and real trading results are correlated.
How to Estimate Strategy Robustness?
Sometimes a strategy can show great results because of just one really good trade. Sometimes a strategy shows good results in backtesting, but fails in real trading because of Curve Fitting. Exploring such situations allows for determining the reasons why the strategy was profitable in backtesting, but turned out to be losing in real trading. We’ve analyzed numerous similar situations and discovered the following criteria that can be used in strategy robustness estimation:
- Overall Profitability – a strategy cannot be counted for profitable if it doesn’t generate any profit;
- Profit Factor – the number of profitable trades cannot be much smaller than the number of losing trades;
- Walk-Forward Profitable Runs Count - the number of profitable runs cannot be much smaller than the number of losing runs;
- Total Profit Distribution Across Runs – profit should be distributed more or less evenly between the runs;
- Winning Trades Distribution Across Runs – winning trades should be distributed more or less evenly between the runs;
- Max Drawdown, Max Drawdown of Single Run – the capital drawdown should be limited to some value;
- Number of Trades – there should be set the minimal number of trades so that you could be sure your strategy has enough information for making decisions to get into trades;
- Walk-Forward Efficiency – the more robust a strategy is, the higher efficiency it should show during the Walk-Forward Optimization;
- Custom Fitness Function – you can use Custom Fitness Function for determining strategy robustness according to your own criteria.
How to Estimate a Strategy?
Prior to performing the Walk Forward Optimization or Matrix Optimization it is possible to set up the criteria according to which the robustness of the strategy will be estimated:
The robustness results can be viewed after the optimization in the lower part of the optimization report window:
If the defined criteria turned out to be too harsh you can click on the link with the test result to get into the report settings window and to change the robustness settings.
The results for the new criteria will be applied on-the-fly without the need for performing another optimization.