Portfolio and Multi-Strategy Walk Forward Optimization

Questions about MultiCharts and user contributed studies.
quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Portfolio and Multi-Strategy Walk Forward Optimization

Postby quantarb » 24 Sep 2012

Hello, I was wondering when the portfolio back tester will have walk forward optimization. Right now you can back test baskets of stocks but without walk forward optimization there is no way to validate how well the strategies will perform out of sample.

Another great feature would be multi-strategy walk forward optimization. Suppose I have ten trading strategies but I only want to trade the top five trading strategies with the best in-sample performance.

The only solution I see for now is to do walk forward optimization by hand by finding optimal parameters and forward test those parameters one by one. I think this feature is crucial since there are some strategies such as pairs trading that you can back test using the portfolio back tester.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby Henry MultiСharts » 25 Sep 2012

Hello quantarb,

We are going to improve multiple aspects of Portfolio Backtester by the end of the year.
The improvement you have mentioned is already in our "to-do" list.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby quantarb » 25 Sep 2012

That is great news Henry.

One major improvement to walk forward optimization would be the ability for the out-sample of data strategy to access enough historical data so it can start trading on the first day.

For example, suppose I only want to test a strategy that involves the 50 day moving average but I only forward test for three months (60 trading days). Right now I have to wait 50 days before the strategy can make a trade. The strategy only has 10 days left to trade.

If I were to optimize the length of the moving average, the computer would most likely pick a shorter length moving average simply because they offer more trading opportunities.

It would be great if I can set an option where for out of sample testing I can use historical data outside of my out of sample data so my 50 day moving average is calculated on the first day.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby Henry MultiСharts » 31 Oct 2012

quantarb, 50 day moving average cannot be calculated on the first day because it implies summation of 50 values and division by 50. Only after the value is calculated for the requested length, an order can be generated on the 51st bar.

segut
Posts: 14
Joined: 01 Nov 2012
Has thanked: 5 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby segut » 01 Nov 2012

That is great news Henry.

It would be great if I can set an option where for out of sample testing I can use historical data outside of my out of sample data so my 50 day moving average is calculated on the first day.
This can lead to data-snooping possibly.

segut
Posts: 14
Joined: 01 Nov 2012
Has thanked: 5 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby segut » 01 Nov 2012

Hello, I was wondering when the portfolio back tester will have walk forward optimization. Right now you can back test baskets of stocks but without walk forward optimization there is no way to validate how well the strategies will perform out of sample.
Out-of-sample testing and wfo are not necessarily related. You can do oos testing without wfo.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby quantarb » 04 Nov 2012

quantarb, 50 day moving average cannot be calculated on the first day because it implies summation of 50 values and division by 50. Only after the value is calculated for the requested length, an order can be generated on the 51st bar.
Hi Henry,

Thank you for your reply. I understand that with the current way Multicharts performs walk forward optimization this is not possible.

In my example, I want to optimize the look back period of a moving average for my strategy.

Current Walk Forward Optimization
In-Sample Data Set: January 01, 2011 to December 31, 2011.
Out of Sample Data Set: January 01, 2012 to March 31, 2012. (60 trading days)

If I want to test the 20 day moving average, the strategy would have to wait 20 trading days before it could start trading. This alone would consume 33% of my out of sample data. My only options are to use much shorter length moving averages or to dramatically increase my out of sample data.

What is I am asking is the option of having three data sets when using Walk Forward Optimization.

New Walk Forward Optimization
In-Sample Data Set: January 01, 2011 to December 31, 2011.
Indicator Data Set: December 01 to December 31, 2011. (20 trading days)
Out of Sample Data Set: January 01, 2012 to March 31, 2012. (60 trading days)

Under this framework, when Multicharts performs the out of sample back testing it will use data from the Indicator Data set which provides enough data to calculate the 20 daily moving average value for January 01, 2012.

The strategy cannot perform any trades outside of the out of sample data set. This will eliminate the possibility of any data snooping. There are many benefits to doing walk forward optimization this way.

1. This greatly reduces the data overhead between rolling optimizations. Under this approach, my out of sample data is really 60 trading days. With the current walk forward optimization framework, my out of sample data is only 40 days since the strategy has to wait 20 days before making a trade.

2. This improves the accuracy of the optimization process. Long term indicators are being heavily penalized under the current walk forward framework because they consume too data. In my example, 33% of my out sample of data was basically unusable for trading. I don’t see how it is feasible to use something like the 200 day moving average.

3. This better reflects how trading strategies are actually deployed from research to production. If I were to trade my example strategy live starting on January 01, 2012, it would make no sense to exclude data from before January 01, 2012. Doing this would mean that I would have to wait 20 trading days before I could start trading. Yet this is what I would have to do if I were to follow the current walk optimization methodology. In the real world, I would use historical data from before January 01, 2012 to calculate my indicators so I can start trading immediately.

I know I'm asking enough, but I think this is a crucial feature that will benefit many of your users.

User avatar
swz168
Posts: 120
Joined: 30 Sep 2010
Location: Germany
Has thanked: 47 times
Been thanked: 32 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby swz168 » 05 Nov 2012

Quantarb, you just wished something, that I want too! Thanks for posting!

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Portfolio and Multi-Strategy Walk Forward Optimization

Postby Henry MultiСharts » 21 Nov 2012

Hello quantarb,

Thank you for your suggestion. It has been forwarded to our developers.
In the meantime you can simply add the required amount of data to your out of sample data to have actual 60 trading days.

You can also find this information interesting:
Our implementation of the genetic optimization is based on the Genetic Algorithm library (GAlib), a description of which can be found here - http://lancet.mit.edu/galib-2.4/Overview.html. This page can provide answers to what exactly happens inside the algorithm. A more general view of how it functions can be obtained from http://en.wikipedia.org/wiki/Genetic_algorithm


Return to “MultiCharts”