Continuous Walk Forward Analysis (CWFA)

Questions about MultiCharts and user contributed studies.
quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 06 Dec 2013

Hi guys,

I figured out a way to implement Continuous Walk Forward Analysis (CWFA) on Multicharts. The main idea behind CWFA is using the optimal trading parameters for each out of sample period. As a result, your trading strategy is able to run continuously without interference from periodic re-optimization. CWFA accomplishes this by using date flags to set the parameters for each OOS period.

For example,

Code: Select all

if Date > 1/1/2012 and Date < 1/31/2012 then
begin
Parameter1 = optimalvalue1;
Parameter2 = optimalvalue1;
end;

if Date > 2/1/2012 and Date < 2/27/2012 then
begin
Parameter1 = optimalvalue2;
Parameter2 = optimalvalue2;
end;
The strategy uses an optimal parameter set for January and a different parameter set for February. Hard coding the optimal parameter set for each period can get tedious very quickly. This approach quickly becomes unwieldy if you want to frequently re-optimize your trading strategies say after the end of every day. Luckily, I have devised a better approach for doing this.

However, there are still parts of the process that I have to do manually, because I don’t have complete access to Multicharts API. Hopefully, the thread will get the ball moving and gather more support for CWFA. I already requested this as a feature in the project management.

https://www.multicharts.com/pm/viewissu ... no=MC-1479

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 06 Dec 2013

Trading strategy code template for using CWFA

I created a simple long only trading strategy that buys when price reaches the lower Bollinger Band, and sells when the price reaches the upper Bollinger Band. The trading strategy will serve as an example for implementing CWFA. I want to optimize the look back period and the number of standard deviations used to create the upper and lower band. Below is a sample code template for incorporating CWFA into your trading strategy. The strategy will run on SPY 30-minute bars.

Code: Select all

Input: Length(50), NumDevs(2), Optimization(True);
Variables:
varLength(50), varNumDevs(2),
optimalLength(50), optimalNumDevs(2);


//Set Optimization to true when you're optimizing your trading strategy
//The strategy parameters will be set to input: Length and NumDevs allowing you to find the optimal value
//Set Optimization to false when you want the strategy to use optimal parameters values found in the custom DataSeries Data2
//Data2 contains the optimal values for each OOS day that was found during optimization for each parameter
//The strategy parameters will be set to optimal values contained in Data2
//Condition1 is to ensure that the strategy will only runs on OOS data when the input "optimization" is turned off

if Optimization = True then
begin
varLength = Length;
varNumDevs = NumDevs;
Condition1 = true;
end

else if Close of Data2 <> 0 then
begin
Condition1 = true;
optimalLength = Open of Data2;
optimalNumDevs = Volume of Data2;
varLength = optimalLength;
varNumDevs = optimalNumDevs;
end

else
begin
Condition1 = false;
end ;

if MarketPosition = 0 and Condition1 then
begin
Buy next bar at BollingerBand(Close, varLength, - varNumDevs) Limit;
end;

if MarketPosition = 1 and Condition1 then
begin
Sell next bar at BollingerBand(Close, varLength, varNumDevs) Limit;
end;
Last edited by quantarb on 06 Dec 2013, edited 3 times in total.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 06 Dec 2013

Here are the steps to the procedure I devised for implementing CWFA

1. Set the input Optimization to “True”, and use traditional walk forward optimization to find the optimal trading parameters: Length and NumDevs for each out of sample period. In my example, I used an IS period of 1 year and OOS period of 1 day which means the strategy is re-optimized everyday on 1 year worth of data.

Image

2. Export the report to excel and remove everything except the Out-of-Sample Data Ending interval and optimal values for each parameter.

Image

3. The data needs to be reformatted so you will be able to import the data back into Multicharts as a custom symbol. You have to trick Multicharts into thinking this is a stock. I created a new custom symbol called CustomDataSPY which stores the data. I used the optimal length for the Open, High, Low, and Close and the optimal NumDevs as volume.

Image

Image

4. Add the custom symbol to your chart with the trading signal as the second instrument.

Image

5. Change the input Optimization to “False”, and your strategy now will run using the optimal parameter set for each OOS period which this is one case.
Attachments
image5.png
(42.81 KiB) Downloaded 3370 times
image4.png
(39.62 KiB) Downloaded 3402 times
image3.png
(133.83 KiB) Downloaded 3369 times
image2.png
(148.22 KiB) Downloaded 3380 times
image1.png
(101.13 KiB) Downloaded 3371 times

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 06 Dec 2013

CWFA provides several major advantages over the Walk Forward Optimization (WFO) process that Multicharts currently uses.

1. You are able to generate a single backtest report that shows how your trading strategy would have performed out of sample data from periodic re-optimization. The current WFO report isn’t detailed as a standard backtest report, and the results are very difficult to interpret.

Image

This backtest shows the trading performance of the strategy if it were re-optimized daily. It uses over 120 different optimal parameter sets!

2. CWFA unlike traditional WFO does not unfairly penalize frequent re-optimization. In traditional WFO, the strategy only has access to OOS data when trading each OOS segment. This means the strategy must always wait to have enough new data to calculate trading signals after each optimization. CWFA does not have this problem allowing to you recalibrate your trading strategy as often as you like.

Frequent re-optimization will cause strategy performance to suffer. My walk forward report shows that all of my OOS Net Profits are zero meaning the strategy never managed to get enough data to trade once out of sample.

3. Re-optimization in traditional WFO can cause synchronization problems with your trading strategies and broker account, because the change is applied to the entire time period.

For example, your trading system currently enters you into a long position into your broker account. You optimize your trading strategy again, and the new parameters erase your current long position in your strategy. What do you are supposed to do now? CWFA avoids this issue, because it keeps track of the parameter changes.
Attachments
image6.png
(93.66 KiB) Downloaded 3369 times

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 06 Dec 2013

It would be wonderful if the developers at Multicharts can make CWFA into a standard feature. I had to perform a ton of unnecessary steps, because I had to extract the data from the WFO report manually. Then, I had to load the data back as a custom instrument for the strategy to access.

CWFA can be broken down into four simple steps.
1. Find the optimal parameter set for each OOS period using standard WFO.
2. Generate a data series from the optimal parameter set with their corresponding OOS date period, and then store the data series somewhere so it can be access again in the future.
3. Have the strategy access the data series and set its trading parameters to the values found in the data series.
4. Run the trading strategy with optimal parameters on OOS data only.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 10 Dec 2013

Does anyone know of a dll that is able to transfer data between Microsoft Access and Multicharts. I'm thinking about storing the optimal parameter values in a database and then retrieving the values from a database onto a indicator. This will be more much elegant approach than using custom symbols.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby Henry MultiСharts » 10 Dec 2013

Hello quantarb,

Thank you for your suggestion. All feature requests are forwarded to the management of the company and are evaluated in a timely manner. Please note that even though we value your opinion not all requests can be implemented due to the fact that some features do not fit into our current roadmap.

As for Microsoft Access to MultiCharts dll - I am not aware of a dll like that.
There is ElExcel.dll available for free over the Internet for Excel - MultiCharts communication.
That is also possible to create a dll to read the data from text files into PowerLanguage code.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 11 Dec 2013

Hi Henry,

Thank you for letting me know that Multicharts is considering implementing my suggestion. Also, do you know where I can download ElExcel.dll? I tried to search for it on the forum, but only came across old threads where the download link no longer works. Does ElExcel only work with one instance of excel or am I am able to access data from multiple excel spreadsheets at once?

User avatar
ABC
Posts: 718
Joined: 16 Dec 2006
Location: www.abctradinggroup.com
Has thanked: 125 times
Been thanked: 408 times
Contact:

Re: Continuous Walk Forward Analysis (CWFA)

Postby ABC » 11 Dec 2013

Does ElExcel only work with one instance of excel or am I am able to access data from multiple excel spreadsheets at once?
It only works with one instance of Excel, but you can access different sheets within one Excel file. More than one file at the same doesn't work as far as I recall.

Regards,
ABC

SUPER
Posts: 646
Joined: 03 Mar 2007
Has thanked: 106 times
Been thanked: 84 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby SUPER » 11 Dec 2013

Hi Henry,

Thank you for letting me know that Multicharts is considering implementing my suggestion. Also, do you know where I can download ElExcel.dll? I tried to search for it on the forum, but only came across old threads where the download link no longer works. Does ElExcel only work with one instance of excel or am I am able to access data from multiple excel spreadsheets at once?

try this link:

http://www.tsresearchgroup.com/en/prod.php

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby Henry MultiСharts » 11 Dec 2013

Also, do you know where I can download ElExcel.dll?
Please check here.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 11 Dec 2013

Thanks for all the help so far guys. I went to the link you guys posted. I am still waiting for TradersXchange to approve my account before I can download the file. On their forums, I stumbled on ADE(All Data Everywhere), and this might be better suited for what I'm trying to do.

"ADE includes the ability to save data to text files and load it back."

I read the documentation for ADE. It can shows how to save and store values of indicators in a text file and retrieve it for future use. The problem I'm running into is how do I store values using ADE when the calculations aren't done by signal but rather than the walk forward report. Anyone have any idea how to implement this?

Would there be a way for me to problematically store the values generated from walk forward optimization using ADE? If this is not currently possible, perhaps this would be the best way for the developers at Multicharts implement CWFA. I really like how ADE is able to store indicator data for each symbol.

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Continuous Walk Forward Analysis (CWFA)

Postby JoshM » 11 Dec 2013

The problem I'm running into is how do I store values using ADE when the calculations aren't done by signal but rather than the walk forward report. Anyone have any idea how to implement this?
Not sure if I follow you, but the calculations in a WFA are still done by the signal; the inputs and time period is just different. You can just output the data on the last bar on which the calculation is performed, probably with LastBarOnChart if my memory serves me well.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 11 Dec 2013

The problem I'm running into is how do I store values using ADE when the calculations aren't done by signal but rather than the walk forward report. Anyone have any idea how to implement this?
Not sure if I follow you, but the calculations in a WFA are still done by the signal; the inputs and time period is just different. You can just output the data on the last bar on which the calculation is performed, probably with LastBarOnChart if my memory serves me well.
Thanks to JoshM’s suggestion, I figured out how to output the optimal parameter values for each out of sample day without having to refer to the WFO report. Now I just need to figure the best way to store and retrieve this data.

I'm thinking about using ADE (All Data Everywhere) for storing and retrieving the data I want to store the data along with the symbol and strategy name. This way there will be no misunderstanding if I decide to use the parameter "Length" again for a different strategy. Anyone experienced with ADE (All Data Everywhere) know the best way to implement this?

Code: Select all

Input: Length(50), NumDevs(2), Optimization(True);
Variables:
varLength(50), varNumDevs(2);

if Optimization = True then
begin
varLength = Length;
varNumDevs = NumDevs;

if LastBarOnChart and BarNumber <= 13 then
begin
Print(FormatDate("MM-dd-yyyy", ELDateToDateTime(Date)), Length, NumDevs);
end;
end;

if MarketPosition = 0 then
begin
Buy next bar at BollingerBand(Close, varLength, - varNumDevs) Limit;
end;

if MarketPosition = 1 then
begin
Sell next bar at BollingerBand(Close, varLength, varNumDevs) Limit;
end;
Image
Attachments
image1.png
(62.15 KiB) Downloaded 3265 times

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 13 Dec 2013

I tried to follow the ADE documentation, but I'm not sure where I went wrong. It seems for some reason, ADE is not storing the data. I get the following error when I set Optimization to false. I got Multicharts to output the data correctly in my previous post. Any help or suggestion would be greatly appreciated.

Image

Code: Select all

Input: Length(50), NumDevs(2), Optimization(True);
Variables:
varNumDevs(2), varLength(50),
Class(GetStrategyName), InfoMap(MapSN.New);

if Optimization = True then
begin
varLength = Length;
varNumDevs = NumDevs;

if LastBarOnChart and BarNumber <= 13 then
begin
Value1 = MapSN.Put(InfoMap, "Length", Length);
Value1 = MapSN.Put(InfoMap, "NumDevs", NumDevs);
Value1 = ADE.PutBarInfo(Class, GetSymbolName, 390, ADE.BarID, InfoMap);
end;
end else
begin
varLength = MapSN.Get(InfoMap, "Length");
varNumDevs = MapSn.Get(InfoMap, "NumDevs");
Value1 = ADE.GetBarInfo(Class, GetSymbolName, 390, ADE.BarID, InfoMap);
end;

if MarketPosition = 0 then
begin
Buy next bar at BollingerBand(Close, varLength, - varNumDevs) Limit;
end;

if MarketPosition = 1 then
begin
Sell next bar at BollingerBand(Close, varLength, varNumDevs) Limit;
end;
Attachments
image1.png
(10.29 KiB) Downloaded 3252 times

Trade23
Posts: 3
Joined: 27 Dec 2013

Re: Continuous Walk Forward Analysis (CWFA)

Postby Trade23 » 03 Jan 2014

Have you solved the issue with the error message? Please post the solution.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 33 times

Re: Continuous Walk Forward Analysis (CWFA)

Postby quantarb » 07 Jan 2014

Have you solved the issue with the error message? Please post the solution.
No, I have not. There is currently an issue with using walk forward optimization where Multicharts will stop the WFO a few days before current date even though you're using an OOS of 1 day.



Return to “MultiCharts”