some detail questions about WalkForwardTest

Questions about MultiCharts and user contributed studies.
oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

some detail questions about WalkForwardTest

Postby oliveagle » 11 Aug 2011

two questions:
1. how indicator values and trading algorithms are calculated in OOS. plz see my attachment for detail.

2. How trade is managed in OOS.
If I have a open position in the first OOS session, MC will let it open or close this position at the end of the OOS session.
If MC will keep the openning position, will it affect the calculation in next OOS session?
Attachments
OOS_Calculation_Question.JPG
OOS_Calculation_Question.JPG (63.35 KiB) Viewed 201 times

oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

Re: some detail questions about WalkForwardTest

Postby oliveagle » 12 Aug 2011

Any one knows about it? thx a million

User avatar
TJ
Posts: 6550
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 965 times
Been thanked: 1892 times

Re: some detail questions about WalkForwardTest

Postby TJ » 12 Aug 2011

oliveagle wrote:two questions:
1. how indicator values and trading algorithms are calculated in OOS. plz see my attachment for detail.

2. How trade is managed in OOS.
If I have a open position in the first OOS session, MC will let it open or close this position at the end of the OOS session.
If MC will keep the openning position, will it affect the calculation in next OOS session?

What is OOS?
What is IS?

oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

Re: some detail questions about WalkForwardTest

Postby oliveagle » 13 Aug 2011

OOS = Out of Sample
IS = In Sample

check this out for detail : http://www.automated-trading-system.com/walk-forward-testing/

TJ wrote:
oliveagle wrote:two questions:
1. how indicator values and trading algorithms are calculated in OOS. plz see my attachment for detail.

2. How trade is managed in OOS.
If I have a open position in the first OOS session, MC will let it open or close this position at the end of the OOS session.
If MC will keep the openning position, will it affect the calculation in next OOS session?

What is OOS?
What is IS?

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: some detail questions about WalkForwardTest

Postby JoshM » 15 Aug 2011

oliveagle wrote:two questions:
1. how indicator values and trading algorithms are calculated in OOS. plz see my attachment for detail.

2. How trade is managed in OOS.
If I have a open position in the first OOS session, MC will let it open or close this position at the end of the OOS session.
If MC will keep the openning position, will it affect the calculation in next OOS session?

Good questions. Can you elaborate on your first question? I assume that both IS and OOS use the MaxBarBack strategy setting, so if this value is set to 20 bars, the strategy in both samples will begin calculating after 20 bars.

Regarding question 2, I assume that MultiCharts closes all open trades at the end of the IS walk-forward. According to the MultiCharts 5.5 manual (page 218) MC uses the data from the IS to trade the OOS - and in my view the performance data wouldn't be complete if any open trades aren't closed at the end of time period.

It would be great to hear MC Supports opinion on this.

Regards,
Josh
These users thanked the author JoshM for the post:
oliveagle

oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

Re: some detail questions about WalkForwardTest

Postby oliveagle » 15 Aug 2011

JoshM wrote:
oliveagle wrote:two questions:
1. how indicator values and trading algorithms are calculated in OOS. plz see my attachment for detail.

2. How trade is managed in OOS.
If I have a open position in the first OOS session, MC will let it open or close this position at the end of the OOS session.
If MC will keep the openning position, will it affect the calculation in next OOS session?

Good questions. Can you elaborate on your first question? I assume that both IS and OOS use the MaxBarBack strategy setting, so if this value is set to 20 bars, the strategy in both samples will begin calculating after 20 bars.

Regarding question 2, I assume that MultiCharts closes all open trades at the end of the IS walk-forward. According to the MultiCharts 5.5 manual (page 218) MC uses the data from the IS to trade the OOS - and in my view the performance data wouldn't be complete if any open trades aren't closed at the end of time period.

It would be great to hear MC Supports opinion on this.

Regards,
Josh


Yes, MaxBarBack applies on each calculation, both IS and OOS.

I just did a Walk Forward Test(or WFT) manually trying to understand the mechanism of MC's WFT.

The whole process seems like this:
1. Load IS data (let's say 480 bars), and reference N bars back(e.g.: 240 bars), then we got 720 bars shown on chart.
2. MC starts to optimize the strategy algorithm from bar #241. The indicator value at this bar should be valid since we calculated its value from 240-(minimum ref back bars of indicator) bars ago.
3. Get the best params based on optimization rule ( Standard or customized rule)
4. Then Load OOS data(let's say 240 bars), and reference 240 bars back . Now we got 480 bars on chart.
5. Calculate strategy algorithm from bar #241 based on the best params picked at step 3.

The WFT process of MC is the same as other kind of WFT softwares. Based on the manully WFT I did, here are two things I want to mention:

Code: Select all

   1.As Josh said, both IS and OOS use the MaxBarBack strategy setting.
   2.All open positions at the end of test session will not included in Performance Report.

Here is the excel file of the test which will give u a clear picture of this problem I hope.
Attachments
WFT_checkup.xls
(28.5 KiB) Downloaded 70 times

oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

Re: some detail questions about WalkForwardTest

Postby oliveagle » 15 Aug 2011

oliveagle wrote: 2.All open positions at the end of test session will not included in Performance Report.


This introduced another question: how can I let MC closes those open positions at the end of each test session automatically? I don't like a large amount of floating either profit or loss out there if I have many OOS segments to test.

Any body knows about it? thx


Return to “MultiCharts”