Walk Forward Optimization in Trading Algorithms

Questions about MultiCharts and user contributed studies.
flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Walk Forward Optimization in Trading Algorithms

Postby flipflopper » 22 Mar 2010

So we all know markets cycle. Identifying what cycle (RTM or Trending) we are in and using appropriate trading strategies is one part of the puzzle.

The other piece to the trading puzzle is if you can you improve real time results by incorporating periodic walk forward optimization in your trading algorithm that tries to adjust parameters based on these identifiable market cycles.

In its simplest terms I want my trading strategies to automatically re-optimize my parameters periodically based on real time data.

Does anyone know of any books or studies done on this topic? This is something I have never heard discussed.

glen demarco
Posts: 35
Joined: 16 Nov 2009
Contact:

Postby glen demarco » 22 Mar 2010

flipflopper,

If you search the internet there are several books and numerous web sites. One well know book that comes to mind is Evidence Based Technical Analysis by David Aronson. What your looking for is exactly what the Multicharts Walk Forward optimizer already does.

Hope this helps,


Gleb

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 22 Mar 2010

Solutions for this frequently involve either evaluating a number of systems in parallel and "hopping" from one equity curve to another based on recent performance (e.g. switching to the most upwardly sloping curve over X bars or any number of similar schemes), voting amongst those systems with currently "good enough" performance, etc. In EasyLanguage alone, the solutions are non-trivial, but it can and has been done any number of times.

That's the objective approach based on recent hindsight performance e.g. a re-optimization. The adaptive approach that isn't reoptimization per se would be to measure something such as length of trend etc. and factor that into the strategy itself. In this way you wouldn't directly be measuring equity curve / performance statistics, but you would be measuring the underlying instrument's statistic that you care about such as whether it's in a trend and to what extent, then taking this into account in the strategy itself - for instance by turning off trading under certain conditions, or by varying the length of the moving averages / oscillators / etc. accordingly.

Both of these broad categories of approaches are essentially what I would call meta-strategies - for instance a meta-strategy might be "turn off my strategy if X happens", or "trade the best of these 5 according to X measure", or "reoptimize every X bars including Y bars each time in-sample". They're an attempt to wrap another layer around the strategy itself, in order to make it more robust - holding up longer out-of-sample without having to be taken off-line manually, or manually re-optimized.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 23 Mar 2010

flipflopper,

If you search the internet there are several books and numerous web sites. One well know book that comes to mind is Evidence Based Technical Analysis by David Aronson. What your looking for is exactly what the Multicharts Walk Forward optimizer already does.

Hope this helps,


Gleb


Hi Glen,

Yes Multicharts does do walk forward optimization. What I am trying to accomplish is incorporating this into the actual trading algorithm. Thus my goal is to identify HOW OFTEN to optimize my inputs.

I know this sounds like horrible curve fitting but in my mind I feel it may be one of the keys to the markets.

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 23 Mar 2010

A third party optimizer's cluster analysis feature is normally used to identify the optimum in sample period, out of sample period (if any) and reoptimization interval for maximum robustness, or in other words, percentage of profitability maintained out of sample after reoptimization.

This works by repeatedly doing a series of walk forward optimizations, using different in sample periods and at different intervals, to see what results in the best outcomes.

Like any other repeat test, it is a possibility to have data mining bias or overfitting take place, but other than "rules of thumb" it's about the only reasonable way to figure out when you should reoptimize in a methodical, empirical manner.
Last edited by Bruce DeVault on 23 Mar 2010, edited 1 time in total.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 23 Mar 2010


"reoptimize every X bars including Y bars each time in-sample".


Great post.

The above is exactly what I'm talking about. Are there any books where I can study any of the concepts you mentioned?

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 23 Mar 2010

A third party optimizer's cluster analysis feature is normally used to identify the optimum in sample period, out of sample period (if any) and reoptimization interval for maximum robustness, or in other words, percentage of profitability maintained out of sample after reoptimization.

This works by repeatedly doing a series of walk forward optimizations, using different in sample periods and at different intervals, to see what results in the best outcomes.

Like any other repeat test, it is a possibility to have data mining bias or overfitting take place, but other than "rules of thumb" it's about the only reasonable way to figure out when you should reoptimize in a methodical, empirical manner.


Now we're talking. Any suggestions?

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 23 Mar 2010

Probably the easiest way to do this "off the shelf" is using the Grail System (http://www.thegrailsystem.com/). It's based loosely on Robert Pardo's books, including Evaluation and Optimization of Trading Systems, a book to which I was a contributor of some of the optimization techniques material and was also a pre-release reviewer for content. Quantevo is also an authorized reseller for the Grail products, and we've done quite a lot of consulting work about these types of techniques (going back to about 2005 with the Grail approach specifically as an example, and with other similar methodologies before that) and mechanisms for assuring system robustness, including building custom optimizers which have the types of functionalities you're describing on a variety of trading platforms. Building a custom optimizer is usually not warranted for individual traders, but if you're interested in an off-the-shelf product that can do a cluster analysis like I described above e.g. tell you basically how frequently it would be best if you re-optimized, I would say the Grail GWFO is probably the easiest way to go, and you're welcome to email me outside of the forum if you'd like more information on this or more specifically tailored advice.

wegi
Posts: 111
Joined: 02 Jun 2009
Has thanked: 3 times
Been thanked: 12 times

Postby wegi » 26 Mar 2010

There was an article last year in the magazin "Traders" (german), with the Titel: "Self adative System" from Clayburg.
He discribes one method, which works with tradestions. The system optimzes the defined inputs and change to an other input, if they work better. This work in backtests and in real trading modus.
If i understood the article right, you can define which inputs should be optimized, e.g. sma length from 10 - 20. And you can define some criteria when the system should change to an other sma length.

Please google for clayburg adaptive system, and you will find a link with the sourcecode for TS at boerse-direct de

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 26 Mar 2010

Yes Clayburg's work tends to be in this category of tracking virtual equity curves and curve hopping - he's had some ideas out there either for free or for a small fee for about 5 years now, and written a few articles over the years about these same concepts. They're talked about extensively on his website, which is clayburg.com. I believe he may still be giving away the EL source code to some of it from his 5 years or so ago presentation at one of the Expos on his website.

Clayburg's stuff isn't a "general solution" like an external optimizer can be in that it doesn't answer the questions like how often to reoptimize etc. for any given strategy without extensive modifications/rewriting, but it's an example of how you can make a strategy that does the curve hopping, if you design it that way from the ground up and are willing to take that into account in the design.

There's a danger of some degree of "chasing your tail" also if you use parallel functions, in that if the "best" results aren't stable enough, by the time you switch over, it's too late and they're already past their prime, so when this approach doesn't lend itself well, that tends to be the reason why.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 30 Mar 2010

Bruce,

The most valuable thing one could sell would be Easy language source code for self optimizing a set of inputs.

That way one could include that in the algorithm... chose the in sample and out of sample periods and let it go!

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 31 Mar 2010

There are some key architectural reasons why EasyLanguage/PowerLanguage doesn't make this trivial to do, and therefore it can't just be a quick "plug-in" that works generically with any EL strategy. It can be done, but it requires that the strategy be built with this in mind, rather than just something that layers on top of anything else.

You also have to beware of "tail-chasing" - in some cases, ironically, too frequent re-optimization may actually decrease performance.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 31 Mar 2010

The tail chasing comment might be true... but in this case your historical results have the highest chance of becoming a reality.

This might seem like double tail chasing but theoretically you optimize the in sample and out of sample data series time frames. Which is eventually what I would want to end up doing.

Again all this probably won't make a HUGE difference in the bottom line but would be interesting to explore.

Do you know which software makes it easiest to create self optimizing strategies?


Return to “MultiCharts”