Faulty programming in backtesting/optimization?

Questions about MultiCharts and user contributed studies.
moses
Posts: 54
Joined: 16 Nov 2012
Has thanked: 34 times
Been thanked: 7 times

Faulty programming in backtesting/optimization?

Postby moses » 17 Sep 2016

Hello,

I have the following observation:

Let's say I am performing a backtest on daily data for the past 10 years, and the strategy returns 100 trades. This takes less than 2 seconds for my PC to compute.

Now, if I check the 'use bar magnifier' (1 minute data), the test takes much longer to complete.
This effect is much more pronounced on optimisations, where NOT using bar magnifier completes the tests thousands of times faster!

Now, the problem I see is that, for a 100 trade test, the program should only find the entry point of those trades (you only need the daily data to do this, so it should be very fast), and then only for those 100 trades use the 1-minute data to compute exits etc.
But the time it takes to calculate using bar magnifier, makes me believe that the program goes and looks at minute data continuously for some reason.
Can someone confirm, and maybe we can ask for a code change?

Thank you
Andreas

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Faulty programming in backtesting/optimization?

Postby TJ » 17 Sep 2016

Hello,

I have the following observation:

Let's say I am performing a backtest on daily data for the past 10 years, and the strategy returns 100 trades. This takes less than 2 seconds for my PC to compute.

Now, if I check the 'use bar magnifier' (1 minute data), the test takes much longer to complete.
This effect is much more pronounced on optimisations, where NOT using bar magnifier completes the tests thousands of times faster!

Now, the problem I see is that, for a 100 trade test, the program should only find the entry point of those trades (you only need the daily data to do this, so it should be very fast), and then only for those 100 trades use the 1-minute data to compute exits etc.
But the time it takes to calculate using bar magnifier, makes me believe that the program goes and looks at minute data continuously for some reason.
Can someone confirm, and maybe we can ask for a code change?

Thank you
Andreas

Every test is a new test.

Every change in parameters is a new test.

moses
Posts: 54
Joined: 16 Nov 2012
Has thanked: 34 times
Been thanked: 7 times

Re: Faulty programming in backtesting/optimization?

Postby moses » 17 Sep 2016

I think I didn't explain well.

What I am saying is that a test on, say, daily data (with 'use bar magnifier' ON to 1 minute) should look for entries using ONLY the daily data if that is all that is needed, and use the 1-minute data only when necessary, eg for exits.

The times of my tests show that with bar magnifier on, the software goes around somehow and uses the 1-minute data for EVERY daily bar, even if not one in a trade, and this, again somehow, ends up creating huge testing times.

tony
Posts: 420
Joined: 14 Jun 2013
Has thanked: 30 times
Been thanked: 81 times
Contact:

Re: Faulty programming in backtesting/optimization?

Postby tony » 17 Sep 2016

If you are not using IOG then you will calculate on bar close. On a daily chart that's about 250 calculations for one year. Now if you use IOG mode and no bar magnifier now you are calculating on OHLC or 4 times per bar for 1,000 calculations for one year. Now turn on both IOG and bar magnifier and you just went exponential on the number of calculations you are asking your machine to perform.

So it sounds like you need to decide whether you want to trade intrabar and thus use IOG=true or only at bar close and thus use IOG=False.

moses
Posts: 54
Joined: 16 Nov 2012
Has thanked: 34 times
Been thanked: 7 times

Re: Faulty programming in backtesting/optimization?

Postby moses » 19 Sep 2016

I am not using IOG (but I am using bar magnifier) - but even on a daily data test that enters only on the close of the bar, the difference in calculation time between using bar magnifier and not using it, is huge.
What I am saying is, the program should only use the bar magnifier data on the exits, thereby saving a lot of processing time.

tony
Posts: 420
Joined: 14 Jun 2013
Has thanked: 30 times
Been thanked: 81 times
Contact:

Re: Faulty programming in backtesting/optimization?

Postby tony » 19 Sep 2016

What I am saying is, the program should only use the bar magnifier data on the exits, thereby saving a lot of processing time.
So you are asking MC to perform a script calculation on every possible tick which exponentially increases the calculations and thus adds time. Additionally when you use bar magnifier, especially during market hours of the instrument traded (I believe) you will notice longer time to collection bar magnifier data from your data provider.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: Faulty programming in backtesting/optimization?

Postby evdl » 19 Sep 2016

If I understand moses, he enters trades on daily basis and exits are done during the day (= intraday). Not every day a trade occurs and he probably suggest, not to use the intraday calculation for the days there is no trade. Saving backtesting time on those days.

In theory this would speed up the backtesting a lot. And I think for entries, most of the time barclose entry would suffice on any timeframe. But for exits with stoplosses it is nice to have the barmagnifier to know if your stoploss or profit target is hit intrabar.

The way MC is backtesting now, it will have to change significantly for this I guess. And that is maybe not feasible. Also because is depends a lot on the strategy someone is using.

With barstatus you can speed up your code. There is no way at this moment to my knowledge, to use barmagnifier or IOG on only certain parts of your code. If that would be possible, you can code it yourself. But that is not the case at the moment.
Last edited by evdl on 19 Sep 2016, edited 1 time in total.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Faulty programming in backtesting/optimization?

Postby TJ » 19 Sep 2016

A lot is dependent on your code.
If your code is not efficient.
eg. if it calls for analysis even if you have no position opened,
then whether you have a position or not is a moot point.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: Faulty programming in backtesting/optimization?

Postby evdl » 19 Sep 2016

A lot is dependent on your code.
If your code is not efficient.
eg. if it calls for analysis even if you have no position opened,
then whether you have a position or not is a moot point.
If I'm not mistaken, even if your code is efficient and appropriate use of barstatus, it will always use the Barmagnifier resolution to process all the data on the chart. Which will cause extra backtesting time. But I could be wrong.

tony
Posts: 420
Joined: 14 Jun 2013
Has thanked: 30 times
Been thanked: 81 times
Contact:

Re: Faulty programming in backtesting/optimization?

Postby tony » 19 Sep 2016

A lot is dependent on your code.
If your code is not efficient.
eg. if it calls for analysis even if you have no position opened,
then whether you have a position or not is a moot point.
If I'm not mistaken, even if your code is efficient and appropriate use of barstatus, it will always use the Barmagnifier resolution to process all the data on the chart. Which will cause extra backtesting time. But I could be wrong.
I would agree. Take an instrument like ES. Without bar magnifier a simple one minute bar has 4 calculations, OHLC. Now turn on bar magnifier and you like have 50 or 60 calculations, a 15-fold increase. Seems to me that is where the source of increased testing time would come from versus an inefficient or redundant script. Though that can add to it.


Return to “MultiCharts”