Backtest vs Real exotic phenomenon

Questions about MultiCharts and user contributed studies.
Yohyon
Posts: 10
Joined: 03 Dec 2018
Has thanked: 1 time

Backtest vs Real exotic phenomenon

Postby Yohyon » 28 Feb 2021

Hi, I have studied all the material on differences between RT and BT and I understand that the match will never be perfect, especially with limitprices. Nevertheless, I run into a very strange phenomonen that is not related to anything I have read about this subject:

1. I run a backtest on tickdata from IQfeed with 30 days of history.
2. I close all positions before the close of the day every day. My strategy trades about 20-30 times per day, always the same symbol.
3. I run the same backtest with 5 days of history from the same source and the same interval
4. I compare the last backtest with the last 5 days of the 30 day machtest and I get a completely different outcome...

Does anyone notice similar differences? Any idea on the rootcause?

Thanks.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Backtest vs Real exotic phenomenon

Postby TJ » 28 Feb 2021

What is your chart resolution?

How many bars do you have on the chart?

Tick chart?
Take a look at the bars -- are they the same?

Yohyon
Posts: 10
Joined: 03 Dec 2018
Has thanked: 1 time

Re: Backtest vs Real exotic phenomenon

Postby Yohyon » 28 Feb 2021

It is 1.000 ticks from IQFeed. You are right. I should have checked that, but the charts are not identical.. I guess I have to check with IQFeed. Thank you and sorry for bothering you with a non MC related issue.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Backtest vs Real exotic phenomenon

Postby TJ » 28 Feb 2021

It is 1.000 ticks from IQFeed. You are right. I should have checked that, but the charts are not identical.. I guess I have to check with IQFeed. Thank you and sorry for bothering you with a non MC related issue.
No problem.

The bars between the charts are not identical because the starting tick is different.
The 1,001st tick will arrive at a different time/price, and might start a new bar at a different time/price.

Yohyon
Posts: 10
Joined: 03 Dec 2018
Has thanked: 1 time

Re: Backtest vs Real exotic phenomenon

Postby Yohyon » 28 Feb 2021

Thank you TJ. I thought of that as a possible explanation once you pointed me in this direction. Is there any way to prevent that? In the settings or so? I think this issue is causing a significant difference between real time and backtest. Not necessarily in terms of long term returns, but in day-to-day trade-list.

I would like to compare BT with RT after end of session, or is that useless anyhow? In the past I used AB and NT. BT vs RT comparison was close. But that was with stocks. There was some slippage, but basically the same trades, but now I trade ES and I have significant differences between RT and BT. Frequently completely different trades. It is not caused by LMT prices. I send long orders @C+.25 LMT and short @C-.25 and they do get filled.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Backtest vs Real exotic phenomenon

Postby TJ » 28 Feb 2021

The only repeatable BT are on the time-based charts.

Yohyon
Posts: 10
Joined: 03 Dec 2018
Has thanked: 1 time

Re: Backtest vs Real exotic phenomenon

Postby Yohyon » 01 Mar 2021

I understand. Thank you. I will focus on comparable performance then.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Backtest vs Real exotic phenomenon

Postby Mydesign » 01 Mar 2021

Hi,

Just a hint: make sure your Data Range in Instrument settings is set as "From Date to Date" instead of "x Days/Bars back". That way you will always get same bars construction and thus same backtest results.

tpredictor
Posts: 108
Joined: 05 Mar 2014
Has thanked: 1 time
Been thanked: 12 times

Re: Backtest vs Real exotic phenomenon

Postby tpredictor » 17 Mar 2021

When you say only "time based"-- I have backtested range bars and got the exact results in live vs backtest in NT except for 1 tick difference in live (NT gave 1 tick avg fill improvement over live). Is there a good resource on what bars we can backtest on and the limitations?

User avatar
joebone
Posts: 175
Joined: 05 Sep 2018
Has thanked: 53 times
Been thanked: 4 times

Re: Backtest vs Real exotic phenomenon

Postby joebone » 18 Mar 2021

using the "break on session" ability should restart the tick chart at session break to. That should help you standardize tick charts over longer periods..... At least i think so....


Return to “MultiCharts”