Reality versus bactesting

Questions about MultiCharts and user contributed studies.
sternbergleonel
Posts: 12
Joined: 03 Oct 2018

Reality versus bactesting

Postby sternbergleonel » 10 Oct 2018

Hi

I realize that the default backtesting does not take the spread for Forex into account and that there are ways that it can be made to do so. But my question is shown on the next two uploads. One is for a backtest and the other one is for the live trading for the same currency pair and time period. Why does the backtest not record all the losing stoploss trades? Is it related to the fact that in backtesting all the data is there and it makes the best choice, while in live trading the data is sequentially analyzed?
Attachments
Live Trade.jpg
(270.11 KiB) Downloaded 772 times
Backtesting AUD_NZD.jpg
(242.55 KiB) Downloaded 772 times

User avatar
c0ntango
Posts: 70
Joined: 13 Sep 2018
Has thanked: 3 times
Been thanked: 16 times
Contact:

Re: Reality versus bactesting

Postby c0ntango » 11 Oct 2018

Hi Leonel,

I've looked at your screenshots, and here are a few thoughts I'd like to offer you:

- Those stop losses look really tiny. Based on this behavior your model has a chance of being overfit. Try developing a system without a stop loss at all. If it's profitable, you know the system has an edge and the market is likely to go in your favor when you enter. Then, if you choose to do so, you can add a "catastrophic" stop loss that can mitigate some of your risk in case of a black swan event.

- The algo seems to be "always in market" on the backtest. It will be almost impossible for you to reproduce the results it gives you in live trading because "turning points" will usually depend on the starting point with this type of system. If you want to really see what happens, you should start the backtest from the same moment in time as you're starting the live trading. Right now we are almost like comparing two different strategies.

- Drawing the indicator on both the backtest and the live trading would help to see the levels in question. If the two algos are ran at the same time, this would help with the comparison.

- The data that is chosen is not the "best" as you say. It depends on your algo - where you choose to enter and exit and how (next bar on open, this bar on close, limit orders, stop orders, etc.). MC doesn't "choose" it just executes your instructions.

- Although I've tried to point out some of the biggest issues but if you've done all the above and nothing helps, you could always try running the algo on tick data with IOG and debug from there although I don't recommend starting from here as tackling the issues above should come first.

If you need any help developing a profitable system and making sure it performs exactly as on the backtest, feel free to PM me.

-Ben

Zheka
Posts: 223
Joined: 13 Jan 2016
Has thanked: 8 times
Been thanked: 53 times

Re: Reality versus bactesting

Postby Zheka » 11 Oct 2018

First, you should compare apples to apples: you Live chart is based on Trade (is it mid-point for Oanda?), while you backtest is based on Bid.

Second, with this kind of spreads (9-10pips!?? really?):
- your backtest is "unrealistically good" by design - your buy orders are executed at much better prices (bid) than they would in reality (ask)
- you should not hope to make money with any short-term system

To see the true picture, you really need to use the "bid-ask" mode for backtesting.

User avatar
c0ntango
Posts: 70
Joined: 13 Sep 2018
Has thanked: 3 times
Been thanked: 16 times
Contact:

Re: Reality versus bactesting

Postby c0ntango » 11 Oct 2018

Zheka,

Good points - the reason I didn't bring the bid/ask problem up is two-fold,

1. I thought there are much more serious issues here just by looking at it, and
2. This is a both long & short system and even with a "long only" system I haven't found a good way to get a realistic back test on the spread (other than calculating 2x spread & slippage and entering it to transaction costs manually) because on the long side the ask price applies when you enter the market, the bid price applies when you close the position, and vice versa for the short, so your backtest will be distorted one way or another anyway. Not to mention a long/short system... Or maybe there's just setting for this I haven't discovered yet?

-Ben

sternbergleonel
Posts: 12
Joined: 03 Oct 2018

Re: Reality versus bactesting

Postby sternbergleonel » 11 Oct 2018

Thank you both for your thoughts....Lots to think about here.
Leo.

sternbergleonel
Posts: 12
Joined: 03 Oct 2018

Re: Reality versus bactesting

Postby sternbergleonel » 11 Oct 2018

Couple of thoughts
Definitely, will put a conditional statement on the strategy that if the spread is above 3 pips (it's a no go) . Does that sound right? With regards to the discrepancy: The strategy I have has intrabar trading turned on, while as I understand the back testing has only the open, close, high and low. It looks like when you have intrabar on, the backtesting seems to pick the best of the these four values. Which brings meto the general question is the backtesting useful at all when you have the intrabar trading on?

User avatar
c0ntango
Posts: 70
Joined: 13 Sep 2018
Has thanked: 3 times
Been thanked: 16 times
Contact:

Re: Reality versus bactesting

Postby c0ntango » 11 Oct 2018

It doesn't pick the "best", it picks the level you specify - as described earlier. ;)

As I described above - I think you may have many strategic and conceptional issues here. IOG should be the very last thing for you to be thinking about here.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Reality versus bactesting

Postby TJ » 11 Oct 2018

This thread might give you some reading insights.

[FAQ] Autotrade / Backtest / Optimization
viewtopic.php?t=10811

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Reality versus bactesting

Postby wilkinsw » 12 Oct 2018

Hi

I realize that the default backtesting does not take the spread for Forex into account and that there are ways that it can be made to do so. But my question is shown on the next two uploads. One is for a backtest and the other one is for the live trading for the same currency pair and time period. Why does the backtest not record all the losing stoploss trades? Is it related to the fact that in backtesting all the data is there and it makes the best choice, while in live trading the data is sequentially analyzed?
The problem looks straight forward to me:

What’s triggering your live stops is different to what’s triggering your bactested stops. Make them both “last traded” if possible. Otherwise make them both mid-price. Failing that, you’re left playing around extended backtesting mode and trying to get that to match real time stop triggers.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Reality versus bactesting

Postby wilkinsw » 12 Oct 2018

I only trade fx futures and am curious about oanda and similar......

Can a user specify what triggers a stop (bid/ask/mid price/last traded) with the broker???

if not; that sounds awful!! Trade futures instead ;)

User avatar
c0ntango
Posts: 70
Joined: 13 Sep 2018
Has thanked: 3 times
Been thanked: 16 times
Contact:

Re: Reality versus bactesting

Postby c0ntango » 20 Oct 2018

Not really. All I know is LMAX, and they only have a option to trigger the trade when the opposite order type (bid-ask) was hit, but in case of intraday strategies keep the order from being triggered at least 50-80% of the time so it's not such a great feature to use. That's as close as I've seen it get.


Return to “MultiCharts”