[Backtest] Limit fill behaviour very wrong  [SOLVED]

Questions about MultiCharts and user contributed studies.
wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

[Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 29 Mar 2018

Hi,

I believe it was MC10 where the limit fill price improvement bug was fixed.
viewtopic.php?f=1&t=32734&p=103664&hili ... nt#p103664
There are examples where price improvement does happen though in real trading and these are not currently reflected in the backtest engine.

Please, please, please can MC fix this ASAP as it represents a huge bug as I'll demonstrate below............


For example:

If I have one instruction "order1" to buy next bar at price=100 limit and another one "order2" to buy next bar at 100 limit (same price by coincidence).

Let's say "order1" has a 10 tick stop and "order2" also has a 10 tick stop.........

Then, if my signal is setup to only trade one position at a time.......

Then if next bar I get filled long at 100. Then, if I'm stopped at at 90...........

"Order2" can now trigger (same bar is possible).......

Question: So, where will "order2" get filled??????????

Answer Real life: at the best offer. Correct behaviour in Classic backtesting mode: at 90.

Current MC11 behaviour: "Order2" filled at 100.

Current MC9 behaviour: "Order2" filled at 90.

MC, please understand that when a new limit order is sent.... if the current price is better than the limit order price then price improvement WILL happen.

This MC11 behaviour actually has huge ramifications as anyone running multiple buy/sell orders within one signal whilst only allowing a maximum of one position will be potentially getting very significantly penalised!

MC9 behaviour (correct):
mc9.PNG
(29.91 KiB) Downloaded 1753 times
MC10 behaviour (very wrong):
mc10.PNG
(10.9 KiB) Downloaded 1753 times
MC10 wrong behaviour continued:
mc10 bad fill assumption.PNG
(6.91 KiB) Downloaded 1743 times
https://www.multicharts.com/pm/public/m ... es/MC-2422

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: [Backtest] Limit fill behaviour very wrong

Postby Henry MultiСharts » 29 Mar 2018

Hello wilkinsw,

Please send me the requested data for replicating this case.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 03 Apr 2018

Hi Henry,

Just sent you everything you will need (hopefully).

Also an additional folder to replicate another bug where we get filled at a worse price than real world, following an open/close gap (viewtopic.php?f=1&t=50228&p=123738#p123709).

Thanks!

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: [Backtest] Limit fill behaviour very wrong

Postby Henry MultiСharts » 14 May 2018

Hello wilkinsw,

Both issues have been confirmed and are targeted to MultiCharts 12 Release.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 29 Jun 2018

Hi Henry,

Has this now been fixed for MC12?

Thanks,

Will

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: [Backtest] Limit fill behaviour very wrong

Postby Henry MultiСharts » 02 Jul 2018

Hello Will,

The fix is available in MultiCharts 12 Release Candidate.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 03 Jul 2018

Hi Henry,

I've noticed that the solution for MC12 has indeed corrected limit fill placement but unfortunately now incorrectly places the fills for stops.

You can reproduce using the exact same files I sent you before and comparing MC12 to MC11 and early MC9.

What was fixed in MC12:
The completely bogus limit order fill worsening has now been fixed in MC12: If upon exiting a long position, a subsequent secondary signal (same bar) sends a marketable buy limit order, the fill is placed against the likely prevailing price...ie the same price as the stop fill that exited the primary long position.

What bug was added in MC12:
Once our secondary buy marketable limit is filled...IF our stoploss is immediately elected (even though price hasn't moved) then: this secondary stop exit fill is NOT placed against the likely prevailing price (e.g. the same price as the entry) but instead is placed against the next available datapoint.

"Next available datapoint" would only approach being accurate if the user had access to tick history and was using Bar Magnifier "BM". If, however only using 1min BM or NO BM....."next available datapoint" can be far removed from anything approaching an accurate model.

This erroneous placement of secondary stop exit fills was not an issue in previous versions so I'm hoping this will be a quick and easy fix?

This new bug adds false slippage just like the original false limit fill placement did.


Correct MC9 fills:
correct behaviour as per MC9.PNG
(220.46 KiB) Downloaded 1529 times
MC12 BM OFF: MC12 BM ON(1min):

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: [Backtest] Limit fill behaviour very wrong

Postby Henry MultiСharts » 05 Jul 2018

Hello wilkinsw,

Please send me the data for replicating this behavior.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 05 Jul 2018

You already have it Henry. I'll send you the link again. All you have to do is to open the workspace in MC12 and compare to MC11 (and MC9).

Thanks,

Will

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 17 Jul 2018

So........

I spent over an hour on teamviewer with Henry and MC and I clearly identified for them the new backtest stop fill placement bug.

To summarize, a second postion gets opened intrabar, in the event that the stop for said position is immediately executable.... it won't get placed against the prevailing price. MC will instead wait for the next datapoint and mark the stop fill against that price point.


"next datapoint":

If not using BM "next datapoint" will be the LOW of the bar. That is frankly mental behaviour, as in real life A) one would not wait a prolonged period of time to submit and triggered stop and B) They would be highly unlikely to get slippage in the form of the LOW EVERY TIME!!!!!!!!!!!!!!.

Therefore the new behaviour ONLY works if users backtest with BM=ON and tick level granularity. Otherwise it is wrong and deranged!!

Users: how does that sound???

Speaking for myself; in MC I use 1min BM over 10years+ of data. I have about 6 signals within one strategy and MC12 inserts alot of error on stop fill placement.

In portfolio backtesting, with no BM?! OMG. Can you imagine?! Bar the primary position intrabar....all subsequent positions, if they have a stop triggered, will have it placed at the WORST POSSIBLE PRICE IN THAT BAR. Sorry to use Caps and bold but I can't believe I've got to a point where MC are simply ignoring me and persisting with the deployment of a buggy MC12.

Please, please, please MC go back to the drawing board and revert fill behaviour to early MC9 BUT with the limit price improvement bug removed (that MAtricks pioneered).

Thanks!!!!

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 17 Jul 2018

Further info......

Some of you might be thinking "why on earth does he have multiple entries intrabar"?

Fact is that is rare. But it is possible nonetheless. The simple goals is to be able to accurately model where possible. For some reason MC's developers have actively inserted inaccuracy in MC12. It is horrifying to know that that means intrabar stop fill placement at the worst possible price for anything other that primary stop triggers.


Examples:

These examples are illustrative only just to prove a point. Pleas be aware that MC think this is correct and accurate behaviour, so I'll need support from other users if we are to change their mind.

Primary entry with immediate stop trigger, MC12,11,9:
single entry and stop.PNG
(34.45 KiB) Downloaded 1462 times
Secondary entry included with immediate stop trigger, BM OFF. MC9:
multi entry 1 mc9.PNG
(46.98 KiB) Downloaded 1462 times
multi entry 2 mc9.PNG
(55.96 KiB) Downloaded 1462 times
MC9 conclusion:
For illustrative purposes only. In the absence of bid/ask data, this is the closest to real life we can expect.
-We get long
-Our stop is immediately triggered (for most users that will be next tick...so pseudo immediately). That fill in most case, will be somewhere between the entry price and the next worse price.
-We are flat, and so permitted to submit our secondary entry order (same price for illustrative purposes)(pyramiding is off). Again I'd expect up to a tick of slippage in most cases on that order
-Our secondary stop is the same price as the primary stop for illustrative purposes, and so psuedo-immediately submitted. Again, I'll want to factor in some slippage costs into strategy properties to account for occassional slippage here too.
-To conclude...best model is all fills placed at same price with slippage factored into costs

MC12:
multi entry 1 mc12.PNG
(12.2 KiB) Downloaded 1462 times
multi entry 2 mc12.PNG
(29.93 KiB) Downloaded 1462 times
Secondary stop placed against the low! literally what is going on?! No more questions your honour. Please get this fixed. MC you need to rollback. It can't be that difficult to see the error, really?! MC's a great platform but lets not get in the habit of baking in bad architecture into new releases PLEASE!


User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: [Backtest] Limit fill behaviour very wrong  [SOLVED]

Postby Henry MultiСharts » 26 Jul 2018

MC12:
multi entry 1 mc12.PNG
multi entry 2 mc12.PNG
Secondary stop placed against the low! literally what is going on?! No more questions your honour. Please get this fixed. MC you need to rollback. It can't be that difficult to see the error, really?! MC's a great platform but lets not get in the habit of baking in bad architecture into new releases PLEASE!
This issue will be fixed in MultiCharts 12 Release.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: [Backtest] Limit fill behaviour very wrong

Postby wilkinsw » 26 Jul 2018

Thanks Henry.

You guys are the best!


Return to “MultiCharts”