Backtest Limit Orders with incorrect fills  [SOLVED]

Questions about MultiCharts and user contributed studies.
User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Backtest Limit Orders with incorrect fills

Postby MAtricks » 08 Jul 2014

Let's try another approach...

•1 tick Point bar chart to make this example as easy as possible
•1 tick Profit target with a random entry signal

In a perfect world things would go like this:

Image

Now I change the setting to be realistic with my limit order fills:

Image

Notice that once we chose the more realistic feature, the profit target is now filling with TWO ticks instead of one?

Why does this setting change the simulated fill?! It should verify that price penetrated through it by 1 tick, not to fill at 1 more tick. The executed price of a limit order should never change! (even if there is a gap in historical price, the live price WOULD fill at your order and you would fill the gap)

So as it is, Multicharts has two options to test limit orders: 1. Execute if price reaches the order. or 2. Execute at the price + an improvement of profit of the inputted tick amount. Both of which create a highly unrealistic back-test. Many users of this platform back this up and are still PMing me about how they're so surprised that nothing was done about this crucial problem I've been reporting for so long.

Back-testing with limit orders in Multicharts is not safe until this bug is fixed.
Attachments
With.png
(64.31 KiB) Downloaded 5996 times
Without.png
(67.34 KiB) Downloaded 6000 times

zammo
Posts: 14
Joined: 28 May 2014
Has thanked: 3 times
Been thanked: 1 time

Re: Limit Orders with incorrect fills

Postby zammo » 23 Jul 2014

I agree entirely that this issue needs sorting out.

To have a limit order fill at a better rate in a backtest is entirely counter productive.... This doesnt often happen in the real world so it shouldn't ever happen in testing software....

I was shocked to see a strategy I have written being filled a tick better on about 50% of trades!

Multicharts, if you want to keep this feature as is, thats fine... But you need to add a feature for getting filled at the limit price you choose and not beyond..... Even if the 'data' has traded past the price.

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Re: Limit Orders with incorrect fills

Postby MAtricks » 02 Aug 2014

Is there a fix for this bug in an upcoming version? To be able to back-test limit orders correctly is much more than a trivial request.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: Limit Orders with incorrect fills

Postby Andrew MultiCharts » 26 Aug 2014

Hello,

The price checking doesn't go over next bar (let's name it bar X), so if there is no such price outside the next bar (X), the order will not be filled on that bar (X). On the next bar (X+1), the checking find the price and executes the order on this bar (X+1). It cannot go back and execute the order on the previous bar (X).

With current architecture it is possible to change something in the software not to move the order execution price in such cases, but the order will not be filled at the next bar (X) as you expect, it will be filled on the same bar as it is filled right now (X+1), but with the correct price. In some other cases it will look like an order filled outside of a bar. Is that what you are looking for?

Please see the attached picture (zooming in may be required) to see the bar and the price where the order would be executed if we implement the improvement.
Image

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Re: Limit Orders with incorrect fills

Postby MAtricks » 26 Aug 2014

Price IS there regardless if a 'bar' isn't - bars don't define price, price defines the bars. If MC is creating the bars incorrectly, that's an issue to look into. If not, then something is wrong with the order placement in back-tested scenarios which needs to be looked at. I wish I could spend the time on how this should be implemented... I cannot. I've burned way too much time into this topic in the past two years.

MC back-tests on these orders are NOT following how it actually places them. Run a demo strat and match it up with the back-test. A back-test should be identical to a demo placement test.

If my strategy places an order to buy at 100 then my live order would most definitely buy at 100. Why would I want my back-tests to show the buy at 99 when my live orders would definitely buy at 100?

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: Limit Orders with incorrect fills

Postby Andrew MultiCharts » 26 Aug 2014

Limit order assumptions do not check prices outside current bar. This is not an issue, this is expected behavior. What is demonstrated by you in this is thread is not a bug in the software, this is how it is designed. If you would like us to improve it, I described how it can be done in my previous post. Please make sure you understand advantages and disadvantages of that possible improvement and let me know if this suits you.

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Re: Limit Orders with incorrect fills

Postby MAtricks » 26 Aug 2014

Working as built is not always working as it should.

All I want is for the back-tests to be a back-tests. ie: emulate what would happen in live trading. However this is done is up to MC.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: Limit Orders with incorrect fills

Postby Andrew MultiCharts » 26 Aug 2014

Unfortunately sometimes it is simply not possible to change something without re-writing half of the software. I understand you want as realistic backtesting as possible, but at the moment we cannot do what you want 100% as you as you expect it work, though i explained what kind of improvements are possible in current software architecture.

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Re: Limit Orders with incorrect fills

Postby MAtricks » 26 Aug 2014

Ah... Sounds like a big oopsie which explains the past replies I've received on my requests for realistic back-tests. :)

So your suggested patch:

It sounds like it will fill tested orders on the correct price, but it will fill on the incorrect bar? What problems would we likely run into with this being implemented?

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 27 Aug 2014

Hi Andrew,
In some other cases it will look like an order filled outside of a bar. Is that what you are looking for?
For me, yes exactly that.

Using your picture with the yellow arrows as the example......pretend the time/price point that the arrows point to didn't happen. Instead the bar opened a tick higher and continued moving up (leaving a tick gap). In this example, yes, MC would need to show the fill a tick below the bar in question at the correct limit order price (outside of the bar).

In live trading the chart in the above example would most likely still have had a gap, but you would have been filled by the end of the bar before. So the tick higher open is confirmation that you would've got the fill (for the sake of historical simulation) ...... it's then up to MC where to place the fill......in the gap below the open or in hindsight on the bar before (i think in the gap is more logical).

If the example was a 10 price gap with the target limit order square in the middle of the gap, then again you'd have to place the limit fill at this target price in the middle of the gap.

I invite people to pull apart this proposition.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: Limit Orders with incorrect fills

Postby Andrew MultiCharts » 27 Aug 2014

It sounds like it will fill tested orders on the correct price, but it will fill on the incorrect bar? What problems would we likely run into with this being implemented?
If an entry took place not at X bar, but at X+1, all further strategy behavior can change (for example if the profit target was supposed to be executed at bar X+1, but it is issued only at X+2 and the prices on the chart moved away from the profit target). It all depends on how your strategy is written.

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 288 times

Re: Limit Orders with incorrect fills

Postby MAtricks » 27 Aug 2014

It sounds like it will fill tested orders on the correct price, but it will fill on the incorrect bar? What problems would we likely run into with this being implemented?
If an entry took place not at X bar, but at X+1, all further strategy behavior can change (for example if the profit target was supposed to be executed at bar X+1, but it is issued only at X+2 and the prices on the chart moved away from the profit target). It all depends on how your strategy is written.
I believe that this could be easily worked around whereas the current model is impossible to workaround. I'm in favor of this..

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 28 Aug 2014

I'm in favour too.
If an entry took place not at X bar, but at X+1, all further strategy behavior can change (for example if the profit target was supposed to be executed at bar X+1, but it is issued only at X+2 and the prices on the chart moved away from the profit target). It all depends on how your strategy is written.
The fact is you don't know if your initial target order would've been filled on Bar X or bar X+1 when applying the above examples to live trading. You'd think chances are that it'd be bar X. However there is also a chance that the effect of you order would have held the price there a little longer resulting in a fill at the start of X+1 (meaning no gap at the open of X+1). There is no practical way of knowing in hindsight as to whether placing the target fill on X or X+1 is correct.

So in live trading an X+1 target fill is certainly possible, using the above examples as shown by Andrew and MAtricks. Therefore, if other downstream orders and trades are effected by the X+1 fill then at least this is something that could/would happen in live trading anyway.

My point: These incorrect price improvements don't happen in live trading. X+1 target fills will happen fairly often in live trading. Downstream orders being impacted by X+1 target fills will also be effected often in live trading. Will X+1 target fills be more frequent in historical simulation than live trading? Certainly, but with far less of an impact and with possible workarounds compared to limit order price improvement, which never happens in live trading**.

**the only exception is an end of bar new limit order being generated that can get filled at a better price if the open of the next bar presents a price improvement.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: Limit Orders with incorrect fills  [SOLVED]

Postby Andrew MultiCharts » 12 Sep 2014

The behavior of the feature "limit order assumption (when price goes beyond X points) for backtesting" will be improved in future version of MultiCharts. Please keep track of the following feature requests:

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 16 Jul 2015

I've just realised that there is another scenario where price improvement IS possible with a limit order...........

At the session open!

If I have a buy limit order working over the weekend, for example, and the open on Sunday (Globex) evening gaps 100 prices below my limit order.....

Then I get filled at the open, not at the limit price!

It is very important that the new rules in MC Beta take this into account.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 07 Aug 2015

I've just realised that there is another scenario where price improvement IS possible with a limit order...........

At the session open!

If I have a buy limit order working over the weekend, for example, and the open on Sunday (Globex) evening gaps 100 prices below my limit order.....

Then I get filled at the open, not at the limit price!

It is very important that the new rules in MC Beta take this into account.
The order will be filled at limit price in MultiCharts 9.1 Beta 1 when you set "Fill limit order when trade price goes beyond limit price by 0 points".

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 07 Aug 2015

I'm saying it shouldn't be filled at the limit price in this situation.

The open/session restarts is a time where price improvement DOES occur if the open is at a better price than ones limit price. You literally get matched at the opening price.

If the new MC beta is now filling limit orders only at the limit price (which is correct in other scenarios, not this one), then this will need changing.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 07 Aug 2015

I'm saying it shouldn't be filled at the limit price in this situation.

The open/session restarts is a time where price improvement DOES occur if the open is at a better price than ones limit price. You literally get matched at the opening price.

If the new MC beta is now filling limit orders only at the limit price (which is correct in other scenarios, not this one), then this will need changing.
In this case you need to disable the limit order assumption.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 07 Aug 2015

I might be misunderstanding you.

In the following scenario what happens in MC beta:

I'm GTC limit bid at price=1200.
Market closes at price=1201. I'm not filled (under any assumption).
Market opens at price=1100.

So at what price does MC fill me under the two options:

1) "Fill limit order when trade price goes beyond limit price by 0 points"?
2) "Fill limit order when trade price goes beyond limit price by 1 points"?

They should both be the same in any event for this situation. Ie fill price=1100. Can you clarify for me please.

My point is simply, when "Fill limit order when trade price goes beyond limit price by 1 points"=true, I should not see price a improvement once MC has checked that limit price has been exceeded and fill conditions met. AND with this assumption in place I CAN still see an improvement fill price in the example set out above because this IS the CORRECT behaviour.

You stated that I would only be filled at my limit price in the close/open gap example above. Under ALL circumstances this is wrong. Forgive me if I've misunderstood.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 10 Aug 2015

wilkinsw,

Both of your cases will be filled the same way - when limit order assumption is enabled the limit order is filled without price improvement at limit price.
When limit order assumption is disabled the limit order can be filled with price improvement at limit price or better.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 10 Aug 2015

Well that is simply incorrect behavior and needs to be changed.

Session close/open gaps through limit prices need to be treated differently to

intraday close/open gaps.

This beta solution to improving the behavior of limit order fills (as introduced by "MAtricks") therefore is currently wrong.

The current beta solution (to fixing false price improvements when fill assumption is ON) is only correct if NOT carrying positions overnight.

Can MC change this please?

Thanks.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

MULTICHARTS 9.1 BETA 1

Postby wilkinsw » 11 Aug 2015

I've chosen to post here to make sure other users can give feedback on an issue on the new fills assumptions as described here,http://www.multicharts.com/discussion/v ... 4&#p108164 that are in the 9.1 beta version:

The new fill assumptions addressed a flaw when selecting:"Fill limit order when trade price goes beyond limit price by 1 points" in previous versions. Previously, price improvement could occur as MC could only place a fill where there was a trade on the bar following confirmation that the limit price had been exceeded. Sorry if that sentence is long and confusing.

Of course in real life you get filled where your limit price is no matter what. Therefore MC has chosen, quite rightly, to address this flaw.

However! price improvement does occur if the limit price is exceeded at the start of a new session. If you have a GTC order in the market it is filled at the opening price if the limit price is exceeded. If your system resubmits the Day order then you'll also get a price improvement (if the prevailing bid/offer exceeds the limit price).

MC have told me that running "Fill limit order when trade price goes beyond limit price by 1 points" will mean that your backtest will show fills with no price improvement, even at the session open as described above.

In short, this means that the improvement to the "Fill limit order when trade price goes beyond limit price by 1 points" assumption is still wrong in 9.1 beta. Indeed if you backtest on daily bars with this assumption on in beta then it will be very wrong indeed.

MC, please change the "Fill limit order when trade price goes beyond limit price by 1 points" behavior to allow price improvement at the open.

Other MC users please share your thoughts.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 11 Aug 2015

Hello wilkinsw,

There is no need in moving the conversation on the subject from one topic into the other.
Your suggestions will be evaluated by our team and we will post the solution once it is available.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 25 Aug 2015

Hello wilkinsw,

I have consulted with our developers regarding your inquiry. In MultiCharts all orders in backtesting are GTC. Basically you are requesting adding other TIF support for backtesting. This is not a trivial functionality and it requires significant changes in MultiCharts core components. Before we evaluate adding it in the future we would like to see the user demand for such improvement. You may want to submit a feature request to the Project Management of our web site so other users can vote for it: https://www.multicharts.com/pm/

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 25 Aug 2015

Henry,

I simply think that you and Multicharts are not understanding me, unfortunately. Maybe I have used incorrect terminology somewhere?

How am I asking for a different TIF to GTC?? I have no idea where you got that from at all!

To be clear, when talking about "price improvement" I'm referring to a fill at a better price than that specified by the limit order.

At the start of a new session price improvement for the fill of a limit order can happen. That's it. Nothing more to add. Modelling in MC has to reflect that.

When I asked you if the rework of MC's limit order fill assumptions will stop this possible price improvement you said "yes". This should NOT be the case, full stop. Behavior that never occurs in the real world should not be added to MC Beta.

Regardless of the selection of "Fill limit order when trade price goes beyond limit price by 1 points", price improvement of a limit order fill is possible at the start of a new session.


That really is it. It is quite simple. You've told me that a certain behavior is occurring in MC beta re limit fills and I've told you it's wrong. Can you address this please and/or demonstrate I'm wrong. I'm not asking for new order types!! The following is supplementary information.........

MAtricks raised the issue that price improvement is falsely occurring when "Fill limit order when trade price goes beyond limit price by 1 points" is selected. MC Beta has addressed this raised issue.

HOWEVER, there are two occasions when price improvement does actually occur in real life and selecting "Fill limit order when trade price goes beyond limit price by 1 points" should not effect these.......

1) If a new order (a marketable limit order) is submitted after the close of a bar......... then the fill will occur at the prevailing bid/offer at that moment (usually at a worse price, but price improvement can happen too).

2) If an existing/working limit order is gapped through at the open/start of a new session. Depending on how your broker handles your order, then you get matched at the OPENING price. This opening/fill price, in this example will be better than the limit price of the order...hence a price improvement. Even if you are submitting a fresh limit DAY order at the open... there will still likely be a price improvement. If the same "gapping through" of a limit order occurs intraday from the close of one bar to the open of another, then price improvement must NOT occur, even if there is no historical tick for MC to match against (as MAtricks correctly identified).

To put it another way, the MC Beta amendment to limit order fills when selecting "Fill limit order when trade price goes beyond limit price by 1 points" is 100% wrong if backtesting on daily bar charts, as bar to bar gaps will always give limit order price improvements on daily charts.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 25 Aug 2015

I've attached an example erroneous fill, from a inter-day gap in the eurostoxx recently (60min chart)
Beta fills assumption.PNG
Impossible fill
(22.95 KiB) Downloaded 5193 times
This fill is simply impossible!!! MC has created a new bug by trying to fix an old one. The fill should be placed at the open of the new session. If you roll out the next version of MC with these fill assumptions it will be a huge step backwards indeed! Please stop dragging your feet and address this very obvious error. MC needs to add this exception to the "Fill limit order when trade price goes beyond limit price by x points" rule.

Thank you.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 25 Aug 2015

beta fill assumption intraday.PNG
(18.02 KiB) Downloaded 5223 times
The intraday functionality of the Beta amendment is fine and is indeed an improvement (see attached)

Please, please, please try and see that MC have created a new issue here. Once fixed, you've got something very exciting here!

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 26 Aug 2015

I've uncovered another bug....

When turning off "Fill limit order when trade price goes beyond limit price by x points" the fill price should AGAIN not be improved.

It should still show a fill at 1182.4 (as in the previous post) and NOT 1182.5, as 1182.4 is the price specified to the limit order!:
Attachments
price improvement with assumption off.PNG
(21.3 KiB) Downloaded 5233 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 26 Aug 2015

Hello wilkinsw,

We will get back to you with the analysis results once our engineers study the information you have provided.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 09 Sep 2015

Any news?

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limit Orders with incorrect fills

Postby Henry MultiСharts » 15 Sep 2015

We are evaluating adding this improvement in the next minor platform update.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 15 Oct 2015

Hi Henry,

Has the issue been remedied (allowing price improvement at the open of a new session), in:
https://www.multicharts.com/traders-blog/?p=1482

?

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: Limit Orders with incorrect fills

Postby wilkinsw » 04 Dec 2015

Any updates?


Return to “MultiCharts”