+1 888 340 6572
MultiCharts Project Management
previous_open_issue.png
Go to the previous open issue
previous_issue.png
Go to the previous issue (open or closed)
star_faded.png
Please log in to bookmark issues
bug_report_small.png
Open Bug report MC-2545

"Backtesting Assumptions" is Flawed and creates incorrect back test results

action_vote_minus_faded.png
-1
Votes
action_vote_plus_faded.png
next_issue.png
Go to the next issue (open or closed)
next_open_issue.png
Go to the next open issue
Description

This issue relates to the dialog box found under Format Signals/Properties/Backtesting/Backtesting Assumptions

Sadly I just lost a long post I made, which seriously annoyed the hell out of me. I will now try to keep the post shorter and to the point.

The "Backtesting Assumptions" dialog box needs to change to 2 choices, as currently it does not do what is intended correctly.

My suggestion is to split it into 2 choices, one relating to BIDS prices (Buys/Short covering) and the other to ASK prices (Shorts & Sells).

Why? I will try to explain. The discussion only relates to LIMIT ORDERS, and is particularly important for forex traders, where the SPREAD is everything.

It is absolutely essential for everyone, including MC support staff and software engineers to understand that Buy and sell orders MUST be treated differently.
When executing buys, it is ONLY the ASK price that is relevant, and when executing sells, it is ONLY the BID price that is important.
Also, appreciate that the vast majority of back testers are only plotting the BID/traded price. We can sometimes add the ask, but in most cases it is simply not needed nor relevant, and only complicates the back testing.

Currently, the 'Fill limit order when trade price goes beyond limit price by' makes the SAME order/fill adjustment for both buys AND sells. THIS IS 100% FLAT OUT WRONG.
Here's why.

When trying to buy at a limit, we need to adjust for 2 things. The spread, and the amount price must exceed our buy limit order to make sure we are reliably filled. This becomes more important the larger our order size is. So for example, let's assume we have a buy order in the EURUSD which has a 6 tick spread @ 1.15250. Now since the plotted price on our screen is the BID, it means we must actually see traded/Bid price drop to 1.15244 or lower to get filled. Currently we can accurately do this via the checkbox under 'backtesting assumptions' Great. So we would put in a value of say 7 (6+1) to make sure we got filled.

Now lets move to sells or shorts. We have a limit order to sell @ 1.15350. This means we only need to see the charted bid/traded price to show 1.15350 to be filled, and perhaps add 1 tick for reliability. But what your back test will in fact show is that you don't get filled because we put in a value of 7, as per the prior paragraph. Price would need to show a value of 1.15357 to be filled. THIS IS 100% FLAT OUT WRONG. All we need is 1.15350 or 1.15351. The only adjustment required is the 1 tick for reliability of fills. This issue gets even worse when dealing with Forex markets that have progressively wider spreads.

I hope everyone can understand what I have explained, and begin to appreciate the PROFOUND importance of getting this right is. I'm actually shocked that we are up to V12 and this has continued unchanged since whatever version it originated in. Perhaps most MC users are futures traders where this is less of an issue.

I need MC support and their software engineers to fully appreciate this is a serious problem and needs to FIXED ASAP.

My suggestion is to split the dialog box into 2 choices. One that impacts Buys, and one that impacts sells. These 2 boxes will ALWAYS have different values. Simply because the plotted price is only the BID/trade price. In the FOREX market you will NEVER be filled on a buy at the bid. In the futures markets this is less of an issue, and perhaps why this problem hasn't come to the surface sooner.

If anyone has any further comments, insights, corrections or suggestions, please speak up as soon as possible. And PLEASE Vote to have this corrected, even though the whole concept of voting on a bug fix is utterly absurd.

Steps to reproduce this issue
Nothing entered.
Comments (8)
#1
user-offline.png  Will (wilkinsw)
Jan 12, 2019 - 04:02

AdrianP,

I understand you want to accurately model fills using just Bid data in FX?

Firstly, I think the first solution is Extended Backtesting Mode. Have you tried that? It sounds like if you use both bid and ask data; then you might have a solution.

Secondly, you might be right on one front: MC could do with adding features unique to FX traders. Off the top of my head though, can't you just use mid price data and then add 0.5*spread+1??? Can you get mid price data??

I'm not an FX trader. I think if you want to put forward solutions for FX traders, then you need to detail how various brokers provide their FX data (bid, mid, trade??) before then forwarding possible solutions.

If you are telling me that mid price is not available and are also saying broker spreads are relatively static:

Maybe MC could add a radio button that says:

   "If FX Bid data: {auto tick if spread is defined, below}
                          spread: _______"

MC can then modify the behaviour, if spread is defined, for function: "Fill limit order when trade price goes beyond limit price by ____ points".... to now treat sell fills like it does normally in classic backtesting. Then treat buy fills as having to be: (trade{bid} data-X_points) - spread.

What do you think??

#2
user-offline.png  Zheka (Zheka)
Jan 12, 2019 - 05:43

Do you assume that FX traders use Bid prices because MT4/5 uses them?

Use mid-point price for more accurate results, which you can get either as as average of bid-ask or straight from IB.

To prototype most ideas, either Bid or Ask streams or Mid-point price with a fixed spread in a Classic mode will do.

To validate time-sensitive ideas (like trading around news), you will need to use both Bid and Ask and an Extended trading mode.

Spread is "random" and using "noise" as input will allow you to curve-fit your back-test results but has little bearing on live trading results.

Also, if your broker allows you to place limit orders which become part of "the book" (like IB), then it can get filled regardless of displayed bids/asks.

#3
user-offline.png  AdrianP (AdrianP)
icon_reply.pngJan 12, 17:15, in reply to comment #1

Hi Will,

Thanks for your response, but it is mostly negative and misguided. The problem I discussed impacts EVERY back test, be it FX or futures. It impacts futures less though since the spreads are usually or often just one tick. And when most people are using larger time frames, it impacts even less. The highest impact is to Forex traders using small intraday time frames.

I haven't tried extended back test mode. I will look into it.
Adding ask data will NOT solve the problem I discussed. I'm not sure how one would even use it when plotting bars of some nature.Unless you did a back test on 1 tick charts, how would MC even know what to match up? Since bid and ask data is constantly changing. And not at the same time. Anyway, adding ask data just adds a 100% completely unnecessary complication, RAM hogger, time slower, coding issues etc etc. It is one of the key reasons MC added the 'backtest assumptions'. It eliminates all those issues.

The suggest improvements I have made has NOTHING TO DO with being unique to forex traders. Your comment shows a lack of understanding of what I explained.

No you can't just use mid-price data. Once again you show lack of understanding of the issue and back testing.

Since you aren't an FX trader, why are you responding to a thread that you believe only relates to FX traders? FX traders of any decency will be acutely aware of everything I have discussed.

And all your other suggests that are FX related keep showing me you don't understand what I am talking about.

PLEASE PLEASE....may I ask that only professionals respond to this post. Otherwise we get distracted and taken off on some pointless tangent.

'''Will wrote:'''

AdrianP,

I understand you want to accurately model fills using just Bid data in FX?

Firstly, I think the first solution is Extended Backtesting Mode. Have you
tried that? It sounds like if you use both bid and ask data; then you might
have a solution.

Secondly, you might be right on one front: MC could do with adding features
unique to FX traders. Off the top of my head though, can't you just use mid
price data and then add 0.5*spread+1??? Can you get mid price data??

I'm not an FX trader. I think if you want to put forward solutions for FX
traders, then you need to detail how various brokers provide their FX data
(bid, mid, trade??) before then forwarding possible solutions.

If you are telling me that mid price is not available and are also saying
broker spreads are relatively static:

Maybe MC could add a radio button that says:

   "If FX Bid data: {auto tick if spread is defined, below}
                          spread: _______"

MC can then modify the behaviour, if spread is defined, for function: "Fill
limit order when trade price goes beyond limit price by ____ points".... to
now treat bid fills like it does normally in classic backtesting. Then
treat sell fills as having to be: (trade{bid} data+X_points) - spread.

What do you think??

#4
user-offline.png  AdrianP (AdrianP)
icon_reply.pngJan 12, 17:27, in reply to comment #2

Zheka,

I love that you are trying help, or I hope that is why you posted a reply, but please...you clearly do not understand what is being discussed. May I suggest you spend time learning about what I discuss, before making suggestions when you aren't a professional, experienced in the whole situation being discussed.

Mid-points won't resolve the problem.

My comments are definitely not related to what MT4 does. Only beginners use MT4. It is NOT a professional platform.

Once again, professionals don't do things 'because it will do' LOL

I made no reference to trading around news, and that has nothing to do with my discussion.

SPREAD is NOT random...where on earth do you come up with such nonsense?

I don't curve fit but once again, this has nothing to do with the original post.

All brokers allow limit orders to be placed. But few allow you to be part of the book, where yes, it is technically possible o get filled on the bid. But even then, you run a huge risk of not getting filled, and isn't a realistic expectation over the long run. So you have effectively reinforced my original message and the reasons why it needs to be corrected.

When we run a back test, we want to run it where that it the computer says we were filled, then there is a high probability that in fact we would have been filled real time. Simply because one stranger hits a bid, with your IB account, does NOT mean you were filled. And if you were trading ES, it is almost 100% certain you weren't filled.

I ask readers once again , PLEASE, only respond to my original message IF you are a professional and fully understand the issues I am discussing. Frankly, it is black and white, and not even up for discussion in terms of accuracy. And for everyone else, try to read, replicate, and understand what is being discussed before replying with your own suggestions.

My post is merely here to meet MC support recommendations. Really quite absurd, as this is a FLAW, error in coding, and needs to be fixed. It isn't something that should need voting on.

'''Zheka wrote:'''

Do you assume that FX traders use Bid prices because MT4/5 uses them?

Use mid-point price for more accurate results, which you can get either as
as average of bid-ask or straight from IB.

To prototype most ideas, either Bid or Ask streams or Mid-point price with
a fixed spread in a Classic mode will do.

To validate time-sensitive ideas (like trading around news), you will need
to use both Bid and Ask and an Extended trading mode.

Spread is "random" and using "noise" as input will allow you to curve-fit
your back-test results but has little bearing on live trading results.

Also, if your broker allows you to place limit orders which become part of
"the book" (like IB), then it can get filled regardless of displayed
bids/asks.

#5
user-offline.png  Will (wilkinsw)
Jan 12, 2019 - 23:23

AdrianP,

You are probably the most obnoxious user of Multicharts I have ever come across. I took time out to try and help a user, whom I felt sorry for, who is clearly lost.

Almost every single post of yours throughout the forums are abusive and unsubstantiated.

It looks like nobody knows what the flip you are talking about, I'm afraid. You are just wasting your own time.

MC12v4 fill assumptions for limit orders; are spot on as far as I've tested. I've done a lot of work in this area; check the forums.

Just a tip: You should be able to explain your case in such a way that a stranger on the street would understand. Getting frustrated with other users who aren't following you is a reflection on the quality of the original post!

#7
user-offline.png  AdrianP (AdrianP)
icon_reply.pngJan 13, 01:42, in reply to comment #5

I knew I would get an angry rude response from you. I am well versed with your type. If you continue to show such character I ask ask the forum moderator to band you.

How condescending is your reply. you felt sorry for me? Seriously? Your reply showed absolutely no indication of that. Then you say nobody knows what I am talking about? Seriously? I only just made my post yesterday LOL And the post was NOT made for people to reply to. It was made at the request of MC support staff. so please, show some common sense, maturity, courtesy and professionalism.

And simply because you have tested something does not mean it is correct. I can demonstrate with ease that it is not, and have done so to MC support staff. I stand 100% behind my post, with 100% confidence in what it states. I hope you can show some humility and discover and learn what it is I am talking about.

NO ONE here appreciates you starting off with a supposed innocent reply, when it has become clear you had an agenda. And it WASN'T to help. Experienced posters know your type. Please remain focused on the issue and NOT on the person making the post.We are not here to prop up your insecurities.

I do not have MC12v4. It has not even officially been released to my knowledge. But there is no indication in the change log the issue initially discussed has been corrected.

And no, I should not need to discuss it in a manner a person in the street would understand. That's just silly. Unless of course you are talking about getting a person from the street, teaching them about trading, and all about MC and how it works, then spending ages showing them how to do a back test, and all the problems that it can lead to. Is that what you mean?

My post is directed toward MC support staff and the software engineers only. That is the ONLY reason I posted here.
If they update the program then everyone benefits, even those who are unaware or see no need for the change.

I'm a professional Will, I hope that one day you can be too, and act in a manner appropriately.

I appreciate genuine help, but yours clearly was not, and please don't try to deny it. Yours was designed as an attack. And I went through each of the issues you raised, and dealt with them to indicate how they were relevant or simply would not solve the problem I discussed. These are facts and can be proven. Ad no, I have no intention of doing as such. All the information needed by MC software engineers is in my first post, which is 100% accurate.

PS...I will no longer be wasting my time responding to your messages. say what you will, but we will all know the true situation.

'''Will wrote:'''

AdrianP,

You are probably the most obnoxious user of Multicharts I have ever come
across. I took time out to try and help a user, whom I felt sorry for, who
is clearly lost.

Almost every single post of yours throughout the forums are abusive and
unsubstantiated.

It looks like nobody knows what the flip you are talking about, I'm afraid.
You are just wasting your own time.

MC12v4 fill assumptions for limit orders; are spot on as far as I've
tested. I've done a lot of work in this area; check the forums.

Just a tip: You should be able to explain your case in such a way that a
stranger on the street would understand. Getting frustrated with other
users who aren't following you is a reflection on the quality of the
original post!

#8
user-offline.png  Will (wilkinsw)
icon_reply.pngJan 13, 02:47, in reply to comment #7

AdrianP,

Have you noticed the voting tool at the top? You are currently failing in your efforts to convince others of the issue. Until you succeed the chances of your issue being resolved remain negligible.

So, AdrianP(P=professional??), please up you efforts and walk us through your issue. In our prop firm we are satisfied with how fills are placed for limit orders in MC12r4 (we weren't in the first MC12, nor MC11..... also <MC11 had other issues which were resolved).

I posted here simply in an effort to help (as I promised I would on the forum when answering a misguided post of yours).

You are a very angry person.

MC's accuracy is very important to my colleagues and me: If you've identified an issue...please walk us through it with examples. I don't see how using a mid-price or extended backtesting or the other solution I proposed, don't satisfy you. You are welcome to disagree. But I would advise you do so with counter arguments.... not rants. Please raise the tone of your responses or you will quickly find yourself isolated.

'''AdrianP wrote:'''

I knew I would get an angry rude response from you. I am well versed with
your type. If you continue to show such character I ask ask the forum
moderator to band you.

How condescending is your reply. you felt sorry for me? Seriously? Your
reply showed absolutely no indication of that. Then you say nobody knows
what I am talking about? Seriously? I only just made my post yesterday LOL
And the post was NOT made for people to reply to. It was made at the
request of MC support staff. so please, show some common sense, maturity,
courtesy and professionalism.

And simply because you have tested something does not mean it is correct. I
can demonstrate with ease that it is not, and have done so to MC support
staff. I stand 100% behind my post, with 100% confidence in what it
states. I hope you can show some humility and discover and learn what it
is I am talking about.

NO ONE here appreciates you starting off with a supposed innocent reply,
when it has become clear you had an agenda. And it WASN'T to help.
Experienced posters know your type. Please remain focused on the issue and
NOT on the person making the post.We are not here to prop up your
insecurities.

I do not have MC12v4. It has not even officially been released to my
knowledge. But there is no indication in the change log the issue initially
discussed has been corrected.

And no, I should not need to discuss it in a manner a person in the street
would understand. That's just silly. Unless of course you are talking
about getting a person from the street, teaching them about trading, and
all about MC and how it works, then spending ages showing them how to do a
back test, and all the problems that it can lead to. Is that what you mean?

My post is directed toward MC support staff and the software engineers
only. That is the ONLY reason I posted here.
If they update the program then everyone benefits, even those who are
unaware or see no need for the change.

I'm a professional Will, I hope that one day you can be too, and act in a
manner appropriately.

I appreciate genuine help, but yours clearly was not, and please don't try
to deny it. Yours was designed as an attack. And I went through each of
the issues you raised, and dealt with them to indicate how they were
relevant or simply would not solve the problem I discussed. These are
facts and can be proven. Ad no, I have no intention of doing as such. All
the information needed by MC software engineers is in my first post, which
is 100% accurate.

PS...I will no longer be wasting my time responding to your messages. say
what you will, but we will all know the true situation.

'''Will wrote:'''

AdrianP,

You are probably the most obnoxious user of Multicharts I have ever come
across. I took time out to try and help a user, whom I felt sorry for,
who
is clearly lost.

Almost every single post of yours throughout the forums are abusive and
unsubstantiated.

It looks like nobody knows what the flip you are talking about, I'm
afraid.
You are just wasting your own time.

MC12v4 fill assumptions for limit orders; are spot on as far as I've
tested. I've done a lot of work in this area; check the forums.

Just a tip: You should be able to explain your case in such a way that a
stranger on the street would understand. Getting frustrated with other
users who aren't following you is a reflection on the quality of the
original post!

#9
user-offline.png  AdrianP (AdrianP)
Feb 09, 2019 - 03:59

Just wondering when I can know that this issue is being looked at and assessed?

If every MC Forex back tester fully understood what they were doing, then I guarantee 100% would be voting for this modification.

But sadly, after 1 month, we haven't had even a single response from any Forex back testers, showing that either there are none using MC, or they are just blindly moving forward without a clue what they are doing. Thinking the numbers MC spits out are good, when in fact they aren't even remotely accurate.

With the programming adjustments required being simple and elementary, it seems like a no-brainer change relative to the massive increase in accuracy. My guess is its probably less than 60 minutes coding to correct the issue. As a comparison, it would be like tightening a screw on a race car engine, and finding your power is boosted by 50%. That's how serious this coding issue is.

History
Issue basics
  • Type of issue
    Bug report
  • Category
    Not determined
  • Targeted for
    Not determined
  • Status
    Under Review
  • Priority
    Not determined
User pain
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
Affected by this issue (0)
There are no items
People involved
Times and dates
  • Posted at
  • Last updated
Issue details
  • Reproducability
    Not determined
Attachments (0)
There is nothing attached to this issue
Commits (0)
There are no code checkins for this issue
Duplicate issues (0)
This issue does not have any duplicates