In this screen shot it's overly optimistic:
-IBOG is turned off
-Entry logic is:
Buy next bar open limit ;
Sellshort next bar open limit ;
As you can see, the price doesn't fulfill a tick beyond the open price on over half of the orders.
Henry, I believe this to be incorrect. Why would we ever want to place an order at the open of the last bar? There's no world that this makes sense in... I did tests in TS to compare and sure enough.. for once they had it right. Also while referencing the Easy Language Essentials Guide I find this quote on page 95:When the script is calculated on the bar close (IOG Off) - it takes the current bar's open value and forms the order price. The order is “sent” on the next bar open.
If you want to send an order with the open price of the current bar then you need to turn on IOG, once the strategy is calculated on bar's opening tick you can assign this opening price to your order and send the order on the next tick.
Code: Select all
Open Next Bar
Strategies, by default, calculate on the close of each bar and generate orders into the next bar. However, traders
often want to base their strategy rules on the open of the next new bar. Open Next Bar syntax allows the
calculation engine to delay the strategy calculation until the opening tick of the next bar, and allows you to
reference that opening price in your strategy conditions for orders on that same bar.
Usage Example:
Buy next bar at open next bar - .05 Limit;
(The syntax ‘Open Next Bar’ refers to the opening price of the next bar.)
In this example, we delay the strategy calculation with ‘open next bar’ until the opening tick and then
use that opening price in our order.
Code: Select all
buy next bar C-ticksize ;
Sellshort next bar C+ticksize ;
Orders are filled differently in TS backtesting. In TS you can have orders filled between the bars where there was no actual price action. In MultiCharts we do not fill an order without an actual price for it.Henry, I believe this to be incorrect. Why would we ever want to place an order at the open of the last bar? There's no world that this makes sense in... I did tests in TS to compare and sure enough.. for once they had it right. Also while referencing the Easy Language Essentials Guide I find this quote on page 95:When the script is calculated on the bar close (IOG Off) - it takes the current bar's open value and forms the order price. The order is “sent” on the next bar open.
If you want to send an order with the open price of the current bar then you need to turn on IOG, once the strategy is calculated on bar's opening tick you can assign this opening price to your order and send the order on the next tick.
http://www.TS.com/~/media/Fil ... tials.ashx
Errors like this create extremely optimistic back-tests...Code: Select all
Open Next Bar
Strategies, by default, calculate on the close of each bar and generate orders into the next bar. However, traders
often want to base their strategy rules on the open of the next new bar. Open Next Bar syntax allows the
calculation engine to delay the strategy calculation until the opening tick of the next bar, and allows you to
reference that opening price in your strategy conditions for orders on that same bar.
Usage Example:
Buy next bar at open next bar - .05 Limit;
(The syntax ‘Open Next Bar’ refers to the opening price of the next bar.)
In this example, we delay the strategy calculation with ‘open next bar’ until the opening tick and then
use that opening price in our order.
The limit order is not executed on X "points". A Limit order is executed at the specified price or better. The execution price does not change. It is verified that the price went through the order X amount of points and the order is filled on the nearest actual price present on the chart that satisfies the limit order execution conditions. Order cannot be filled between the bars without price action.I guess this follows on this topic so I'll post it here..
When turning on FILL LIMIT ORDER WHEN TRADE GOES BEYOND LIMIT PRICE BY [x] POINTS, the limit orders are executed on X "points" in a back-test which is wrong. The execution price should not change, it should just be verified that the price did in fact go through the order X amount of points.
Example:
Orders are placed the yellow and blue arrows, but the back-test is showing that they're +1 tick of profit because of turning on this valuable but broken feature:
Turning off the option causes the strategy to be recalculated. Without this option you have different execution for orders before the mentioned bar. As you can see the Short entry filled on a different bar, therefore the provided picture represents a different calculation that cannot be compared to picture1.Now I turn it off which shows me the correct execution price, but doesn't filter out the trades that price didn't go through 1 "point" (should be called tick) as is needed with a back-test.
You hit on a big issue I think that has been pointed out with backtesting limit orders (I reviewed the most recent release and I don't believe it was fixed).
If I go and check "FILL LIMIT ORDER WHEN TRADE GOES BEYOND LIMIT PRICE BY 1 POINTS" (which by the way "points" really means "ticks") --->For example, I would assume that if my strategy entered a short limit order on the @ES for 1600 that I would only get filled at 1600 if the next bar actually went to 1600.25. This is not the case.
However, I think we enter trades with limit orders to protect ourselves. That is, we want a specific price point. In backtesting, we can only be assured of a fill on a limit order if the price goes through our price point. As the original poster put it "The point of the backtesting assumption is to only allow trades if the market moves pass the limit price to ensure a fill."
This would seem like a simple fix. I think there is a voting link here to fix it but there has been no progress as of late:
http://www.multicharts.com/pm/viewissue ... _no=MC-149
Just a quick update since we didn't get a response from anyone at MC regarding this issue.
TS apparently offers the ability to have the price exceed the limit order for the strategy order to be filled in backtesting. Under TS, you go to Format->Strategies->Properties for All then click on the Backtesting Tab. Click on the second option "Fill entire order when trade price exceeds limit price." So if you enter a limit order on a stock for $10 next bar, then the price must go to 10.01 for the order to be filled in backtesting.
MC - I know you guys can fix this!
There is no issue with the mentioned functionality. The feature works as it is designed to work. You are just asking for a different behavior described in the mentioned PM entry (Variant 2):Multicharts has that feature as well.. It just doesn't work -that's the threads topic
I think we all agree that the limit order can be filled at the limit price or better. What we are requesting is that MC offer an option in later editions that allows you to ensure that the limit price you specify in the strategy satisfies BOTH of the following (1) price exceeds the limit price you specify in the strategy limit order AND (2) you are filled at the limit price you specify in the order (and not the limit price + [X] points).The limit order is not executed on X "points". A Limit order is executed at the specified price or better. The execution price does not change. It is verified that the price went through the order X amount of points and the order is filled on the nearest actual price present on the chart that satisfies the limit order execution conditions. Order cannot be filled between the bars without price action.
I am afraid there is a misunderstanding here, either from our end or from yours. Let me make sure we are talking the same things.I think we all agree that the limit order can be filled at the limit price or better. What we are requesting is that MC offer an option in later editions that allows you to ensure that the limit price you specify in the strategy satisfies BOTH of the following (1) price exceeds the limit price you specify in the strategy limit order AND (2) you are filled at the limit price you specify in the order (and not the limit price + [X] points).
If this were possible, then we could ensure that "worst case" we get filled at the price we specify and "best case" we get filled at a better price.
Everything described in this thread is expected behavior. if you want it to work differently, please describe it and we will discuss it with our management.I'm unsure where to go from here.. I stare at this platform 18 hours a day in real time and am frustrated with the flaws that you say aren't there.
That is negative, Sir. The first best available price and it meets the limit order execution criteria here is open = 100. The order would be filled at 100.Buy limit at 100:
Under the current setup Option #1:
- Fill limit order at Y price or better (lower): OHLC of entry bar=100-101-99-100; the order would be filled at 99 because this is the first best available price and it meets the limit order execution criteria. In reality, this price is at the tail of the bar so there is no guarantee that you would get a fill.
That is negative, Sir. The backtesting engine sees that the price 99 (100-1) is present (low). The first best available price and it meets the limit order execution criteria here is open = 100. The order would be filled at 100.Under the current setup Option #2 (and just assume that N=1):
- Fill limit order at Y price or better (lower), but only if there N ticks beyond (100-N); OHLC =100-101-99-100; Since N = 1, the backtesting engine sees that the price 99 (100-1) is present (low), the order is filled at 99 because this is the first best available price and it meets the limit order execution criteria
You are welcome to provide us with the following set of files demonstrating a different behavior, i'd be glad to study the case:Under the option I am suggesting:
- Fill limit order at Y price: OHLC=100-101-99-100; the order would only be filled at 100 (the limit price in the strategy) only because the price went to 99.
Henry,The limit order is not executed on X "points". A Limit order is executed at the specified price or better. The execution price does not change. It is verified that the price went through the order X amount of points and the order is filled on the nearest actual price present on the chart that satisfies the limit order execution conditions. Order cannot be filled between the bars without price action.I guess this follows on this topic so I'll post it here..
When turning on FILL LIMIT ORDER WHEN TRADE GOES BEYOND LIMIT PRICE BY [x] POINTS, the limit orders are executed on X "points" in a back-test which is wrong. The execution price should not change, it should just be verified that the price did in fact go through the order X amount of points.
Example:
Orders are placed the yellow and blue arrows, but the back-test is showing that they're +1 tick of profit because of turning on this valuable but broken feature:
Turning off the option causes the strategy to be recalculated. Without this option you have different execution for orders before the mentioned bar. As you can see the Short entry filled on a different bar, therefore the provided picture represents a different calculation that cannot be compared to picture1.Now I turn it off which shows me the correct execution price, but doesn't filter out the trades that price didn't go through 1 "point" (should be called tick) as is needed with a back-test.
If you don't mind, i'll let myself explain what Henry mentioned in his post releated to the pictures.Henry,
This is not realistic. Which is where I was going with this topic initially. A limit order is filled at its price or better. However, that "or better" does NOT happen very often.. and especially not every bar. If a buy limit order is placed at 100, it will fill at 100, not 99. However, this function that you're saying is working properly is telling me exactly that. It's something to look into...
Thank you, Matricks. We do appreciate your time and effort. Our first goal is to make sure there is no bug. If there is one, we do our best to fix it. If what you are looking for is a feature not yet available in the software, we can discuss its implementation for future versions of MC if you describe what exectly you would like us to do.Both of you,
I'm sorry if my frustration has shown through my text. I feel like I report issues and then my work is finished, but I'm simply argued with until I walk MC through the problem step by step which I don't think is fair. I just want MC to be the best.... and frankly I think that I've put in a lot of free work to better this product. Maybe I'm not the best at explaining the issues. We're doing good, but there are several of these issues that are simply NOT realistic regardless if they're "expected behavior" or not.
The way I described the backtest with the fills at 99 under Option #1 and #2 is the actual results that I got in backtesting the strategy regardless of how the engine should work. If you look at the picture I attached, you will see a long entry at the low of the bar when actually the limit order was for the middle of the bar. This happens over and over again in backtesting. Will forward you the requested information. Thanks Andrew!That is negative, Sir. The first best available price and it meets the limit order execution criteria here is open = 100. The order would be filled at 100.Buy limit at 100:
Under the current setup Option #1:
- Fill limit order at Y price or better (lower): OHLC of entry bar=100-101-99-100; the order would be filled at 99 because this is the first best available price and it meets the limit order execution criteria. In reality, this price is at the tail of the bar so there is no guarantee that you would get a fill.That is negative, Sir. The backtesting engine sees that the price 99 (100-1) is present (low). The first best available price and it meets the limit order execution criteria here is open = 100. The order would be filled at 100.Under the current setup Option #2 (and just assume that N=1):
- Fill limit order at Y price or better (lower), but only if there N ticks beyond (100-N); OHLC =100-101-99-100; Since N = 1, the backtesting engine sees that the price 99 (100-1) is present (low), the order is filled at 99 because this is the first best available price and it meets the limit order execution criteria
I believe this is where there is some misunderstanding. What you consider to be an issue is not actually one. It is the difference in results of 2 separate independent strategy calculations and further i will do my best to explain why the results are different, why it is expected and simply cannot be the other way.From you pointing that out, I see an even larger issue.. but your conclusion was that there wasn't any?
My first point is that by changing literally any setting of backtesting/auto trading, you make the script be recalculated and not simply recalculated to give you the same results, different settings = different results. So when you change any setting, you should expect different calculation results (orders generated/filled/not filled at different bars/prices and so on).The orders change.. which is odd because all I did was select and deselect the FILL LIMIT ORDER WHEN TRADE PRICE GOES BEYOND PRICE BY 1 POINT option.
This is the key point. Backtesting, forward testing (real-time calculation, AT is off) and automated trading results can be different in some cases. This is expected because some factors affecting real-time strategy execution results (at broker account) cannot be emulated in backtesting and in forward testing.if you see my live execution you will notice that REAL orders do not change. The entire point of looking at how these back-tests "execute" is to make them as close to the real world as possible. While comparing live execution and the back-test and changing only one feature, this is the result. I'm unsure how we can conclude that this is a working product?
If the back-test doesn't match the live execution, something is wrong. (cyan and yellow arrows are live execution).
A strategy is recalculated (backtested) when:The chart was trading and all I did was copy/paste the window so I had the live orders + the back-test.
MAtricks, as i understand, the first picture shows auto-trading results and the second picture shows backtesting results. I have to say that the 2 pictures showing different results do not show any problem.Those 2 pictures say it all..
I understand your goal to have the same backtesting, forwardtesting and real-time auto trading results. I do agree that it would be extremely useful, however as i previously said, there are some limitations, which prevent backtesting from being as realistic as auto-trading in real time.Live executed orders along with the back-tests that are different when changing one fill option. This doesn't make me feel warm and fuzzy about Multicharts back-tests its orders..
Thank you. We will study the case and i'll post the results here.The way I described the backtest with the fills at 99 under Option #1 and #2 is the actual results that I got in backtesting the strategy regardless of how the engine should work. If you look at the picture I attached, you will see a long entry at the low of the bar when actually the limit order was for the middle of the bar. This happens over and over again in backtesting. Will forward you the requested information. Thanks Andrew!
Incorrect. A gap in price is a gap in price, NOT a gap in orders for price. Just because Multicharts' bars do not have a price illustrated on the chart doesn't mean there is no price at the exchange. Think of a vertical line of price from $1 to $10000000 instead of between calculated blocks/quantities/time periods of this price (bars on a chart).Orders are filled differently in TS backtesting. In TS you can have orders filled between the bars where there was no actual price action. In MultiCharts we do not fill an order without an actual price for it.Henry, I believe this to be incorrect. Why would we ever want to place an order at the open of the last bar? There's no world that this makes sense in... I did tests in TS to compare and sure enough.. for once they had it right. Also while referencing the Easy Language Essentials Guide I find this quote on page 95:When the script is calculated on the bar close (IOG Off) - it takes the current bar's open value and forms the order price. The order is “sent” on the next bar open.
If you want to send an order with the open price of the current bar then you need to turn on IOG, once the strategy is calculated on bar's opening tick you can assign this opening price to your order and send the order on the next tick.
http://www.TS.com/~/media/Fil ... tials.ashx
Errors like this create extremely optimistic back-tests...Code: Select all
Open Next Bar
Strategies, by default, calculate on the close of each bar and generate orders into the next bar. However, traders
often want to base their strategy rules on the open of the next new bar. Open Next Bar syntax allows the
calculation engine to delay the strategy calculation until the opening tick of the next bar, and allows you to
reference that opening price in your strategy conditions for orders on that same bar.
Usage Example:
Buy next bar at open next bar - .05 Limit;
(The syntax ‘Open Next Bar’ refers to the opening price of the next bar.)
In this example, we delay the strategy calculation with ‘open next bar’ until the opening tick and then
use that opening price in our order.
In the example above - even if you generate an order with the current bar’s Open limit price– the order will be filled at the same price on the same bar. That will not change the behavior.
This is considered to be the correct and expected behavior.
I am sorry for barging in again, but i may help you with this one too.Incorrect. A gap in price is a gap in price, NOT a gap in orders for price. Just because Multicharts' bars do not have a price illustrated on the chart doesn't mean there is no price at the exchange. Think of a vertical line of price from $1 to $10000000 instead of between calculated blocks/quantities/time periods of this price (bars on a chart).
Not sure i understand you correctly. Please elaborate.Please answer this question: When would we as consumers use the OPEN price of the Last Bar? .. think about that one for a secondExpected behavior or not, that is pretty silly..
At the moment the feature is not scheduled for any particular MC version.Any luck with this update (or requested update)?
Are you referring to the Extended Backtesting issue?I have been working on these types of issues for a long time. I still wait for replies to my forwarded files, workspace etc. So I continue my efforts in an attempt to develop a work-a-round.
Please come to our live chat during working hours (6:30 am – 2:45 pm EST) to demonstrate it, let our operators connect to your computer remotely in order to study the issue.1. Write a simple strategy with an input for X and Y.
Buy Next Bar at X Limit
Sell Next Bar at Y Limit
2. Create a 5 minute chart, 10 point chart, 100 tick chart and 100 change chart all on the same instrument. I will use Euro future as an example.
3. Select 2 levels that over the screen view have been hit many times. I am selecting 1.3750 and 1.3755.
4. Snap 2 horizontal lines on each chart at those exact prices.
5. Apply strategy with those levels selected to each chart.
6. Try various configurations of IOG on/off and magnifier on or off.
7. Look for signals that hit the lines and don't.
8. Run Playback and do the same thing.
9. Run it receiving live market data and trade it live or demo acct your choice.
10. Try changing the code to something on the bar that would be as easy to identify as a price level. Example use the high of the bar to replace a level. I can not get a limit order to be placed using the high of the previous bar as the value. Nor can I use the low.
11. Sorry forgot to add open a fresh chart with the same number of bars available to the strategy and compare to the above.
Gather all the information.
While I am still running my tests on Ver 3 just released,
I so far see that there is a big difference in results based on bar type. That to my eyes, minute charts seem to be the best. I am guessing that is because you can't (should not need to ) magnify the other data types in extended mode.
Doesn't seem to be the case on my end:Also I have discovered that on 64 rel 3, if you have a time chart, magnify the data and then turn on IOG, it crashes..everytime.
Sorry, you did not mention this beforeI thought I would comment on what you presented. It is more pronounced when in extended mode.
Still looks good:Also I have discovered that on 64 rel 3, if you have a time chart, magnify the data and then turn on IOG, it crashes..everytime.
When extended mode is used, orders are executed at asks and bids, not at base data series, so orders can be executed outside of base data series bars because the was such ask or bid price.You can have the data streams visually present and the signal will still be at a price that does not exist.
Could you provide us with following files?Example
BID 100
ASK 101
SIGNAL 104
Exactly what I've been trying to explain in several threads like this one.I would like to comment on this discussion from a real-life markets perspective (as opposed to intended code behaviour). I too widely use limit orders for the subsequent bar's order (usually based on the current bar's Close-1 tick, if going long).
In live markets, if a limit order is traded through (i.e. last bar's Close was 100, I have a limit order to go long at 99, and the bar low is 98), then the exchange will match the trade at the limit price. Guaranteed.
I understand that in backtesting, the bar in which the order resides may not include that price, e.g. a gap may mean the bar high is 98, but that doesn't matter. The fill IMHO should be recorded at 99 as that is what it would have traded at live. Not the 98 bar high.
The last thing a backtester wants is fills that are too good vs. live trading.
I have voted for this "feature request", and think it is an important one.
I have checked that: David responded to you on Tuesday, February 18, 2014 7:05 PM (our local time).All of this was sent to Dave weeks ago.
No reply
I also found the following screenshot demonstrating how our engieners tested your setup on our end:Hello Don,
I apologize for the delay in responding. The engineers have analyzed the behavior that you reported. You are using Extended Backtesting mode which means that your orders are filled at Ask and Bid prices which can be far from current Trade price. It is normal behavior. You may want to disable Extended Backtesting mode.
If you have any further questions, please do not hesitate to contact MultiCharts.
Best Regards,
Dave Masalov
I have checked that: David responded to you on Tuesday, February 18, 2014 7:05 PM (our local time).All of this was sent to Dave weeks ago.
No replyI also found the following screenshot demonstrating how our engieners tested your setup on our end:Hello Don,
I apologize for the delay in responding. The engineers have analyzed the behavior that you reported. You are using Extended Backtesting mode which means that your orders are filled at Ask and Bid prices which can be far from current Trade price. It is normal behavior. You may want to disable Extended Backtesting mode.
If you have any further questions, please do not hesitate to contact MultiCharts.
Best Regards,
Dave Masalov
From the screenshot we can see the orders are filled at asks and bids when extended mode is used. Please provide us with a new set of requested files if you still have the problem. The issue has not been confirmed yet.
We do not have any files demonstrating the situation. Please provide us with following files those demonstrate the problem you described:Andrew,
My last attempt on this issue. It is quite frustrating to point out, that which to me is clear, but apparently I am lacking the ability to communicate it clearly.
Buy Signal 104
Ask is 101
Trade is 100.50
Bid is 100
Sell Signal 95
There is no price data at the price of the signal. It was not offered at, never traded at those prices.
Thank you,
Don
Don, an issue cannot be fixed if it is not demonstrated/reproduced/studied. We haven't received any similar reports. All the files you had sent us were studied and engineers confirmed there is no problem. If you would like us to help you with the problem, please come to our live chat to demonstrate it and let us collect all required information ourselves.Andrew,
I will have to put this off. I need to find that which has been sent already.To the best of my understanding, I sent everything requested to Dave. He confirmed receipt and said the issue was being looked at. I believe his request was everything your asking for.
As a suggestion. If MultiCharts made it possible for a customer to log into a computer at MC, so we could show the issues on the canned indis, strategies, software, setup etc. Show the problem and be done with it. I have spent way to much time on this 1 issue to just present it.
I have other issues behind this one. Each on its own a significant problem.
I can promise you, it is not unique to my setup.
Although I know this is not the intent, from a customers perspective, it begins to feel like I need to prove the problem exists. That those reviewing the information, is to find fault with the request rather than the software. Again, I know this could not possibly be the intent, but how it feels to get across a problem.
Some users have no experience and therefore what they may present might have the need to be questioned. It maybe for them, they need to be told to plug in the computer and turn on the monitor. There are those however with a lot of experience and while it is possible for them to overlook the obvious, it is not as likely. In my case, my first support person in the field of financial software was Bill Cruz, so I have been at this awhile and it would be great to move past the obvious stuff like telling me using bid and ask "could be higher or lower than a trade price. Bottom line is Extended backtesting especially with non-time charts has a problem in it.
Thank you,
Don
Please describe in details what exactly is not correct with the assumptions for point-based charts."Fill Limit Order when trade price goes beyond limit price" still doesn't work at all with Point bars (mostly where I see the issue). Wondering if a fix is in the making?
These back-testing flaws are what really hurts this platform imo.
But to be more precise, where was our hypothetical price before opening at 1800.25?
- buy limit @ 1800.00;
- sell limit @ 1800.50;
Thank you, MAtricks. Hypothetical price before opening at 1800.25 would be 1800.00 (close of previous bar).But to be more precise, where was our hypothetical price before opening at 1800.25?
sell limit @ 1800.00;
buy limit @ 1800.50;
These would be marketable limit orders and would fill at the market price. So if price was 1800.25, our orders would still fill at these prices:sell limit @ 1800.00;
buy limit @ 1800.50;
MAtricks, i cannot agree it is an error. Limit orders are always orders to be filled at price or better. If you want them to work different way for backtesting to have worse backtesting results, it is a feature request, not an issue of the software.I agree, but this is an error that should be resolved... The price improvement shouldn't be there. Back-tested Limit Fills should be at Limit Prices- always.
Replicating real world fills on each back-tested order should be one of the highest priorities to a trading platform. Limit orders should be filled at the Limit price and only when price trades N ticks through that price. Market orders should be filled at the market price less N ticks for an inputted spread.