Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Questions about MultiCharts and user contributed studies.
oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 13 Oct 2021

Hello,

I noticed that the amount of ticks collected from live stream significantly differs from ticks downloaded from historical data feed. Much less ticks seems to be collected during live session!! The same occurs on 2 different computers.

I couldn’t find anything relative to that but this is a dramatic discover. I hope I missed something with the setup...

Configuration:
- Multicharts MC14 r3 with LMax as broker.
- Data Server Mode is Online with Download missing historical data checked.
- I am not opening Quote Manager, I only open Multicharts or Protfolio Trader while auto trading.
MC04_Live vs historical feed.png
(254.78 KiB) Not downloaded yet

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby TJ » 13 Oct 2021

most of the data feed only provide "Aggregated" ticks.

User avatar
Kate MultiCharts
Posts: 575
Joined: 21 Oct 2020
Has thanked: 7 times
Been thanked: 144 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Kate MultiCharts » 13 Oct 2021

Hello oppidum,

This is a known issue on the LMAX end. The data in MultiChart is displayed as provided by them. Please contact LMAX directly regarding the issue.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 13 Oct 2021

Thanks for your answer TJ.

Thanks for your answer Kate as well. But sincerely I can’t understand at all that such an important issue hasn’t been addressed yet. If the problem comes from LMax then, you, Multicharts as a professional company you should inform and protect your customers displaying a huge disclaimer on all tick charts saying: be careful you are actually developing and testing your strategy on ticks charts and all your work will be useless as live ticks charts won’t at all correspond to historical ticks charts…

Multicharts is such a powerful and wonderful tool but frankly this sounds to me terribly unresponsible.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby TJ » 13 Oct 2021

Thanks for your answer TJ.

Thanks for your answer Kate as well. But sincerely I can’t understand at all that such an important issue hasn’t been addressed yet. If the problem comes from LMax then, you, Multicharts as a professional company you should inform and protect your customers displaying a huge disclaimer on all tick charts saying: be careful you are actually developing and testing your strategy on ticks charts and all your work will be useless as live ticks charts won’t at all correspond to historical ticks charts…

Multicharts is such a powerful and wonderful tool but frankly this sounds to me terribly unresponsible.
MultiCharts only plots whatever is received.

User avatar
Kate MultiCharts
Posts: 575
Joined: 21 Oct 2020
Has thanked: 7 times
Been thanked: 144 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Kate MultiCharts » 14 Oct 2021

oppidum,

I understand your frustration, but the difference between historical and real-time data is quite common for many data vendors, not only LMAX. MultiCharts is a trading platform which displays data as received from the data feed, so data specifics/warnings should be expected from your data provider.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 14 Oct 2021

Thank you Kate for your message.

I understand that a tremendous complexity is behind all this business, either from your side than from the LMax side. No doubt about that. It’s even hard to understand how all this can work at such a speed and such a reliability.
But on the other hand, considering all this incredible technological achievement, it’s hard to understand how the final result delivered to the customer can be so different. In a rough observation, live tick feed seems to be even less than 30% of the historical tick data. Totally inaccurate and useless for auto trading.

I tested Ticks charts, Contract charts, Change charts, the same problem occurs. As time based charts missed so many information (price action, speed, momentum, micro structure movement) it’s really hard to use it when the strategy has been developed using information that can’t be provided by time based charts.

- Q1: Do you have any information on which broker deliver more accurate live vs historical tick feed?
- Q2: Do you know if there is some kind of ratio of the ticks provided in live context vs ticks provided on historical base, or is it totally aleatory?
- Q3: Is there a way to programmatically have an automatic data reload in Power Language?


I attach a screenshot of the strategy connected to a tiny 7 ticks chart resolution for visualization purpose. There is 2 vertical cyan line representing the same time span. On the top of the image is the strategy having been running on live tick feed for a few hours. On the bottom part, exact same thing but after data having been reloaded. I understand it may have some technical limitation somewhere but it's hard to believe both results come from the “same chart”…

.
Attachments
MC06_strat test (7 ticks)70.jpg
(237.1 KiB) Not downloaded yet

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Mydesign » 15 Oct 2021

Hello Oppidum,

As a matter of fact, what you described is my experience too. Welcome to the real world !

You will never get exact same datas between realtime and history backfill unless you use time based datas. Beside MC data handling and the broker datafeed, there is a third factor that makes it so: the Internet itself. Discrepancies are inherents to the way datas travel on the net.
Take 2 realtime tick charts from same datafeed with 2 different ISP and they will differ for sure. So a fortiori between a realtime and an historical chart.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 15 Oct 2021

Yes, that’s true. As usual, devil is in details!

This was a totally unexpected issue even if now after LMax explanations I understand what is the technical limitation behind that. They also proposed a solution that, at least, may make live tick feed getting pretty close to the historical tick feed. It’s up to be tested but it sounds good. Let see…

For the ones who as well don't know this kind of technical situation, what’s going on is that there is a throttling applied to any account’s API which by default is set to 10 updates per second in LMax. This throttle can be pushed up to 1000 updates per second if required by the client. But apparently a good balance has to be find following the network connection on the client side which has to be stable enough to handle the data throughout, trades, etc. It depends as well on the number of instruments traded. So, client need to confirm he accepts this kind of risk and is warn he has to monitor its connection. As the strategy is working on a server, it normally should work but first they will progressively increase the frequency setup.

It will depend on anyone needs, but in my case, as in your case I presume, it is making live auto trading impossible. To illustrate that, if we compare with the historical tick feed, in a live context, a theorical 2.000 ticks chart could end being a chart with a resolution varying from 3.000 up to a 10.000 ticks depending on the trading activity. This is a rough estimation, but the end result is that it is making the detection of some type of movement too aleatory as they are ending being out of their theorical scale.
Last edited by oppidum on 16 Oct 2021, edited 1 time in total.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Mydesign » 15 Oct 2021

Thank you for sharing these infos. I am a long time LMAX customer and I was not aware of such details.
But frankly speaking I am not as much impacted as you are with data mismatch in my trading (mostly index CFDs on range bar charts). 10 ticks/sec is far enough for what I do.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 26 Oct 2021

After performing some test (*) during 3 days to compare live tick feed and historical tick feed (after reload) , the amount of tick loss is as followed:

- With a LMAX account set up with a 10 updates per second the tick loss varies from 52% to 76%
- With a LMAX account set up with a 100 updates per second the tick loss varies from 32% to 56%
- With a LMAX account set up with a 1.000 updates per second the tick loss varies from 15% to 28%

I know that Multicharts can only treat the data received by the data providers but we are talking of such a tremendous difference that I can’t understand that such an issue hasn’t been addressed in some way. This baffling me that you are developing such a fantastic software leaving that aside without proposing some way to “solve” totally or partially such an issue. We are speaking about the integrity and accuracy of the data stream, the number one requirement to autotrade.

SOME IDEAS:
The first thing that came to my mind is, what about having a live tick stream that is not “wrongly” increased tick by tick but a tick stream that would be increased update per update following the data received from the data provider. If a 4 ticks count move occurs between 2 updates, why not send the next tick as been 4 ticks (through a tick number ID associated to each tick from the data provider)?
- OR -
With the same logic, a synthetic second per second tick stream where tick count is calculated each second and is increased by the amount of ticks that occurred within this second. This way customers could have a much realistic live tick feed (see attached screenshot).


(*) The test was done performing a simple bar count per hours. One was done from the live tick feed and the other one after having reloaded the data from historical feed. Following the trading activity the amount of tick loss is varying, the more activity on the instrument the more ticks are “lost”. As well, different problems occur with the 100 and the 1.000 update/sec threshold : connection issues and problem with some orders rejected (see screenshot attached).
Attachments
rejected orders.png
(146.81 KiB) Not downloaded yet
01_illustration tick feeds.png
(5.48 KiB) Not downloaded yet

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby TJ » 26 Oct 2021

. . .
I know that Multicharts can only treat the data received by the data providers but we are talking of such a tremendous difference that I can’t understand that such an issue hasn’t been addressed in some way. This baffling me that you are developing such a fantastic software leaving that aside without proposing some way to “solve” totally or partially such an issue. We are speaking about the integrity and accuracy of the data stream, the number one requirement to autotrade.
. . .
Maybe you are new to trading?

First, this is not an issue - Everyone knows, FX is a jungle. That's just the nature of the beast.
Second, MultiCharts (or any charting software) has no authority to decide which data is correct and which data is wrong.

Have you compared LMAX data to other brokers' data?

If you found LMAX data to be wrong, you must speak to LMAX.
Have you searched the web for similar LMAX complaints?

I agree, the integrity and accuracy of the data stream is the number one requirement to autotrade.
My suggestion is to find an instrument that trades on a regulated exchange, and a professional datafeed that is known to provide accurate data stream.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 27 Oct 2021

Thanks TJ for your advices. Not new in trading, coding for many years now. Just that there is no real reason to start real live trading until having a strong enough strategy. Though the strat was running live from time to time for checking purpose, that’s only by coincidence I graphically discovered this live feed issue

This problem is not only about FX pairs as the same occurs with other non-FX instruments. It's about the live data streams. If we compare live and historical feeds, price movements are the exact same movements (except in some very rare occasion, but this is not the issue here). The problem is that tick charts (or any other data charts) miss between 50 and 75% of its own data feed and this data feed is crucial as it is used to built it's own bar structure.

In other words, it is like if someone use a 20 minutes charts and then, once in live conditions, bars turn to be 55 minutes, others 40, others 70 minutes.. In those conditions, any strategy based on movement analysis get outside of its logical time, speed, and other kind of structure analysis.

I really appreciate LMax, they are so reliable and fast but from their side, they cannot provide other solution than raising the update threshold. But then, pushing that update threshold create network connections issue which is not acceptable for autotrading either.

I am away now but will open other accounts with other European regulated brokers to check how better it can get. Even if the live and historical feed get closer , I suppose the same will happen. Let's check that with other brokers, but I suppose something will have to be coded to dynamically reduce the impact of this tick loss .

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Mydesign » 02 Nov 2021

After performing some test (*) during 3 days to compare live tick feed and historical tick feed (after reload) , the amount of tick loss is as followed:

- With a LMAX account set up with a 10 updates per second the tick loss varies from 52% to 76%
- With a LMAX account set up with a 100 updates per second the tick loss varies from 32% to 56%
- With a LMAX account set up with a 1.000 updates per second the tick loss varies from 15% to 28%
Do you mean tick loss from realtime compared to historical... or the other way around ? Also, are they just "missing ticks" or just "different ticks" ?

I have to say those numbers are quite shocking to the least, and are far from what I think I experience with LMAX.
I am very curious to know about the protocol you used to gather such results. If you could share it I could try to replicate your tests and see if I get roughly same results.

[Edit] I did some quick tests and I see what you mean...

Realtime datas (US SPX 500 mini):

Image

Historical datas after deleting datas in quote manager, clear cache and chart reload:

Image

Note: I have the option "Build the trade ticks from from Bid and Ask ticks" unchecked.

Yet, since most "missing" ticks are actually redundant (no price change), it does not really impact my trading based on range bars. It might however impact you if you rely on counting ticks. Just beware that in Forex and CFD these ticks are not actual trades and are just reflecting bid or ask price change from the broker. As TJ mentionned, only a true regulated exchange can provide true quotes.
Attachments
LMAX Backfill ticks.png
(47.7 KiB) Not downloaded yet
LMAX Realtime ticks.png
(46.41 KiB) Not downloaded yet

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 03 Nov 2021

Thank you very much MyDesign for the tests you made!


TEST:
On my side, the test was made on a small tick resolution chart (10tick for example) on EUR/USD with a simple indicator calculating the number of bars built every hour . First the calculation was made under live tick feed for several days and then the same calculation was done after reloading the data from historical. The tests were made on a server with a perfect internet connection and a ping to LMAX of 1ms.

If you want to make the test on your side, here is the code of the indicator:. The info is displayed on the chart and as well sent to a text file to be imported as csv to Excel to make the comparison.

Code: Select all

input: barCOUNTfromDATE(211019),barCOUNTfromTIME(17),TESTinfo("PC1_10/s"); ///-------------------------------------------------------------------------------------------------------------------- // BAR COUNT ///-------------------------------------------------------------------------------------------------------------------- var:EXP_exportBARcount_FILEname(""),BARtimeHOUR(0),BARnumATnewHOUR(0),barCOUNTfromTIMEpassed(false); var:dd2nomINSTRUMENT("eu"),dd2infoINSTRUMENT(""),dd2RESO(0),dd2barTYPE("TI"); { - 1.0 - set file info ----------------------------------------------} once begin dd2RESO=barinterval of data1; dd2infoINSTRUMENT=dd2nomINSTRUMENT+NumToStr(dd2RESO,0)+dd2barTYPE; EXP_exportBARcount_FILEname="C:\MC\000000_MC exports\02_BARcountPerHOUR\"+formatDate("yyMMdd",computerDateTime)+"_"+LeftStr(formatTime("HHmmss",computerDateTime),6)+"_"+dd2infoINSTRUMENT+"_BARcountPerHOUR"+".txt"; FileAppend(EXP_exportBARcount_FILEname,"DATE ; endTIME interval ; "+dd2infoINSTRUMENT+" ; "+TESTinfo+" ; BN interval CPT"+newLine); end; { - 2.0 - check if new bar ----------------------------------------------} if barstatus(1)=2 then begin BARtimeHOUR=HoursFromDateTime(dateTime); if BARtimeHOUR<>BARtimeHOUR[1] then begin BARnumATnewHOUR=barnumber; //---- print on screen -------------------------- value1=getappinfo(aiHighestDispValue); value2=getappinfo(aiLowestDispValue); value3=getappinfo(aiRightdispdatetime); value88=RGB(120,120,188); value52=TL_New_BN(currentbar,10,currentbar,0); tl_setstyle(value52,3); tl_setcolor(value52,RGB(60,60,111)); tl_setsize(value52,1); value44=text_new_bn(currentbar,value1-0.04*(value1-value2),"BN="+NumToStr(barnumber,0));//+" nbBARsinceLast="+NumToStr(BARnumATnewHOUR-BARnumATnewHOUR[1],0)); text_setcolor(value44,value88); text_setsize(value44,8); text_setBGcolor(value44,RGB(0,0,0)); text_setAttribute(value44,1,true); text_setStyle(value44,1,1); value44=text_new_bn(currentbar,value1-0.06*(value1-value2),"<< "+NumToStr(BARnumATnewHOUR-BARnumATnewHOUR[1],0)+" >>"); text_setcolor(value44,value88); text_setsize(value44,8); text_setBGcolor(value44,RGB(0,0,0)); text_setStyle(value44,1,1); //----- write to file ------------------------- if barCOUNTfromTIMEpassed=false then if (Date-1000000=barCOUNTfromDATE and BARtimeHOUR>=barCOUNTfromTIME+1 and BARtimeHOUR[1]<barCOUNTfromTIME+1) or (Date-1000000>barCOUNTfromDATE and barCOUNTfromTIME=23) then barCOUNTfromTIMEpassed=true; if barCOUNTfromTIMEpassed=true then FileAppend(EXP_exportBARcount_FILEname,formatDate("yyMMdd",DateTime)+" ; "+NumToStr(BARtimeHOUR,0)+" ; "+dd2infoINSTRUMENT+" ; "+TESTinfo+" ; "+NumToStr(BARnumATnewHOUR-BARnumATnewHOUR[1],0)+newLine); end; end;

DATA CHARTS PROS AND CONS:
The data charts like tick’s charts react to the activity so they are the most interesting to use to identify the structure types of the movements. But as you and TJ have already mentioned, the problem of FOREX is that they do not rely on a centralized data. So, following the liquidity providers and the clients on board, the “stream” intensity may vary a lot. For example, with the historical data available on LMAX we can see on the above image that the activity started to increase a lot from mid 2014 and that is as well something to be taken into account.

Image



LIVE FEED ISSUE:
To come back to the live feed isssue, the price movement is exactly the same, even if sometimes some ticks may be lost on the high and low of the bars, but the real issue here is the “density” of the feed stream. Sorry I am not native English speaker so I am probably not using the more appropriate vocabulary. Anyway, LMAX send some kind of screenshots of the tick activity, those screenshots may be received 10 times per seconds (the normal setup) up to 1.000 times per second. But even with this huge throttle of 1.000 updates per second, still 15% to 28% of the ticks were missing on the tests I made.

Image


Following the orderbook speed and activity, the live tick feed that arrives to Multicharts through the LMax’s API set up at 10 updates/second represents only 25% to 50% of the real tick feed. So, to give you an illustration, a 3.000 historical ticks chart could end up, in live condition, being a chart built with a bar representing an equivalent of a 7.434 ticks from the historical feed, then the following bar could be a 4.185 ticks , the following a 8.566 ticks and so on. We are very very far from the theorical 3.000 tick bar chart...

We could say, well let set up a 1.000 tick chart instead of a 3.000 tick chart in order to “adjust” the chart resolution to this missing tick feed but, the other problem is that the bars size may vary a lot following the data received from LMAX's API because the received data depends on the speed of the market.... No need to go further to describe how this live feed get almost unusable for autotrading as all the strategy's logics get "out of scale” for its movements analysis calculations. On the screenshot montage below we can see that the strategy trend detection calculation between live vs historical feed are totally different.

Image


MyDesign, you used the perfect comment, "those numbers are quite shocking", so shocking that I still think that this cannot be true or that I am doing something wrong....

.
Attachments
MC04_Live vs historical feed.jpg
(60.35 KiB) Not downloaded yet
211103a_BarCount.pla
(8.45 KiB) Downloaded 74 times
MC06_strat test (7 ticks)_for MC.jpg
(200.06 KiB) Not downloaded yet
BarNumberPerDAY.jpg
(88.49 KiB) Not downloaded yet

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Mydesign » 03 Nov 2021

Thanks Oppidum for the insight, and btw I am not english native neither :wink:

You definetly made your point, and it is clear that LMAX live datafeed misses far more ticks then expected compared to historical backfill. I will check with your script anyway. Could it be otherwise ? I don't know... Interactive Brokers for instance deals with this matter by aggragting quotes in time based ticks, so you get less ticks both in realtime and historical but they are the same. On the other hand, Saxo Trader just does the opposite: you get far more ticks in realtime than with historical datas (1 tick / 15s).

Now on a practical side, it is then obvious that one has to avoid using tick charts in this case. Even though the pace or density of ticks can be a usefull info, I personnally would not rely on it for Forex and Cfds markets, just like I would not rely on volume info for the same reasons (not real quotes).
.
Yet, with charts built from ticks (range bars, renko, etc.) I experience this discrepancy between live datas and historical datas. It might produce some differences when backtesting, but nothing tremendously damaging. To the contrary, it is a good way to assess a system robustness and avoid curve fitting: if it cannot cope with slighty different datas, just imagine how bad it will behave on unknown datas !

These discrepancies might also occur when backtesting same system on different PCs. In order to always get exact same backtest results regardless the PC used, I make a daily reload of historical datas that overwrite realtime accumulated datas. This is possible since MC14 to automate that with the "historical data download manager" (in Quote Manager). That way I am certain that I have exact same datas all over my PCs and backtests are identical everywhere.

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 08 Nov 2021

Great to hear your feedback on the way brokers are streaming their own live feed! Even if I am developing for many years, it’s something that I wasn’t aware. Following your comments on the broker you tested, IB seems to have the more “coherent” way to address this issue. But obviously this will have to be tested broker by broker following everyone's need.

Small discrepancies wouldn't be dramatical. As you said, the strategy needs to sustain a good span in resolution change to demonstrate its robustness. Indeed, this is an extremely important point as it allow to multiply backtests results to have a greater statistical bias and to trade live with batch of charts to soften the final equity curve. But here, for live trading, the difference between live and historical stream is so enormous with LMAX (with their normal update rate) that it is almost unusable, at least for the way my strategy is working.

In my case, time-based charts and range type charts doesn’t provide the same performances due to the way the strategy was developed. So, for the moment I will first start to explore a way to “correct” the charts and try to get them closer to what they should be with the historical data feed. I will give a new try with the update rate at 1.000/sec. With such a rate, the average tick loss is 20% (deviation 16 to 28%), which is acceptable to build with some coding a kind of “corrected” charts. The problem is that with this update rate some freeze in connection occurred during the previous tests. Anyway, let’s explore that first as it seems that it will be a necessary tool whatever the broker will be.

Again, thanks for your feedbacks Mydesign!

.

small dog
Posts: 17
Joined: 17 Dec 2020
Has thanked: 1 time

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby small dog » 22 Nov 2021

Not exactly the same issue, but I had quite a rude awakening myself. Tested the forex strategy in TS on TS data, and it was producing good results. Transferred it to the MC with the data from Oanda and it fell apart. I have to compare Oanda and TS data, both live and historical, and see if this is because of the difference in the feed or because TS and Multicharts may be executing the code slightly differently.

In any case, I can see several issues here. Spot forex seems to be a bit of a jungle. The data is not centralised, and the discrepancy between vendors can be substantial. So maybe the solution is to switch to forex futures which are traded and quoted centrally. The commission in futures is also more favourable. Also, as time frame increases the discrepancy between data feeds also seems to be less dramatic, so maybe sticking to longer time frames is mitigating the feed discrepancies.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Mydesign » 23 Nov 2021

Hi Small dog,

As already said, it is a good way to assess a system robustness and to avoid curve fitting: if your system just collapses with slighty different historical datas, just imagine how bad it would behave on unknown realtime datas. Admiting this will save you a lot of money, trust me !

small dog
Posts: 17
Joined: 17 Dec 2020
Has thanked: 1 time

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby small dog » 23 Nov 2021

It's actually a good point, I missed it in the earlier posts. The thing is, the system in question does work on couple instruments on broker's data and falls apart in others. So I have to compare the feeds anyway, to see exactly how different they are, in good and bad instances.

Thanks for the comment.

Bentley Wilburforce
Posts: 18
Joined: 22 Feb 2014
Has thanked: 5 times
Been thanked: 2 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Bentley Wilburforce » 03 Feb 2022

Great to hear your feedback on the way brokers are streaming their own live feed! Even if I am developing for many years, it’s something that I wasn’t aware. Following your comments on the broker you tested, IB seems to have the more “coherent” way to address this issue. But obviously this will have to be tested broker by broker following everyone's need.

Small discrepancies wouldn't be dramatical. As you said, the strategy needs to sustain a good span in resolution change to demonstrate its robustness. Indeed, this is an extremely important point as it allow to multiply backtests results to have a greater statistical bias and to trade live with batch of charts to soften the final equity curve. But here, for live trading, the difference between live and historical stream is so enormous with LMAX (with their normal update rate) that it is almost unusable, at least for the way my strategy is working.

In my case, time-based charts and range type charts doesn’t provide the same performances due to the way the strategy was developed. So, for the moment I will first start to explore a way to “correct” the charts and try to get them closer to what they should be with the historical data feed. I will give a new try with the update rate at 1.000/sec. With such a rate, the average tick loss is 20% (deviation 16 to 28%), which is acceptable to build with some coding a kind of “corrected” charts. The problem is that with this update rate some freeze in connection occurred during the previous tests. Anyway, let’s explore that first as it seems that it will be a necessary tool whatever the broker will be.

Again, thanks for your feedbacks Mydesign!

.
Hi Oppidum.

I am having the same problem with Lmax using renko charts. How do you change this update rate?
Thanks
Ben

oppidum
Posts: 30
Joined: 16 Oct 2014
Has thanked: 31 times
Been thanked: 4 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby oppidum » 04 Feb 2022

Hi Ben,

You just need to contact LMax, explain them the problem (and your disappointment!) and ask them to modify this update frequency rate. They use to reply pretty fast.

You will have to test different setups to find the good balance. But keep in mind that even with the maximum frequency (1.000/sec.), +/- 20% of the "real" data is still missing. Add to this that with such an update rate Multicharts gets too unstable as soon as the market starts to move a bit fast...

Good luck

Jupp25
Posts: 50
Joined: 17 Sep 2021
Has thanked: 13 times
Been thanked: 7 times

Re: Issue: great discrepancy between ticks charts from live session vs ticks charts from historical data feed

Postby Jupp25 » 12 Feb 2022

hi oppidum,

the easiest would be, if you just forget everything considering ticks that comes from LMAX!!!
even other values, they`re data is pretty much "crap"!!! if I put on the "spread indicator" on LMAX and IQFeed, its really opening your eyes about the quality of thei`re data. I am with them myself, but also use IQFeed as second(reliable) data stream. In case you`re german, give me a note and we can talk.

greets
chris


Return to “MultiCharts”