Enhancement of accuracy of data collected from IB

Questions about MultiCharts and user contributed studies.
Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Enhancement of accuracy of data collected from IB

Postby Fabrice » 16 Jun 2012

IB data feed provides what they call the "5 seconds bars", which allow to get more accurate OHLC data than the real time data feed provides. It allows an almost real-time correction of the live data feed. It is provided by the reqRealTimeBars() function. SC has recently improved their support for this feature, allowing an aggregation of these "5 seconds bars" to the live data feed. As I have never seen anything about it in MC, I believe it is not supported. As accuracy is important, this should be probably implemented and proposed as an option for those wishing to use it.
Regards.

PS:
Reference for reqRealTimeBars : http://www.interactivebrokers.com/php/apiUsersGuide/apiguide/c/reqrealtimebars.htm
These users thanked the author Fabrice for the post (total 3):
JoshMsptraderducktail

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 18 Jun 2012

Hello Fabrice,

This question has been already discussed in the following topic.

tornadoatc
Posts: 8
Joined: 05 Feb 2011

Re: Enhancement of accuracy of data collected from IB

Postby tornadoatc » 24 Jul 2012

Hello Henry,
Your link to a previous forum topic states that MultiCharts will be looking into supporting the mentioned feature.

Can you provide any updates to the status of this request?

Thanks,
ATC

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 25 Jul 2012

Hello tornadoatc,

There are no updates on the status of this request yet. We are currently focused on the implementation of some other important features.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 19 Oct 2013

Hello,

I wake up this old thread, because with the time, I have seen how much it misses to MC.

For the purpose of accuracy, I double check some closes between 5 min data and daily data. For 2013, I have found at least 4 5-min bars that have their close different from the daily close. My reference is TWS. And do you know what ? In TWS they perfectly match.

This leads naturally to the question : for those errors that I have found, how many are still lurking in my 5-min data ?... Impossible to answer, but probably many.

It may be well to advertise on micro-second accuracy. It would be better to already provide us with the most possible reliable minute-data. And IB provides it : it is called "5-seconds correction". I have seen in the last 8.8 beta 2 that some improvements have been made with automatically reloading missing data in live. Implementing the 5-seconds correction should not be far.

Without a full trust of data, what is the value of backtesting ?...

Thanks.
These users thanked the author Fabrice for the post:
hilbert

User avatar
TJ
Posts: 6521
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 957 times
Been thanked: 1883 times

Re: Enhancement of accuracy of data collected from IB

Postby TJ » 19 Oct 2013

Fabrice wrote:Hello,
I wake up this old thread, because with the time, I have seen how much it misses to MC.
For the purpose of accuracy, I double check some closes between 5 min data and daily data. For 2013, I have found at least 4 5-min bars that have their close different from the daily close. My reference is TWS. And do you know what ? In TWS they perfectly match.
This leads naturally to the question : for those errors that I have found, how many are still lurking in my 5-min data ?... Impossible to answer, but probably many.
It may be well to advertise on micro-second accuracy. It would be better to already provide us with the most possible reliable minute-data. And IB provides it : it is called "5-seconds correction". I have seen in the last 8.8 beta 2 that some improvements have been made with automatically reloading missing data in live. Implementing the 5-seconds correction should not be far.
Without a full trust of data, what is the value of backtesting ?...
Thanks.

On your comparison, are you using live data? or historical data? (ie. a chart made with a reload).

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 21 Oct 2013

Hello TJ,

I would have preferred not to say it but the answer is : both.

The double check is done using historical data. It is an indicator that writes the differences between the closes in the Output window.

And for the live 5-min bars, it is from my experience of having spent so many hours looking at MC charts & IB charts side-by-side. Of course, the differences are usually small, 1 tick here, 1 tick there. But when you trade 5-min bar, this is really a problem : 1 tick above or 1 tick below may change all our mind.

As TWS API provides the solution ("5-sec bars"), this serious issue should not exist. I hope it will be taken in account as it deserves : seriously.

Thanks.

PS: and I am really surprised that other IB users do not complain more about this issue...

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 31 Oct 2013

Fabrice, thank you for your suggestion.

Please vote for the corresponding feature request in the Project Management of our web site:
https://www.multicharts.com/pm/viewissue ... no=MC-1224

All feature requests are forwarded to the management of the company and are evaluated in a timely manner. Please note that even though we value your opinion not all requests can be implemented due to the fact that some features do not fit into our current roadmap.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 05 Nov 2013

Hello Henry,
I have voted for the feature request.
Thanks.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 10 Feb 2014

Hello,
When will you take care seriously of this problem ???
Almost everyday day trading is impossible with MC for me just because of this problem. Just have a look on this pictures :
#1 : Competitive platform 5 min chart ---> correct
#2 : TWS 5 min chart --> correct
#3 : MC.Net 100 ticks --> WRONG
#4 : MC.Net 5 min chart ---> WRONG
Wrong because of the spike on the tick chart (or the big black bear bar on the 5min chart) that should not appear. This spike will of course trigger stops, and that should not occur because in TWS and SC they handle it correctly.

I have been asking for you to seriously cope with this problem but you really do not care.

The workaround ? I use SC for day trading. That is just simply pitiful...

Regards.
Attachments
Screen Shot 2014-02-10 at 11.19.06 AM.png
#4 MC.net 5 min chart
Screen Shot 2014-02-10 at 11.19.06 AM.png (6.94 KiB) Viewed 570 times
Screen Shot 2014-02-10 at 11.18.58 AM.png
#3 MC.Net 100 ticks chart
Screen Shot 2014-02-10 at 11.18.58 AM.png (7.94 KiB) Viewed 575 times
Screen Shot 2014-02-10 at 11.18.25 AM.png
#2 TWS
Screen Shot 2014-02-10 at 11.18.25 AM.png (11.81 KiB) Viewed 594 times
Screen Shot 2014-02-10 at 11.18.13 AM.png
#1 SC
Screen Shot 2014-02-10 at 11.18.13 AM.png (7.45 KiB) Viewed 567 times

User avatar
MAtricks
Posts: 789
Joined: 09 Apr 2012
Has thanked: 286 times
Been thanked: 277 times

Re: Enhancement of accuracy of data collected from IB

Postby MAtricks » 10 Feb 2014

Sounds frustrating :(

I know this isn't what you're asking for, but IB has the world's worst data feed regardless of the platform. Why are you using it? When I was with them (worst experience I've had in this industry), I used IQfeed. Its clean data, connects to MC well, and its cheaper. There are still annoyances with the inefficient symbol mapping feature, IQfeeds very different symbols, and just having another vendor.. It would solve your issue though.

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 10 Feb 2014

Fabrice wrote:Hello,
When will you take care seriously of this problem ???
Almost everyday day trading is impossible with MC for me just because of this problem. Just have a look on this pictures :
#1 : Competitive platform 5 min chart ---> correct
#2 : TWS 5 min chart --> correct
#3 : MC.Net 100 ticks --> WRONG
#4 : MC.Net 5 min chart ---> WRONG
Wrong because of the spike on the tick chart (or the big black bear bar on the 5min chart) that should not appear. This spike will of course trigger stops, and that should not occur because in TWS and SC they handle it correctly.

I have been asking for you to seriously cope with this problem but you really do not care.

The workaround ? I use SC for day trading. That is just simply pitiful...

Regards.

Hello Fabrice,

1) What exact version and build number of MultiCharts are you running? (in MultiCharts go to Help tab-> About)
2) What is the version of TWS you are running ? (TWS Help tab->About->Scroll it down so that we can see the version)
3) What is the status of the option “Generate new tick if total volume changes” in QuoteManager->Tools->DataSources->IB->Settings?
4) Which instrument do you use? Please attach a full sized screenshot of your problem chart.
5) How often do you have such bars?
6) Have you contacted your data provider directly regarding the quality of the provided data?

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 12 Feb 2014

Hello Henry,

Thank you for your answer.

1) Latest version of MC.Net 64 bits : 8.8 Release 8595
2) TWS build 942 1e Nov 19 2013
3) "Generate new tick if total volume changes" is unchecked
4) Instrument : futures CAC 40 (symbol is FCExx). At the moment it is FCEG4
5) Almost *everyday* day this problem occurs (again 4 times yesterday - see pictures)
6) Contacting IB ? Forget it. They do not pay attention for technical issues. I have done it in the past. Will never do it again.

I have always seen this problem for months, not to say for years. If it was not so disturbing to my trading, I would not spend the time to bother you with it. I have really more useful things to do than losing time on forums. But I am getting really angry (not to say more…) by having to use SC just for day trading, because I have bought 2 licenses of MC, have spent countless hours learning easylanguage then .Net. And in the end, I cannot use them just because of a problem that you should solve :
1) IB gives the solution (see the beginning of this thread)
2) I have given to you the solution

Do you want to see by yourself ? So easy : just open a 5-min chart with FCEG4 and the same chart with TWS (or SC if you have it), let it run 2 or 3 days. And you will see...

I understand that implementing volume profile and other funny graphical things could be necessary to sell MC. But for my style of trading, these abilities are almost useless toys if each bar cannot be trusted. And it would be really a shame that MC would not be able to draw the same 5-min bars that TWS and SC are able to draw.

Regards.
Attachments
Screen Shot 2014-02-12 at 12.31.06 PM.png
FCEG4 5-min chart with MC.Net (Feb 11 2014)
Screen Shot 2014-02-12 at 12.31.06 PM.png (81.25 KiB) Viewed 576 times
Screen Shot 2014-02-12 at 12.30.18 PM.png
FCEG4 5-min chart with SC (Feb 11 2014)
Screen Shot 2014-02-12 at 12.30.18 PM.png (64.97 KiB) Viewed 571 times

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 12 Feb 2014

Fabrice, have you tried to turn on the option "Generate new tick if total volume changes" and restart all MultiCharts processes? This option is the one that should resolve the data issue you have.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 17 Feb 2014

Hello Henry,

Since your answer, I have turned on the option you have indicated and observed. The problem has moved from the price to the volume. The price seems ok, but now some 5-min bar volume are totally false (much too big). As mentioned before, just put side by side a 5-min chart of FCEG4 made with MC.Net & one from TWS, you will see.

Regards.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 42 times
Been thanked: 30 times

Re: Enhancement of accuracy of data collected from IB

Postby Fabrice » 18 Feb 2014

@MAtricks

I agree with you about the low quality of data from IB. I have seen several times inconsistencies between their 5-min close and the daily close. In this case, MC has no responsibility. But here is a different issue. MC should be able to do what SC does because :
1) MC officially supports IB as a data feed
2) MC should be able to draw a "simple" 5-min bar chart without any issue regarding the OHLC or volume of each bar. When one claims to have the precision of millisecond, displaying such chart should be trivial.
3) With the same data feed, SC does the job. So should MC.

Regards.

User avatar
Henry MultiСharts
Posts: 8383
Joined: 25 Aug 2011
Has thanked: 1200 times
Been thanked: 2679 times

Re: Enhancement of accuracy of data collected from IB

Postby Henry MultiСharts » 18 Feb 2014

Fabrice wrote:Hello Henry,

Since your answer, I have turned on the option you have indicated and observed. The problem has moved from the price to the volume. The price seems ok, but now some 5-min bar volume are totally false (much too big). As mentioned before, just put side by side a 5-min chart of FCEG4 made with MC.Net & one from TWS, you will see.

Regards.
We will check that.


Return to “MultiCharts”