Interactive Brokers real-time datafeed...

Questions about MultiCharts .NET and user contributed studies.
MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Interactive Brokers real-time datafeed...

Postby MidKnight » 05 Mar 2014

https://www.multicharts.com/pm/viewissu ... no=MC-1224

There are 7 votes for this PM entry. I would like to know why this hasn't been addressed yet. Any charting vendor I've used that supports IB as a datafeed has had this addressed years ago.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 44 times
Been thanked: 30 times

Re: Interactive Brokers real-time datafeed...

Postby Fabrice » 06 Mar 2014

Is it really necessary to say something more, again, again, and again ?…
http://www.multicharts.com/discussion/v ... =1&t=10502

Yes it is absolutely necessary. And this should have been implemented as soon as IB had published the API… yes, years ago...

Regards.

MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Re: Interactive Brokers real-time datafeed...

Postby MidKnight » 06 Mar 2014

I don't check the normal MC forum so I didn't know about it. So this was discussed 2 years ago and still this PM request is an "not determined" state. How could something so crucial remain like that for 2 years.

There are a lot of benefits to using IB as a datafeed. The most obvious to me is that its competitively priced and offers the most markets that I know of across the globe.

Trading is my business and the MC team have become my business partner. It is a risk to my business when my business partner fails to address such crucial issues promptly. I don't think it is unfair to want to know why this hasn't been addressed yet or even properly assessed as shown by the PM - especially now that I see its been discussed several times over the last 2 years.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 44 times
Been thanked: 30 times

Re: Interactive Brokers real-time datafeed...

Postby Fabrice » 06 Mar 2014

Note : as an immediate possible workaround, turning on the option "Generate new tick if total volume changes" (as indicated by Henry) has greatly improved the exactness of price bars (but moved the problem to the volume). "Greatly improved" but not corrected, because the chart is still not accurate : see the TWS and MC.Net 5-min bar charts of FCEH4 - the differences can be easily spotted.
Attachments
Screen Shot 2014-03-06 at 2.47.55 PM.png
FCEH4 as displayed by MC.Net
(23.89 KiB) Downloaded 2988 times
Screen Shot 2014-03-06 at 2.47.43 PM.png
FCEH4 as displayed by TWS
(46.81 KiB) Downloaded 2975 times

MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Re: Interactive Brokers real-time datafeed...

Postby MidKnight » 10 Mar 2014

It has been recommended to me by Henry to uncheck "Generate new tick if total volume changes". When I had it checked, the last price was often out of sync with what TWS was quoting as the last price.

So here I have some comparisions between TWS charts and todays HSI futures. "Generate new tick if total volume changes" is unchecked. There are price differences all over the chart, especially at swing points and high volatility points (such as the days open). The most obvious difference is the HOD made on the very first bar. There is a 49 tick difference.
Attachments
HSI 2014-03-10 MCnet.png
(14.35 KiB) Downloaded 2963 times
HSI 2014-03-10 TWS.png
(25.24 KiB) Downloaded 2970 times

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Interactive Brokers real-time datafeed...

Postby JoshM » 10 Mar 2014

So here I have some comparisions between TWS charts and todays HSI futures. "Generate new tick if total volume changes" is unchecked. There are price differences all over the chart, especially at swing points and high volatility points (such as the days open). The most obvious difference is the HOD made on the very first bar. There is a 49 tick difference.
How did you collect the IB data for both charts?

I'm just brainstorming here, but IB's historical data can be (quite) different from the IB real-time data. In other words, if your MC chart is collected through real-time data, but the IB TWS chart is based on historical data (or vice versa), differences are more likely.

Seems to be related, as I understand it, to the 763 PM issue.

MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Re: Interactive Brokers real-time datafeed...

Postby MidKnight » 10 Mar 2014

Hi JoshM,

Both charts were collected real-time!

I'm well aware that historical prices will differ from real-time prices. That happens with esignal too and isn't unique to IB data. I'm not sure if it is related to that PM you list. The PM item #2 is definitely true and is not an issue with the TWS charts.

My understanding from various forum posts around the net, is that the IB TWS charts real-time will use the "true data" stream if it needs to amend bar highs and lows. I have not seen official comment from IB on exactly how it is done though.

For comparison from yesterdays real-time data collection, here is the same day 5min chart of Hang Seng Futures. Magically the opening bar high now matches what I posted from the TWS yesterday.

I hope this highlights the magnitude of the problem. The more volatile price is, the more obvious and frequent the problem will arise. If you were looking at the MC.net real-time chart you would have no idea that the true high of day was in fact another 49 ticks higher than was shown in my real-time chart.
Attachments
HSI_2014-03-11_MCBackfill.png
(16.83 KiB) Downloaded 2955 times

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Interactive Brokers real-time datafeed...

Postby JoshM » 11 Mar 2014

For comparison from yesterdays real-time data collection, here is the same day 5min chart of Hang Seng Futures. Magically the opening bar high now matches what I posted from the TWS yesterday.

I hope this highlights the magnitude of the problem. The more volatile price is, the more obvious and frequent the problem will arise. If you were looking at the MC.net real-time chart you would have no idea that the true high of day was in fact another 49 ticks higher than was shown in my real-time chart.
I can understand your annoyance with this, but I'm not sure if this is a MC .NET problem or IB problem. For example, N**T seems to have the same problem with IB data (see, amongst other threads, here and here).

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 44 times
Been thanked: 30 times

Re: Interactive Brokers real-time datafeed...

Postby Fabrice » 11 Mar 2014

The last time I have compared 5-min bar charts made with SC, NT and MC, NT was the worst of all. It was about 1 year ago. And besides this, NT - and SC - are not able to backfill efficiently IB data.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Interactive Brokers real-time datafeed...

Postby Henry MultiСharts » 14 Mar 2014

Dear users,

MultiCharts is not in charge of the data that is streamed by the supported data providers.
MultiCharts cannot filter, enhance and fix all the incorrect data that is send by the data provider.
This is the data provider who is in charge of streaming adequate data.

Interactive Brokers provides different data in TWS and to third party applications.
In TWS they have their own filters that cannot be accessed via API.

IB data you can see in MultiCharts is exactly the same data that can be seen in IB API example and that is streamed to all third party applications. We have added filters for the critical issues with IB data reported by our users. The filters are turned on when you enable the option “Generate new tick if total volume changes”.

The PM entry you are referring to is not an issue of MultiCharts. This is an improvement or a new feature for the existing data connection. Please note that even though we value your opinion not all requests can be implemented due to the fact that some features do not provide any actual improvement of the current behavior or do not fit into our current roadmap.

We have tested IB 5 second bars. This is a realtime data snapshot coming once in 5 seconds.
How do you suggest using it? Do you want to receive a realtime tick bar once in 5 seconds instead of every second? We do not think that is a good idea.

Do you want to merge 1 second bars and 5 seconds bars to form a single data series? Or 5 second bars should filter the 1 second bars? This is not a trivial task and there is no obvious algorithm for such functionality.
If you have an idea how to implement this fucntionality and actually improve the quaity of the data received - please feel free to share your thoughts.
Attachments
!ib-5secRT.png
(80.68 KiB) Downloaded 2964 times

MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Re: Interactive Brokers real-time datafeed...

Postby MidKnight » 16 Mar 2014

Hi Henry,

Thanks for asking for user feedback. I need to think about your questions before responding.

IB data you can see in MultiCharts is exactly the same data that can be seen in IB API example and that is streamed to all third party applications. We have added filters for the critical issues with IB data reported by our users. The filters are turned on when you enable the option “Generate new tick if total volume changes”.
While MC may present the same data that is available from the API, there is also more that can be done through the IB that MC is not utilizing - that is the point of this thread and the long ago submitted requests in the PM. I know there is more because I have a good friend who has coded an integration of the 5second true data with the real-time feed in SC all via the IB API. BTW, SC have supported the 5second true data for years.

With regards to your "Generate new tick if total volume changes" turning on the filters, well whatever magic is going in there is just utterly wrong. As I pointed out in a previous thread where you told me to uncheck this option - when that option is enabled, the prices on the chart do not match the IB DOM. So my conclusion from that was, with that option enabled, it is actually dropping ticks. It wasn't just the odd one every now and then, it was nearly continuous all day long and made working a limit order extremely confusing. Maybe this is not an issue in a heavily traded market and would go unnoticed, but in a thinner market like the Hang Seng Futures, it was extremely obvious.

I'll think about your previous questions over the next couple days.

With thanks,
MK

MidKnight
Posts: 343
Joined: 12 Aug 2012
Has thanked: 123 times
Been thanked: 56 times

Re: Interactive Brokers real-time datafeed...

Postby MidKnight » 17 Mar 2014

So I just spoke to my friend (alias kiwitrader) that has coded this himself to work within Siera charts. Here is what he said to me.
Here is the easiest way.
1. print bars to chart up until the last time interval based purely on the true data
2. print the current bar based on the highest of the two highs, the lowest of the two lows, the true data open if its arrived, the live data close
3. when the first true data packet for the next bar arrives, check that the last bar maps to the true data for that bar (just in case something silly happened)
Note that bars start printing up to 5 seconds before the first true data packet for tha bar arrives.
He also says his code will modify the most recently finished bar if the true data bar gives a different high/low/close and modify the current bars open/high/low if the true data is different. Of course, this modification will happen about 5 seconds after due to that is how the true data works.

You could choose to make the true data available to the users and save it under a different symbol name (siera append -TD to true data streams for their local DB) and then leave it up to the user to merge the data through a custom data series. But due to how MC works this would be a problem for users to code indicators on this custom data series because we cannot create this custom OHLC indicator on the chart as a real data series like you can with many other platforms.

Fabrice
Posts: 182
Joined: 14 Jun 2011
Has thanked: 44 times
Been thanked: 30 times

Re: Interactive Brokers real-time datafeed...

Postby Fabrice » 19 Mar 2014

Hello Henry,

Thanks for asking us about this.

I think the objective is quite simple : gives the ability to MC users to have the same minute-chart than TWS, if they wish. I don't think you have to bother with tick chart but I may be wrong, and that may also depend of what is sent by the API providing the 5-sec correction.

It is true that the 5-sec correction introduces a few seconds lag that can make it unusable for trading for some people. For me, it is not a big issue, because my understanding of the market and my decision to place a trade is much more dependent of the exactness of several bars than just the last one.

In the end, MC users would have the choice to have minute charts :
1) made just like today
2) with the 5-sec correction (true data)

The workaround you have given to me (about "Generate new tick if total volume changes") is very satisfying. This issue becomes less urgent to me. Nonetheless, I think that you should implement it for at least 2 reasons :
- some of your competitor do it
- it will put MC one step above its competitor (just like having Volume Profile/Breakdown natively)

The goal is simple : just displays 3 5-min charts of the same symbol, 1 chart with TWS, 1 chart with your competitor and 1 chart with MC enabling the 5-sec correction. When they all are the same, you have won.

Be sure to make it as an option. The way data from IB and charts are displayed currently should still be available.

Best regards.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Interactive Brokers real-time datafeed...

Postby Henry MultiСharts » 19 Mar 2014

Thank you for your suggestions. They have been forwarded to the management of the company. Please note that even though we value your opinion not all requests can be implemented due to the fact that some features do not fit into our current roadmap.


Return to “MultiCharts .NET”