IB Backfill performance

Guest · Postby **Guest** » 04 Apr 2006

I noticed that if i reload multiple symbols at the same time from IB,
from the quote manager log, it looks like multicharts is backfilling the symbols 1 at a time.

Why can't multiple symbols be backfill in parallel ?

Is this a restriction of IB or multicharts ?

thanks

Chris · Postby **Chris** » 04 Apr 2006

I have the same experience, even when I open a new chart with a new symbol the symbols already open stop updating. The strange thing is that Bracket Trader keeps showing changing quotes while the charts in MC don't.

It's true that IBs backfill is very slow, but there is something to imprive in MC, too.

Thank you,

Chris

Guest · Postby **Guest** » 05 Apr 2006

Yeah multicharts backfill from IB is really slow.

I have quotetracker ( for free ) and it backfill multiple charts from IB in seconds !!!

Alex Kramer · Postby **Alex Kramer** » 05 Apr 2006

The IB API limitation that gets in the way here is that you cannot sent a second request untill the first one has been confirned; but for this, we could request immense amounts of data in parallel.

As to the professed speed advantage - only a side-by-side test results can be convincing. Every tool that connects to IB data uses the same API.

R2D2 · Postby **R2D2** » 05 Apr 2006

could the reason for the fact that quotetracker backfills very much faster than MC be that MC always receives one minute data from the IB servers and that causes much more requests than maybe quotetracker which maybe only requests the timeframe which the chart is set to ??(when 15 min is set then mc does 15 times more requests ?!..)

i dont know how QT does it but maybe this would be a consideration ?

ScottB · Postby **ScottB** » 25 Apr 2006

I have written thousands of lines of code to the IB API and in my opinion the issue is as you stated, MC issues multiple one minute requests. The preferred way of doing it is to calculate the length of time and then request the entire data series. The request states the length bars desired from one second to weeks, therefore you can get the data in one minute bars.

My internal application (a strategy execution application) backfills the data very quickly using this approach. Alex is correct in stating that IB will only honor one historical data request at a time, which does make multiple charts load more slowly.

Scott

Alex Kramer · Postby **Alex Kramer** » 25 Apr 2006

What we are able to receive with the current MultiCharts is: up to 2000 ticks per request (API limitation that cannot be overcome), and up to 4 hours of minute data per request (API allows up to a day).

API takes a long time to process requests, which slows the overall process, and reliability is THE problem - sometimes you open the original IB TWS and see no realtime data incoming.

Guest · Postby **Guest** » 25 Apr 2006

Alex you are correct on all points. I am not downloading tick data and never more than a day's worth of minute data so mine is very fast. I have also experienced the issues with IB relative to missing data, etc.

Thanks for your input.

Scott

MPtrade · Postby **MPtrade** » 26 Apr 2006

Alex
can you build minutes charts from tick data in MCH database or we have to download separate ticks and minutes data?

Alex Kramer · Postby **Alex Kramer** » 27 Apr 2006

Yes, this is possible - for instance, if the data source provides tick data only, ticks will be used for creating minutes and other time-based resolutions.

Guest · Postby **Guest** » 28 Apr 2006

I have a problem with minute bars . I have a data in ticks and displays properly no problem but when I try to build minute charts in of line mode
there is only one bar coming from databank. I'm using IB as data provider.
Thanks

Alex Kramer · Postby **Alex Kramer** » 01 May 2006

Please detail on this, I cannot seem to reproduce this issue.

Here's what I just tried:
Cleared the cache and deleted the IBM-IB from NYSE symbol, created it again.
Plotted a 1 tick chart in On-Demand mode.
Closed the chart, switched to Offline mode, created a 1 minute chart - it was created at once from tick data.

What were you trying to do, with what symbol?

Guest · Postby **Guest** » 01 May 2006

thanks for prompt response
I'm working with ES and ZB trying to get 30 min bars
I will try wha t you did an d let u know
thanks

Guest · Postby **Guest** » 02 May 2006

What we are able to receive with the current MultiCharts is: up to 2000 ticks per request (API limitation that cannot be overcome), and up to 4 hours of minute data per request (API allows up to a day).

API takes a long time to process requests, which slows the overall process, and reliability is THE problem - sometimes you open the original IB TWS and see no realtime data incoming.

Looking at the log in quote manager it appears multicharts is requesting minute data in 1-3 minute chunks not 4 hours! This is the main reason refresh is so slow. I guess this is a bug if it is meant to be 4 hour chunks?

Couple this with the fact that data is reloaded from the feed whether it is in the cache or not and this adds up to a serious pain.

Alex Kramer · Postby **Alex Kramer** » 02 May 2006

Could you please send the logs from QuoteManager (in Quote Manager Event Log right-click -> Send Log).
We'll see if MultiCharts requests all the data again (also please indicate for what symbol, in waht resolution and fro what period of time you had that data already cached).

IB Backfill performance

IB Backfill performance

comparison QT and MC ..

mptrade