Downloading 3200 pieces of data requires 8 hours!  [SOLVED]

Questions about MultiCharts and user contributed studies.
User avatar
general
Posts: 9
Joined: 10 Sep 2013
Has thanked: 1 time
Been thanked: 1 time

Downloading 3200 pieces of data requires 8 hours!

Postby general » 16 May 2014

I inserted 1600 stocks into the PRE-SCAN of marketscanner after market closes.
  • Datafeed Provider: Interactive Broker
    MultiCharts (PowerLang): Version 8.8 Release (Build 9188)
    Resolution: 1-day, 2 bars back
    Watch List: disabled
Downloading 3200 pieces of data (2 CLOSE x 1600 stocks) requires 8 hours!
Any faster approach (without extra charges)?
Attachments
faster-download.jpg
(283.17 KiB) Downloaded 464 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Downloading 3200 pieces of data requires 8 hours!

Postby Henry MultiСharts » 16 May 2014

Hello general,

There are multiple reasons of this behavior, two of them are data feed specific:
1. IB Pacing Violation;
2. IB API sometimes becomes unresponsive - this case has been already reported to IB support - there has been no confirmation/solution for it yet;
3. Some MultiCharts data handling internal issues. We have resolved these issues and improved the data loading algorithms. The improvements will appear in MultiCharts 9.0 beta 1. That should speed up MultiCharts communication with the data providers' interfaces, but that will not solve the reasons outside of MultiCharts (#1 and #2).

User avatar
general
Posts: 9
Joined: 10 Sep 2013
Has thanked: 1 time
Been thanked: 1 time

Re: Downloading 3200 pieces of data requires 8 hours!

Postby general » 19 May 2014

Dear Henry,
Thanks for your fast response.

1. After having the "Pre-scan and watchlist" feature, MarketScanner improves a lot. THanks for MultiCharts' good improvement.
I don't mind suspension every 10mins, due to IB's Pacing Violation. But this is not the major reason of failure.

2. We know IB is very a difficult company to deal with. But can MultiChart improve the REQUEST methodology?
e.g. If a particular symbol is stuck (cannot be downloaded after 3 minutes), MarketScanner will REQUEST data for another symbol.

3. About data pacing voilation, IB API processes only 60 requests per 10 minutes;
In other words, IB API can process 1800 requests in 5 hours.
Now I have 1600 symbols, the download process could not be finished more than 8 hours.
May I know the reason behind? Thanks.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Downloading 3200 pieces of data requires 8 hours!  [SOLVED]

Postby Henry MultiСharts » 19 May 2014

The reasons of this behavior have been already specified in my previous post. The data request methodology will be improved in MultiCharts 9.0 beta 1.


Return to “MultiCharts”