Frequent IB pacing violations - is this expected behavior?

Questions about MultiCharts and user contributed studies.
seneca
Posts: 97
Joined: 02 Apr 2012
Has thanked: 38 times
Been thanked: 22 times

Frequent IB pacing violations - is this expected behavior?

Postby seneca » 15 Sep 2013

Ever since I use MC together with IB as broker and data provider, I get hundreds of Historical Market Data pacing violations when I launch MC. I do not use anything else than minute bar charts (mainly CME Futures and Forex), around 30 symbols in total, and do not use any studies that requests tick or second data.

I ignored this in the past, since I assumed that this is expected behavior. But now I wonder if this is really the case, or if this issue is due to my setup.

I know that most IB customers use an alternative data feed (IQ etc.) due to these limitations, but for my purposes, IB data feed is sufficient. I just want to avoid that some day IB gets fed up with my frequent pacing violations, which by now must go into the millions.

arjfca
Posts: 1292
Joined: 23 Nov 2010
Has thanked: 725 times
Been thanked: 223 times

Re: IB pacing violations - is this expected behavior

Postby arjfca » 15 Sep 2013

Ever since I use MC together with IB as broker and data provider, I get hundreds of Historical Market Data pacing violations when I launch MC. I do not use anything else than minute bar charts (mainly CME Futures and Forex), around 30 symbols in total, and do not use any studies that requests tick or second data.

I ignored this in the past, since I assumed that this is expected behavior. But now I wonder if this is really the case, or if this issue is due to my setup.

I know that most IB customers use an alternative data feed (IQ etc.) due to these limitations, but for my purposes, IB data feed is sufficient. I just want to avoid that some day IB gets fed up with my frequent pacing violations, which by now must go into the millions.
Yes this is a normal behavior

IB as a limit on the historical quote that they allow you to download

Here is a link that I found trough Google
https://www.interactivebrokers.com/en/s ... ations.htm

Martin

seneca
Posts: 97
Joined: 02 Apr 2012
Has thanked: 38 times
Been thanked: 22 times

Re: Frequent IB pacing violations - is this expected behavio

Postby seneca » 15 Sep 2013

Martin,

Thanks for your reply, of course I'm aware of the IB historical data limitations. I just wonder if MC could adopt the frequency of its data requests to these limitations, in case IB is selected as data provider.

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Frequent IB pacing violations - is this expected behavio

Postby JoshM » 15 Sep 2013

Thanks for your reply, of course I'm aware of the IB historical data limitations. I just wonder if MC could adopt the frequency of its data requests to these limitations, in case IB is selected as data provider.
If you use minute bars, and start MultiCharts daily (so little data gap to fill), then there's no need for hundreds of violation messages (I don't get these at least).

I thought there was an option to save (or not save) the data in the database (can't find it on the wiki). Judging from your description, might you have turned that option off?

seneca
Posts: 97
Joined: 02 Apr 2012
Has thanked: 38 times
Been thanked: 22 times

Re: Frequent IB pacing violations - is this expected behavio

Postby seneca » 15 Sep 2013


I thought there was an option to save (or not save) the data in the database (can't find it on the wiki). Judging from your description, might you have turned that option off?
Josh,

I think you are referring to "Utilization of cache for real-time price data" in the QM Data Feed settings? This option is activated in my setting.

What is the rough number of violation messages that you get, when you start MC?
Attachments
QM_settings.png
(49.13 KiB) Downloaded 1377 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Frequent IB pacing violations - is this expected behavio

Postby TJ » 15 Sep 2013

How many charts (symbols times resolutions) do you open at start?

The rules for pacing violation is very specific. The application is merciless: if you exceeded the request, you get time out. There are thousands of users around the world, IB's server does not know any better. They don't single you out for pacing violation.

seneca
Posts: 97
Joined: 02 Apr 2012
Has thanked: 38 times
Been thanked: 22 times

Re: Frequent IB pacing violations - is this expected behavio

Postby seneca » 15 Sep 2013

How many charts (symbols times resolutions) do you open at start?

The rules for pacing violation is very specific. The application is merciless: if you exceeded the request, you get time out. There are thousands of users around the world, IB's server does not know any better. They don't single you out for pacing violation.
TJ,

in my current setup I open 26 symbols at start; for each symbol I have two charts: one chart of 1 minute bars and one chart of 15 minute bars. So in total there are 52 charts. As I do not have any additional tools (market scanner, DOM, T&S) open, I think my data requests are rather moderate.
I just wonder if other users with IB data feed also get these frequent pacing violations or not.
I still do not understand, if there is a "logic" in MC when requesting data, which takes the IB data limitations into account or if MC just requests all data at once, which consequently results in the pacing violations.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Frequent IB pacing violations - is this expected behavio

Postby TJ » 15 Sep 2013

TJ,
in my current setup I open 26 symbols at start; for each symbol I have two charts: one chart of 1 minute bars and one chart of 15 minute bars. So in total there are 52 charts. As I do not have any additional tools (market scanner, DOM, T&S) open, I think my data requests are rather moderate.
I just wonder if other users with IB data feed also get these frequent pacing violations or not.
I still do not understand, if there is a "logic" in MC when requesting data, which takes the IB data limitations into account or if MC just requests all data at once, which consequently results in the pacing violations.
You are way over your quota.

IB is not a "data provider" per se. If you need to track so many symbols, you should subscribe to a dedicated data feed.

Most people who use IB as data feed are futures traders, or occasional chartist. They only request 2~3 symbols at a time, the most.

You can avoid the pacing problem by opening one chart at a time, and waiting a few minutes between each chart.


ps.
A trick to circumvent the time out is to shut down and restart TWS when there is a pacing violation.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Frequent IB pacing violations - is this expected behavio

Postby TJ » 15 Sep 2013

Please note that the pacing violation is for historic data requests only.

Real time data depends on your subscription.

Dirk8
Posts: 26
Joined: 24 Mar 2014
Has thanked: 10 times
Been thanked: 1 time

Re: Frequent IB pacing violations - is this expected behavio

Postby Dirk8 » 27 Nov 2015

I have those error messages also a bit too easy in my opinion.

How is it possible that I have the historical data request pacing violation for many many minutes altough I just start up my desktop/workspaces as I quit them the evening before??

It's very frustrating!
If the data is saved on my computer, why MC have to ask the data again..?

When closing the desktop and MC, I saw the processing of the data so it is saved then, isn't?

Or one does really have to go symbol by symbol to choose 'save the data' for this symbol?

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Re: Frequent IB pacing violations - is this expected behavio

Postby bowlesj3 » 13 Aug 2017

TJ,
in my current setup I open 26 symbols at start; for each symbol I have two charts: one chart of 1 minute bars and one chart of 15 minute bars. So in total there are 52 charts. As I do not have any additional tools (market scanner, DOM, T&S) open, I think my data requests are rather moderate.
I just wonder if other users with IB data feed also get these frequent pacing violations or not.
I still do not understand, if there is a "logic" in MC when requesting data, which takes the IB data limitations into account or if MC just requests all data at once, which consequently results in the pacing violations.
You are way over your quota.

IB is not a "data provider" per se. If you need to track so many symbols, you should subscribe to a dedicated data feed.

Most people who use IB as data feed are futures traders, or occasional chartist. They only request 2~3 symbols at a time, the most.

You can avoid the pacing problem by opening one chart at a time, and waiting a few minutes between each chart.


ps.
A trick to circumvent the time out is to shut down and restart TWS when there is a pacing violation.
I have 82 stocks I am tracking in the scanner with two resolutions at present for a total of (82 X 2 = 164) scanner entries and one fairly small script for each of the two groups. I was getting a lot of these IB pacing violation errors with IB's feed until I set the number of bars in the 15 minute chart to match the scanner's bar setting at 520 bars as well as setting the resolution for the 5 minute bar resolution in the scanner to auto detect. I also have the scanner run every 5 minutes to feed the watch list since 1 minute between scans is too fast at times. Now the errors appear to have completely stopped. Some of the stocks are too thin so I will be removing them. However when the U.S. dollar bottoms I may include some U.S. stocks so it could go higher.

janus
Posts: 835
Joined: 25 May 2009
Has thanked: 63 times
Been thanked: 104 times

Re: Frequent IB pacing violations - is this expected behavior?

Postby janus » 16 Aug 2017

I find I can overcome the pacing violation by resetting the connection to the data farms using Ctrl-Alt-F and/or forcing TWS to reconnect using Ctrl-Alt-R. These keystrokes are performed for TWS, not MC.


Return to “MultiCharts”