Beta and IB Pacing Violatoin again...  [SOLVED]

Questions about MultiCharts and user contributed studies.
Guest

Beta and IB Pacing Violatoin again...

Postby Guest » 18 Jul 2007

Just installed new beta and when backfilling charts I get the old Pacing Violation again. In the stable version i was requesting data back 3 months at a time and never got a pacing violation once...Now since i downloaed the beta...ITS BACK. Now I know IB has always had that but I was never getting it before today...Did you change something in the way of how MC requests the data or take away a pause between requests or something?

Guest

Postby Guest » 18 Jul 2007

I heard IB has just changed something, again.

Guest

Postby Guest » 19 Jul 2007

I heard IB has just changed something, again.


Please provide information as to where it was mentioned that "IB has just changed something again".

I see nothing mentioned in IB forum, neither did i see anything on TWSapi yahoogroup.

thanks

Nick
Posts: 490
Joined: 04 Aug 2006
Has thanked: 4 times
Been thanked: 24 times

Postby Nick » 19 Jul 2007

The old version of MC seems OK so I doubt its an IB change.

Can someone give a bit more information on this? I'd like to upgrade as I have broken studies but I'd rather put up with that than struggle getting data each morning.

Cheers.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 19 Jul 2007

We haven't changed IB data plug-in algorithm. All we did is added pacing violation message to Event Log. Before the program simply didn't print the message. So you shouldn't worry about it. If you request 3 months of tick data you always get the Pacing Violation message and will have to wait some time.

Guest

Postby Guest » 19 Jul 2007


Guest

Postby Guest » 19 Jul 2007

We haven't changed IB data plug-in algorithm. All we did is added pacing violation message to Event Log. Before the program simply didn't print the message. So you shouldn't worry about it. If you request 3 months of tick data you always get the Pacing Violation message and will have to wait some time.

what kind of joke is this?

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 19 Jul 2007

Please specific and don't make irrelevant remarks.

Nick
Posts: 490
Joined: 04 Aug 2006
Has thanked: 4 times
Been thanked: 24 times

Postby Nick » 19 Jul 2007

I would guess what the poster above means is that MC should never get a pacing violation under normal circumstance. It should never make more than 60 requests in 5 minutes under any circumstance. (unless perhaps you restart it and it loses count). If there are any pacing violations then the algorithm MC uses is violating IB's protocol and so needs altering.

Obviously to get maximum throughput you need to request maximum size chunks on each request I have no clue what restrictions IB places on this.

Cheers.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 20 Jul 2007

Nick,
It seems you didn't read the poster. He said
"In the stable version i was requesting data back 3 months at a time and never got a pacing violation once..."
I said and am saying once again - it is not possible to get rid of this message if you request 3 months of ticks, because IB allows us to make 60 requests. Each request returns 2000 bars maximum.
IB forces us to wait for 10 minutes after each 60 requests.
So you can download 120,000 (1 second, 5 second, 1 minute or any other bars) and then you have to wait for 10 minutes.
It is obvious that if you request any resolution that is less than 2 minutes it will cause at least 1 access violation message.

Guest

Postby Guest » 20 Jul 2007

Nick,
It seems you didn't read the poster. He said
"In the stable version i was requesting data back 3 months at a time and never got a pacing violation once..."
I said and am saying once again - it is not possible to get rid of this message if you request 3 months of ticks, because IB allows us to make 60 requests. Each request returns 2000 bars maximum.
IB forces us to wait for 10 minutes after each 60 requests.
So you can download 120,000 (1 second, 5 second, 1 minute or any other bars) and then you have to wait for 10 minutes.
It is obvious that if you request any resolution that is less than 2 minutes it will cause at least 1 access violation message.

I don't think anybody is asking MultiChart to do the impossible. If IB is offering 60 requests, that's the most we are expecting. Nothing more.

What we, as users, would expect from our charting software are:

1. request data from the feed according to the feed's procedures.
i.e. if IB is offering 60 requests, don't make 61 requests.

2. when the data request is max'ed (e.g. we have 100 stocks in our portfolio), pop a message window to tell us. Don't let us mistaken that MultiCharts is at fault. Don't let us guess whether MutliCharts has crashed? Don't let us believe that MultiChart is stupid. Tell us, tell us by popping up a message window and say: "Hey, you have requested tooooo many backfills, IB is only giving you 60. You must wait for 10 minutes before MultiCharts will make the next request!".

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 20 Jul 2007


1. request data from the feed according to the feed's procedures.
i.e. if IB is offering 60 requests, don't make 61 requests.

.


We make 60 requests and wait for 10 minutes to make the 61st

2. when the data request is max'ed (e.g. we have 100 stocks in our portfolio), pop a message window to tell us. Don't let us mistaken that MultiCharts is at fault. Don't let us guess whether MutliCharts has crashed? Don't let us believe that MultiChart is stupid. Tell us, tell us by popping up a message window and say: "Hey, you have requested tooooo many backfills, IB is only giving you 60. You must wait for 10 minutes before MultiCharts will make the next request!".


I might be mistaken, but I don’t think it is user-friendly to throw 100 messages. A user can go to Event Log and see what is going on. Actually it is the role of Event Log.

Guest

Postby Guest » 20 Jul 2007

I might be mistaken, but I don’t think it is user-friendly to throw 100 messages. A user can go to Event Log and see what is going on. Actually it is the role of Event Log.

I think you are mistaken.
I don't think it is user-friendly to leave the customer in the dark.
I don't think it is user`friendly to expect the customer to weave through all kinds of hidden places to look for the message.
You afraid that it is not user`friendly because there are 100 messages?
why would there be 100 messages? MultiCharts only stops once after 60 requests are made ! why would there be 100 messages ???
if you don't think customer want to have the messages, then make it an option to disable the message.
Let the user decide what is user`friendly.

Guest

Postby Guest » 20 Jul 2007

Andrew:

You start to sound defensive, which is totally not necessary. The purpose and intent of my post was to express a customer's point of view. I have done that, therefore I will not post on this subject again.

User avatar
gautama2
Posts: 96
Joined: 10 Jul 2007
Has thanked: 1 time
Been thanked: 1 time

Postby gautama2 » 20 Jul 2007

User friendly is, if the user gets what he wants.

Beneath the point that backfill is slow, it would be also good if MC saves the data it alreaday got, so it is not necessary to get new data from IB with every change of the timeframe. There are programs that have smarter solutions (AB gets it faster from IB and also from Metastock files).
As MC has smarter solutions in other areas and better features in programming, it is good enough for me in the moment and i am happy for the purposes i use it, but i would like to see improvement in speed of data handling in the future, as data is the base for everything whatever you want to do with a charting and testing software.
There are possibilities for workarounds, but it would be better to have a software that gets the data as fast as possible at least for those, trading systems automatically with it (which is my aim) and are not only backtesting like me. Even if there is no pacing violation, it is too slow, i.e. when getting data from Metastock files it is as slow as getting data from IB. And also eSignal data is gotten much faster with other programs. A friend of mine gets thousands of quotes per second with TS2000i and esignal plugin from 1995.

There are lots of possibilities to become more user friendly. It doesn't depend on messages, that's true. but it is not a good sign, if one sees, that customers wishes are handled in such a way.

Best regards
Robert

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

  [SOLVED]

Postby Andrew Kirillov » 23 Jul 2007

I might be mistaken, but I don’t think it is user-friendly to throw 100 messages. A user can go to Event Log and see what is going on. Actually it is the role of Event Log.

I think you are mistaken.
I don't think it is user-friendly to leave the customer in the dark.
I don't think it is user`friendly to expect the customer to weave through all kinds of hidden places to look for the message.
You afraid that it is not user`friendly because there are 100 messages?
why would there be 100 messages? MultiCharts only stops once after 60 requests are made ! why would there be 100 messages ???
if you don't think customer want to have the messages, then make it an option to disable the message.
Let the user decide what is user`friendly.

If you run many charts you will have many pacing violation messages and some users will complain that it is annoying. I agree that you can check the option and don't have the message pop up again, but usually users don't do it and send us complaints. This is why we thought that Event Log is better and quite an obvious place to monitor activity.
We have given a thought to this issue and found that it is better to show this messages in the status line which could be customized.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 23 Jul 2007

Andrew:

You start to sound defensive, which is totally not necessary. The purpose and intent of my post was to express a customer's point of view. I have done that, therefore I will not post on this subject again.


Your feedback is important and I didn't want to sound defensive. I just explained our vision which is being changed based your feedback. The only requirement is to be specific and relevant.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 23 Jul 2007

User friendly is, if the user gets what he wants.

Beneath the point that backfill is slow, it would be also good if MC saves the data it alreaday got, so it is not necessary to get new data from IB with every change of the timeframe. There are programs that have smarter solutions (AB gets it faster from IB and also from Metastock files).
As MC has smarter solutions in other areas and better features in programming, it is good enough for me in the moment and i am happy for the purposes i use it, but i would like to see improvement in speed of data handling in the future, as data is the base for everything whatever you want to do with a charting and testing software.
There are possibilities for workarounds, but it would be better to have a software that gets the data as fast as possible at least for those, trading systems automatically with it (which is my aim) and are not only backtesting like me. Even if there is no pacing violation, it is too slow, i.e. when getting data from Metastock files it is as slow as getting data from IB. And also eSignal data is gotten much faster with other programs. A friend of mine gets thousands of quotes per second with TS2000i and esignal plugin from 1995.

There are lots of possibilities to become more user friendly. It doesn't depend on messages, that's true. but it is not a good sign, if one sees, that customers wishes are handled in such a way.

Best regards
Robert


1. It has been discussed dozens of times that MultiCharts does save data into the local database. Moreover it has a cache for faster operations with identical resolutions.
2. Could you send me MetaStock files that work slowly in MultiCharts?
3. It is not correct to compare Metastock and IB data feeds. There are two completely different things.
4. It is not correct to compare TS2000i /eSignal plugin (which is developed by our company. It is formerly known as UniServer) with IB, because eSignal works very well.

User avatar
gautama2
Posts: 96
Joined: 10 Jul 2007
Has thanked: 1 time
Been thanked: 1 time

Postby gautama2 » 23 Jul 2007

Hello Andrew,
i sent you some Metastock Files.

I didnot want to compare TS2000i / eSignal with IB feed, but eSignal MC Feed with TS2000i / esignal feed. Per what i heard by a friend of mine, it is much faster than MC / eSignal. But anyway, this doesn't disturb me much, i just wanted to point out that there definitely IS a slowness per my observation and it cannot just be discussed away.

However MC is a great tool and i'm looking forward to the next version.

Regards
Robert


Return to “MultiCharts”