Delay reports & ("Arrival time" versus "Time Stamp" charts)

Proper Subject:
Delay reporting and
"Inaccurate Arrival Time Charts" versus "Accurate Exchange Time Stamp Charts".

==================================================
To vote here is the link.
https://www.multicharts.com/pm/viewissu ... _no=MC-974


Recently I have become aware that if I run a process that steals all the CPU time from MC (and this process runs for lets say a full minute) then when this process completes MC will build a single 10 second bar that has all the data from that minute when this process was running. In other words the 10 second bar is not correctly built to reflect the times that the trades actually took place on the exchange (the ideal situation obviously).

This was about 2 weeks ago I noticed this. I can not remember what I did to correct the bar. I can not remember if I tried a reload. I think I returned the database from the previous day instead and did a backfill.

To correct this problem properly time stamps would need to come with each tick (stamps that are applied to the tick at the exact time that the trade took place on the exchange and using a clock that is set in sync with any atomic clock). Obviously MC would need to be programmed to build the bars using these stamps. I remember that when I first got MC I requested this (I have not put it in PM yet). By doing it this way MC could notify the trader that data is late (or that a process is stealing all the CPU time from MC and the data is not getting to MC as was the case with my problem described above). I seem to remember emailing IB and they said that stamps are available from the exchange and IB's TWS does deliver these time stamps.

The problem is that if I had not noticed the problem I mention above I would have no way of being notified by MC that the bar is incorrectly built. So although I say it is not an MC problem, come to think of it we can for sure say it is an MC weakness. I think it would be better from a trader's standpoint to have the bars always built correctly but with a constant readout as to the amount of delay between the exchange and the user PC. Traders could use this info to decide to hold back on trading until the data starts arriving at a minimum speed they have required. For example, they may want to be notified if a tick arrives that is more than 10 seconds late. Maybe the feature could be enhanced to actually block orders (involving entry into the market) if the ticks are arriving late by a larger amount (for example 20 seconds). Having MC build the bars correctly using time stamps but also not notifying the user of the amount of delay is almost as bad as building the bars incorrectly and not notifying the user. I only noticed the situation because I saw absolutely not movement in the chart prices on the last tick. I kept watching it and there was still no motion. Then I realized that it was probably the process I was running steeling the CPU. The final confirmation was my process finishing and exactly then the prices started arriving for MC to complete the 10 second bar. I was also able to confirm it because there were 10 second bars missing during the time I was running my process and a very long unusual 10 second bar had been created to capture all the missing data. I think it is much safer for the trader to have MC notify us of the delay.

Another advantage of having bar build using exchange times stamps is that all the MC user charts all over the world would always look exactly the same. Now this is the way it really should be. Not only that a true tick by tick replay could be written and everyone's replay should be built exactly the same way.

I am curious. Can anyone see any down side to this (except for the brokers since it may at times reduce the amount of commissions they get if traders are holding back on orders because things have slowed for some reason)? I guess it is obvious that the PC time would need to be in sync with the atomic clock, and that the execution time stamp would need to be checked at exactly the time of arrival to the MC program and lastly that the time zone differences would need to be taken into account. I am not sure how the time zone issue would be best dealt with but I do not think it would be that hard. Each local government has the right to change its day light savings schedule as myne did about 6 years ago. So the user would need a way of telling MC this. Lastly, I know that for many years other users of other software have asked for this in some variation or another but my understanding is that no software has these features. That is why I am curious if there is a downside to it (for the trader) that I am not aware of. From my perspective, I felt it necessary to actually take MC down and do a backfill to get the bars at least as close as possible to what actually went on at the exchange. That is ultimately what all trading systems need to know as far as I can tell.

This would obviously effect 1 minute bars too (and larger). If the last 30 seconds of data for the minute does not come to MC fast enough for any reason (internet problem, service provider problems, TWS problem, any program in an infinite loop or just hogging too much CPU) then that 30 seconds of data will get dumped into the next 1 minute bar. So in this case you would have 2 bars which are not accurate.

If MC is the only software that can do this (as an option of course) then I think it would add to its popularity. As always something like this should be an option and default to the old behavior.

I am just curious if the IB quotemanager "user server timestamps" is actually exchange time stamps and if this is checked on the 10second bars (or any bars) will not distort if there is a delay. If it does work (without creating any bugs) then all that would be needed is a notification of the amount of delay in a constant reading with maybe color coding (green, yellow, red) and maybe a sound notice.

Regarding the Historical backfill data I also notice "TWS bar timestamp". Again the question arrises "is this an exchange time stamp?".

Both default to not checked.

Anyway, I did create a PM request at this link since no one seems to have a problem with the idea.
https://www.multicharts.com/pm/viewissu ... _no=MC-974

Notice I changed the subject name.

I will list the advantages when I have time later. For now here are some delay reporting ideas.

A single number shows the seconds delay. It will of course be changing constantly.
The background has 3 colors (green, yellow, red).
The trader can put the cursor over the number to see their settings for the 3 colors.
To change the settings and the sounds the trader double clicks on the number.
The trader has options for sound including no sound.
Maybe the color can be picked up in the EL code for bypassing orders if the color is red.

Some may be using more than one data source. So I guess you would need a delay number for each one.

Regarding the above post regarding delay reporting here are some ideas to consider.

This is an idea that others may want to discuss (especially if they use volume which I suspect that most do). I think it would be very powerful if the user code could detect the delay and the study could make a decision to skip identical ticks during fast market conditions and not skip identical ticks during slow market conditions. I am thinking that if the user study was to detect a lot of yellow status over a period of time (not red) the study may decide it is a fast market and want to skip identical ticks. When status comes back to green maybe the study would decide it wants to process identical ticks. As mentioned above if the delay is too great (the delay the user has chosen for red) then maybe the study would want to skip issuing new orders and maybe even close out existing positions.

Another thing the user could do is actually color the bars to match the delay (green, yellow, red). Currently I color the 10 second bars when they are too long (red occurs when the bar is equal to or greater than my stop loss setting). This is another way I get notice of a situation where I want to avoid trading for a while. I may decide to duplicate this for time delays and I would then check the delay status if I see a bar that is yellow, pink or red. So maybe four or five delay settings could be chosen rather than just three.

If anyone else trades the E-Mini ES symbol and is using IB and you want to compare your charts to my charts while using the existing time stamp options in the IB quote manage settings then please send me a private message. I will write code to compare our charts quickly and easily. The goal is to find out if our charts are exactly the same using these time stamps. During these tests I will run the process that bogs down my machine creating the delay to MC to see the impact and if our charts are still built the same. In the mean time, I will run some experiments to see what happens when these options are chosen while the program I have that creates the slow down is running (are the bars missing or has the problem been corrected?).


Advantages of exchange time stamps:
** with charts built on exchange time stamps your charts would reflect actually what happened at the exchange rather than vary randomly depending on internet delay factors or slow downs in your machine or network setup.
** exchange time stamp charts would be the same anywhere in the world for the same symbol.
** there really should be no reduction in speed. the If statement should be simple. Delay Reporting Advantages:
** I do not think the added code would create a noticeable delay. I feel most new features should be options so I would assume that if this was ever put in it would be an option.
** delay reporting would allow you to know there is a delay rather than your being completely unaware of the typical delay and any unusual increase in delay.
** delay reporting could be historically stored and studied.
** if made available to EL code you could even backtest it and at least stop orders if the delay is too large relative to normal delays. It may be that such a feature could have current delay and average delay numbers of X number of ticks.
** if made available to EL code and the EL code has commands to change the "bypass identical ticks" then your code could change this parameter during fast markets if you wished.
** If other brokers provide these exchange time stamps MC could be used to compare broker speed especially if the results can be accessed in EL code and written out for historical study.
** If delay reporting was available and you noticed a problem it could more quickly get you to use Ping Plotter to find the source of the problem. I once used this program to determine that my service provider was the source of my feed being cut out and they adjusted my line speed down a bit to correct the problem. Ping Plotter allowed me to speed up getting it fixed because I mentioned I was using the product to my service provider and they immediately gave it priority. Without ping plotter I would have been wasting my time calling IB and who knows how much longer it would have taken to get it fixed.

If I or anyone else thinks of any new advantages I will add them to the list.

I just conducted the test with IB timestamps. At this point at least, I can say it is an improvement over using "time of arrival" timestamps. Here are the details and the results.

What I did:
In the morning before starting TWS and MC;
I turned on the IB, "Real Time Data - Use Server Timestamps"
I turned on the IB, "Historical Data - TWS bar Timestamp".
I started MC (twice actually with two backfills).
This all went well.

At about 9:05 I ran my process that locked up MC before but this time it failed. I think the last time I did it MC had been running all day (taking more and more memory) and my process before created just enough load that MC could not handle it so it stopped bringing in the feed. Okay so this time I decided to actually cut the internet feed so I just umplugged the cable from the modem to the computer where IB and MC is running (simulating an interenet delay).

IB's TWS gave the normal notice of no data. MC of course stopped the feed. After about 2/3minutes I reconnected the cable. TWS took about 3 login attempts before it reconnected and the data started coming back in.

The Results:

10 second bars:
Once the data was back MC caught up on the 10 second bars properly rather than building one large bar to contain all the missing data during the unplugged state. I would say these bars were correct. Whether they are true Exchange time stamps I am not sure. Maybe someone from the MC-Team can answer this. It would be good to know and if they are not maybe a correction can be made since my understanding is TWS can provide true exchange time stamps.

1 Minute Bars:
These were not correct. There was one bar missing. However I did a reload for 1 day back and this problem was corrected. I am assuming it was corrected properly because I checked the "Historical Data - TWS bar Timestamp" check box mentioned above.

Conclusion:
For now at least I think it is better to use the IB time stamps. It certainly avoids having to return the previous days data from the backups and doing a backfill to correct the unacceptable state of the bars that occurs using "time of arrival" timestamps. If these are true exchange based time stamps then the charts are accurate. The real test would be to have two users running with these check boxes turned on and comparing the data over several days using the script I have written (already written and used with Super to do these types of comparisons before when the missing ticks problem occured). The charts should be exactly the same. If they are exactly the same over lets say a week, it still will not answer the question as to whether these are true exchange based time stamps but at least all the IB TWS users will have exactly the same charts if they have these check boxes turned on.

Delay Reporting:
So maybe this corrects the problem. However, I still think it would be useful to have delay reporting to see just how much delay there is between the actual exchange trade and the arrival. I would be interested in knowing exactly how IB's TWS decides there is a problem and puts up that grid. Does it just use lack of data rather than comparing the exchange based time stamp against the arrival time? How much delay does it accept?

It occurred to me that, regarding the 1 minute bar results above, even when the check box is in place, the fact that I had to reload the charts to correct the 1 minute bars is a problem . As far as I am concerned the delay should be reported to the trader by MC and when the data comes in to do the catch up the 1 minute bars should be correct such that the trader should not have to do a reload. I say this because the delay could be smaller and the trader may not realize it has happened. They may have stepped out of the room for just over 1 minute (it is possible obviously). Again, MC should handle it automatically.

So on the positive side, the 10 second bars auto correct and if the trader is aware they need to do a reload, at least they can get it fixed properly. However it could be better.

I tend to agree MC should automatically catch up once a data feed is operational again after a being broken. The only operational issue I would see is what happens when a study is set to automatic order execution? Will any buy/sell/cover/limit/stop orders also be generated during the bars that are automatically updated? Not sure if one can block such orders if required using LastBarOnChart_s. Perhaps GetAppInfo(airealtimecalc) is another way. As a last resort comparing time_s with the computer's time is another way if there's no built-in feature that works.

One thing to note is I don't need to use TWS server time stamps to see historical bars appear correctly (more or less) after a manual reload, or when MC is started (obliviously, otherwise it would be a mess!). I'm assuming MC uses historical server timestamps by default when accessing historical data even though the check box is not ticked. So, I'm confused as to the purpose of that setting. Can anyone explain?

The real-time one does make sense at first but what happens if the computer's time is ahead of the correct time? Can MC actually go back in the data series and adjust a previous bar even though it has already been completed in real-time? I don't think I would want that anyway! I can understand it going the other way. If the computer's time is behind, server based timestamps would be handy.

That's why I prefer to leave things as they are in the default setting and rely on the computer's time to be accurate by using a clock sync utility, and have a low latency internet connection, which is the case almost all of the time. If one really is desperate to have the most accurate arrival times of tick data then they should use a dedicated connection to the exchange or a suitable data provider using say an extranet type connection rather than relying on an internet connection. Such connections are proprietary and expensive, and typically used only by brokers. A serious high frequency trader relying on timely tick data really hasn't much choice but to use such a dedicated connection if such a trader wants sub second order execution at all times. I presume most don't and are probably happy to lose a bit here and there as long as the traditional internet connection is reasonable most of the time.

I think you are likely correct that the backfill of data has to be using the broker server stamps or else the exchange stamps. This is probably why a reload or restart of MC always changes the RSI when I have restarted in the past (because with the check box unchecked the bars have never actually been accurate and only on the reload are they accurate). Now that I am running with both check boxes turned on I may take a snapshot of the charts and do a reload just to see if they now match (if they do I will be happy).

The issue with having the bars built with exchange stamps is to have them reflect exactly what is actually happening on the exchange. The delay and extra fast catch up of data (and how to deal with it) is actually a completely different issue and still is a problem regardless of the type of time stamp being used. The best way to see exactly what I am saying here is leave the check boxes unchecked, unplug your feed for 3 minutes (during the most active time of the day in the market) then plug it back in and see what you get. If you are running 5 second bars you will have one very strange bar that contains 3 minutes worth of data rather than 12 X 3 = 36 bars that correctly reflect what occurred on the exchange. I am going to make a guess that this very long bar that contains 3 minutes of data is much more likely to mess up the EL code logic that the programmer/trader put together than a series of correctly formed bars that come in late. If the correct formation of bars come in the EL code will react correct and all that happens is the order goes out late (not good but like I said, late is late regardless of the shape of the bars relative to each other). I should probably do this test again myself since my test occurred when a process I was running stopped the feed to MC. With today's test (where I had the check boxes turned on) the bars came in basically instantly all at once as far as my eyes could tell and they appear to be realistic and all the times were correct (none of them missing).

Regarding the processing, I was thinking something along the lines of this.

** MC when it first starts automatically forces the PC clock to sync with the automatic clock (an absolute must for this to work).
** MC when it gets the first data does the exchange stamp versus arrival time check and adjusts the daylight savings adjustment automatically. If the delay is normal(whatever, maybe 5 seconds - I am guessing) then it assumes this is a normal delay. If it is extreme (more than a minute) then it does the daylight savings adjustment process and probably should notify the user of what it did. This would involved 30 or 60 minute adjustments until the delay was within the normal parameters.
** Regarding the exchange stamp time being ahead of the atomic clock time, If an execution time stamp came with a time of 08:00:10 and the PC clock time is 08:00:00 MC would have to assume that the exchange clock is not in sync with the atomic clock and make an adjustment (possibly a few of them until it finally needs no more adjustment). Once this adjustment is made then all stamps that are behind are considered the standard late arrival amount. Here this has nothing to do with the building of the actual bar. Bars are always built with the exchange time stamp regardless. This only has to do with delay reporting.
** If there is a longer delay than normal (lets say 30 seconds) then MC has to assume there is a delay in the feed and I would hope the studies have a way of detecting this (a new command of some sort). It would be executed on one of the RecalcLastBarAfter(?) command executions. It could set a switch so that on catch-up of data the User would have control. Each trader would have to think out exactly how they want this handled.

My understanding of how MC works is this. If exchange stamps are being used then I think this is the way things are stored internally (and completely controls the building of the bars). The way things appear on the time scale is a different matter and is a cosmetic thing meant only to keep the trader happy so they understand the time they are seeing.

Now that I am running with both check boxes turned on I may take a snapshot of the charts and do a reload just to see if they now match (if they do I will be happy).

MC uses the real-time data feed while charting but uses the historical data feed when you do a reload (or start up MC). I've noticed the data is different for these two feeds. IB has admitted this is so, and it's a well known problem for some, which IB refuses to fix. So, you will often see different results after a reload regardless of the time stamp settings.

The issue with having the bars built with exchange stamps is to have them reflect exactly what is actually happening on the exchange.

Why? If you want to trade as the data arrives with or without any latency issues, what does it matter what the time stamp says? If you have a trading rule that fires an order, it will be sent regardless. The only relevant issue are the delays in receiving the data and sending the order. That's why high frequency traders prefer a direct feed to the exchange for their data and a direct connection to their broker for trading. I used to have one with a major bank and it was invariably a split second faster to display data and receive quotes than IB.

Why?

Why not extend that to "Why have a chart?". It is true that some traders use time of sales and do not use the chart. But for me I use only the chart (and I have no complaints at all about my trading system after 12 years of development) so I prefer it to be as accurate as possible. Having it distort constantly because of the use of time of arrival stamps makes no sense to me. I rebuilt my database often (major CPU hog) and it seems that with my computer in the afternoon it can distort the charts (because of the delay it creates to the obtaining of the feed of up to a full minute creating 5 missing 10 second bars and one very large 10 second bar). So what you are saying that every time I do this I should restart MC and do a back fill because my trading system will be screwed up by this distorted chart and RSI level etc. I don't think so. It seems that MC is smart enough to rebuild the charts properly using IB time stamps (at least the 10 second bars). I have two more tests to do to see just how good the use of time stamps really is (one with IB stamps and one without - both mentioned above). Yesterdays test looked like a big improvement. So for me if the 1 minute bars would rebuild as well as the 10 second bars the issue is solved. Whether we want MC to do late arrival reporting is actually a very different issue. TWS does gives us the grid and that may be good enough and it may actually be doing the very same tests I am suggesting (data arriving too late is just a variation on no data at all and both involve an amount of time parameter). That question I have no answer for. It is probably an IB question. In short, Time is important. Traders do not use Mondays chart to trade on Tuesday (it is all a matter of how accurate do we want it - I feel the traders should know what the standard is). I am very sensitive to this because with Qcharts the charts often lagged by 15 seconds and up to 3 minutes behind IB's feed (unacceptable). To distort the chart on top of this is even worse.

Using server time stamps, I brought MC down and brought it back up.

The 10 second bar RSI checkpoints matched exactly to the snapshot before bringing it down.
The 1 minute bar RSI were off by about 1 value.

I then recorded a few key values in the 1 minute bar RSI and took the snapshot.
I restarted MC again and this time the values were the same except for the ones built with live data.

So the real-time IB time stamps work for the seconds data but not for the minute data.

It appears like a bug to me since it should match once these check boxes are checked off.


I tried replacing the restart of MC with the reload. It was worse in that repeat reloads continue to give different RSI values. I am wondering if trades can have duplicate times and the closing price of bars is different. If so there should be a number attached for sequence within time.

Why not extend that to "Why have a chart?".

You answered your own question one way, which is true. I knew one trader who always used a quote screen and nothing else. He was a pro and made millions. My point is different; most traders use charts, the time scale is not what they generally watch. They watch for patterns, breakouts, resistance/support lines, trend lines, moving averages, etc. etc. The timestamps on the tick data are therefore irrelevant. The more relevant issue is the data arrives without too much delay, especially for those who use sub-minute charts. We only place a time scale on the chart so we can look back at the historical data. For real-time trading the timing is irrelevant, simply because we can't change it! The time we live and trade in is the real-time and is here and now, regardless of the timestamps.

If as you indicate you need more accurate times for your own work, then again it's irrelevant provided there is no change between the data that's captured in real-time and when it's reloaded from the historical data feed of IB. However, I question IB's accuracy in that regard as do so many other traders. As I pointed out the more important and relevant issue is the data downloaded from IB via the historical data feed is often different from the real-time one. Their explanation is they use different sampling techniques, presumably for performance reasons, which I consider a bogus and an out-of-date excuse in these days of high speed computers and networks. Whether the data is stored in MC's database is always the historical or real-time version, I'm not sure. If it's historical even though the charts shows real-time data, then you are OK. However, it still leaves a problem for those of us who trade off the MC charts in real-time, either manually or using automatic trading rules. The difference between the real-time and historical version of the data can become very important. I've noticed moving averages and other indicators change their appearances slightly after a reload. This is a direct result of the two types of data feeds. It becomes even more sensitive using other types of indicators, such as the Supertrend. In real-time mode, the trend may be shown as up but after a reload the trend is then down. This happens perhaps once or twice a day on a 1-minute chart. It doesn't really matter that much since like all indicators, it does not always pick the right moves anyway. If I ever found one that did I would be a trillionaire

I can relate to one thing you said for sure. On reload the RSI tends to create tops/bottoms which are consistently higher or lower over all of them. This suggests an equal shift in time stamps (I suspect at the connection point between startups of MC or reloads of MC).

My main point is this and nothing else (complexity need not apply here). The trade time is the time that occurs on the exchange. This is the source. So with all the programming ability these days and all the advantages why not use it. By using a consistent time stamp from the exchange all indicators would look exactly the same on all charts all over the world whether they are produced in real time or recreated after the fact. If it is the best thing to do and not really all that hard (relatively speaking) then why not do it.

I can relate to one thing you said for sure. On reload the RSI tends to create tops/bottoms which are consistently higher or lower over all of them. This suggests an equal shift in time stamps (I suspect at the connection point between startups of MC or reloads of MC).

This is the third time I've tried to explain what's going on. I don't understand why you are ignoring it. The real-time and historical data feeds are different. I'll use the response I got from IB some time ago:

"Please understand that historical data and real time market are calculate completely differently. Historical data is captured with 300 millisecond snapshots that are then used to develop the Open, Close, High, Low and Volume values for the 2 minute bar interval of your chart, while Real Time Market Data is calculate with snapshot requests of 250 milliseconds and only change is fired back. Unfortunately you will not be able to directly compare these values due to the way there are calculated."

In other words IB's data feeds does not always provide real tick data! This is not so apparent during relatively slow trading sessions but when the trading gets volatile and busy, the tick data is not real tick data! I checked it myself when I had a true tick based data feed from a major bank. I could see the tick charts were completely different, with the bank's more accurate than IB's.

So, the real problem I'm more concerned with is IB is not being consistent in the data feeds, not the lack of using time-stamps by MC, nor even the way they snapshot the ticks. If MC was changed such that it always used the historical data feed to plot the real-time charts then we might be better off, and we can talk about utilising the timestamps as you suggest. I'm not sure though if this wouldn't cause other issues, such as even more delays in plotting the data over and above the network latency. I suspect it would otherwise why would they offer a real-time feed in the first place?

In any case, if you really are desperate in acquiring the most accurate tick or second based data in a timely fashion at all times then forget about using IB over the internet. You are wasting your time. You will need to use a more appropriate data provider, or even IB if you use a dedicated line. If you pay for a T1 line that connects directly to one of IB's data centers, you will receive the data a quicker than a regular Internet line. Not sure of the tick data though. It may have the same issues as I explained above. In that case you will need to use one of the few data providers that doesn't aggregate and snapshots the ticks as explained above.

Please understand that historical data and real time market are calculate completely differently. Historical data is captured with 300 millisecond snapshots that are then used to develop the Open, Close, High, Low and Volume values for the 2 minute bar interval of your chart, while Real Time Market Data is calculate with snapshot requests of 250 milliseconds and only change is fired back. Unfortunately you will not be able to directly compare these values due to the way there are calculated."

Interesting. Way back (when I first got MC) I emailed IB and asked if exchange time stamps were being supplied and they said yes. What you are describing sounds like a snapshot taken on their server (Not exactly the same answer). I am thinking we asked them a different question. I asked "do they supply exchange time stamps?" (something the Esignal people said they were going to use with Qcharts before I left). You may have simply asked "what is the difference between the two feeds" (something that is irrelevant to the issue actually that I brought up if there is in fact an exchange based time stamp available - see bolded wording below). This is probably why your mentioning the millisecond sampling did not register with me since I am suggesting major surgery to MC and using only one source rather than two or three. I must think like a banker I know who runs the U.S. division of a small bank (the top guy there). He says people come to him with problems and he simply says "I do not want to hear problems. I want to hear solutions (more than one) and what they will cost me". I am sure this type of thinking is a large part of why he is the 2nd from the top in that bank. I do not think of this problem as something that can not be solved so anything outside this realm of thinking just kind of bounces off (In stead of saying "I do not want to hear it" I simply do not hear it). When we think this can't be done for whatever reason we can blind ourselves to solutions (restrict our creativity).

Yes, without the source time stamp coming from the original computer that applies the trade and with an additional numeric count to ensure that two trades occurring at the same time were differentiated (ensuring the open and close tick is always the same tick) then it is unsolvable. Maybe this is a simplification. Maybe during high volume trading there is a split across computers. I also gather there is a cleanup process after the fact but I gather that is much later.

The interesting part is the second bars do rebuilt perfectly with the real time feed (indicating what you are saying I guess). It sounds like the simple solution is to ignore the minute feed and build all MC charts off the real time feed (1 minute to 60 minute and even daily bars for that matter). I am sure this would resolve a lot of across bar mismatch complaints I have heard since I have been using MC. However maybe that is not fast enough. It also may take too much disk space and MC would need to apply the compress or consolidation to the larger bar sizes. This is more along the lines of what I am thinking is the proper solution (the charting softwares do the disk saving consolidation process when the user needs it). Additionally for those who may be interested in this info, why not put an optional vertical line on the charts where this consolidation has taken place. From the line back is consolidated (open, high, low, close volume data) and after that is real time tick data from the live feed. Even better supply a reserve word to give them the last consolidated bar date and time and let them color the consolidated bars if they want. If all data on all charts is built from one source time stamp and only one of the feeds (the lowest level tick feed which seems to be working) I see no reason that indicators should change upon restarts or catch-up after delays. When merging upon restart for the coming day, any time stamp older than the consolidated time stamp (which is a closing price for the bar) is simply rejected. Indicators process consolidated bars with 4 ticks (open, high, low, close) and volume is totaled for the bar. Another way is the standard format with one record containing all this data. I do not see any reason this can not be programmed.

On an added note, the bolded solution above would probably solve the problems and if the time stamp being used is the exchange time stamp all the better. I mention several advantages to this in a post above. For me it is more of an annoyance actually. I use 1 minute bar RSI levels and 10 second bar RSI levels as bounce points on most of my trades. I will not get into the rules. However I will mention I do not like to automate this and I also drop arrows on the 1 minute bars at key RSI levels and I get the RSI accurate to 2 decimal points showing in the text of the arrow. I have it on my todo list to have the EL code recreate these arrows if I have have to restart MC. The question I have now is can I even do this since the RSI is not the same on restart (that is the annoying part). I could store the exact time where the arrow is located but would that be the swing point in the new RSI plot? I do not have the answer at the moment. If it is not then I have to continue what I do now which is to drop them all again manually.

So, it sounds like there will be no change (along the lines of what I would like to see) since it sounds too costly to change and not worth it. On the positive note, as we both mentioned above, this is not the main factor in a trader's success (obviously). Life goes on. For me it is simple - LOL - Never let MC go down during the day nor reload the charts, never run on a machine that is too slow, never run a process that hogs all the CPU away from MC, etc, and I will never know the difference These I can control. Then all I have to worry about is the power going out or an internet interruption.

Bear in mind, the architecture, both at IB and MC, were designed when the internet was at 56k and cpu were single cored and running at less than 1 GHz. You were rich if you had 1 GB of memory.

Bear in mind, the architecture, both at IB and MC, were designed when the internet was at 56k and cpu were single cored and running at less than 1 GHz. You were rich if you had 1 GB of memory.

Yes, I totally agree. I had TS4.0. It was 16 bit and I brought it to a halt with too much code - LOL. Things sure have changed and the rate seems to be speeding up. My complaint is not actually speed. Hardly. To me having MC (tapping directly off IB's TWS) is so much faster than Qcharts that I still feel like I am in heaven. I just feel that with all this improvement in technology these days it really does not make any sense to me that charts are rebuilding differently upon restart and getting distorted upon catch-up after delays in feed. Attaching an exchange based time stamp to every tick that is transmitted to control the build of a chart is a small fraction of the types of data such as movies that are sent over the internet these days (and the programming is nothing compare to sending a man to the moon) so to me it does not seem like an impossible problem. I am always optimistic however. Considering the rate things are improving, I am sure it will be corrected in time.

Regarding, my suggestions for reporting delays (Like I said above) they are not complaints about speed when things are running correctly. However, on the other hand, if the exchange time stamp was available as standard procedure I sure would be interested in knowing how long it takes before I get that tick after it actually trades on the exchange (it is not a complaint but rather a curiosity). But if things are not running correctly the story changes. If the exchange stamp was available why not add in the feature of delay reporting so users know the normal delay they are getting and when to get concerned about any increase in delay (they set the parameters for when they are notified and when they want to stop trading or handle things differently than normal). Again, these ideas too will likely some day be standard considering the rate things are changing/improving.