Missing data - EURUSD 13/05/08 - 14/05/08

MKlein · Postby **MKlein** » 19 May 2008

There's a big chunk of data missing in EURUSD from about 0000H GMT 13/05/08, about 24h of missing data in all, but only in the n-minute timescales. It was there this morning - it's not there any more. Could it be reinstated please?

hedonQ · Postby **hedonQ** » 19 May 2008

same with cable. here's the time sequence on the chart:

05/14-20:00 ---> 05/15-20:01 to 5/15-23:59 ---> 05/16-00:00

data for 5/14 after 20:00 and 5/15 before 20:00 is missing.

George.

Marina Pashkova · Postby **Marina Pashkova** » 21 May 2008

Hi Guys,

Thank you for reporting the problem.

Did you try closing and running MCFX again?

Thank you.

Fabrice Daniel · Postby **Fabrice Daniel** » 21 May 2008

No the data are missing for everybody for several days now. Even after restarting the computer.

MKlein · Postby **MKlein** » 21 May 2008

Did you try closing and running MCFX again?

Did you try looking at the charts?

Marina Pashkova · Postby **Marina Pashkova** » 23 May 2008

Hello everybody,

The problem has been confirmed. It will be corrected over the weekend.

Thanks for reporting!

Fabrice Daniel · Postby **Fabrice Daniel** » 23 May 2008

I suggest you to put in place an automatic detection system. It's very easy to identify such problem with a scricpt into MCFX counting the number of 5, 15 min and 1 hour bars per 24 hours (and 21 or 22 hours for Friday) as well as the consistency of the daily bars (one per day in the week).

A not perfect but simple and efficient tool which cost less than 1 hour of work for an Easy Langage Newbie ...

Marina Pashkova · Postby **Marina Pashkova** » 27 May 2008

Dear Fabrice,

Detecting gaps is easy. The problem is to eliminate the cause of those gaps and make sure that no data gets lost in future. This is exactly what our engineers are working on right now.

Regards.

Fabrice Daniel · Postby **Fabrice Daniel** » 28 May 2008

I understand, but my suggestion purpose was to avoid having a customer report on the 19 May and a confirmation of the problem four days latter ... on the 23 May ...

Hoppefully no MCFX user is a real trader ...

...

And hoppefully nobody paying $99 / month for a 24 instruments data feed (whow ... that's probably the highest cost per instrument ratio in the world) ... is waiting for a service resolving the problems on data in less than 4 days ...

So please improve your data feed correction process ... errors exist in all system but one of the main difference between free and paying services is how fast the issues are fixed. And it's not a question of days but a question of hours (2 hours is a maximum).

Marina Pashkova · Postby **Marina Pashkova** » 29 May 2008

Hi Fabrice,

Our prices are totally reasonable. Could you please name a charting product with the functionality comparable to that of MCFX, supporting a built-in data feed and costing less?

As for the problem we've been discussing, it will be solved over the weekend - because the fix will involve stopping the server.

Regards.

MKlein · Postby **MKlein** » 30 May 2008

I think your prices are reasonable - certainly for the Basic version to which I subscribe - but let's not get away from the main issue. I reported the missing data on Monday. It took two days to receive a dismissive reply when you might have simply looked to see if the data was in fact missing. It then took a further two days to confirm the problem. I know you must have pressures on your time, but that response doesn't inspire confidence. A simple automated system could have alerted you to the missing data before it came to the notice of your customers.

MKlein · Postby **MKlein** » 30 May 2008

There's another gap now, in GBPUSD (I haven't checked other pairs) 12 05 08 (DD MM YY) - no data at all between market open and 1200GMT in minute and hourly charts. That also affects the open price of the daily bar.

David R · Postby **David R** » 03 Jun 2008

Regarding price, I agree that MCFX is very well priced. I used the $99/mth as I use some automation. However I would pay $500/month to get solid reliability.

The reason I have not switched to another chart provider, is that MCFX is in fact my favorite. It is intuitive (mostly), a pleasure to work with, and it can perform some tricks that I have only seen TS do (for way more money).

I just hope they can get some of these ugly issues resolved sooner than later.

MKlein · Postby **MKlein** » 04 Jun 2008

I would pay $500/month to get solid reliability.

Now let's not get carried away!

Marina Pashkova · Postby **Marina Pashkova** » 09 Jun 2008

Hi guys,

The uptime of the server that you have been using up to now has been 97%, which is much better than the performance of the old server.

The uptime of the new culster of servers will be close to 99.9%, which is a highly reliable performance.

Best regards.

MKlein · Postby **MKlein** » 09 Jun 2008

The uptime of the server that you have been using up to now has been 97%, which is much better than the performance of the old server.

97% may not sound bad, but that translates as 43 minutes of downtime per day, if I've understood your terms correctly. I think you must be aware that nobody who takes trading at all seriously would even consider a product with that level of unreliability. If MCFX had revealed those figures at the time I subscribed, I would be with one of your competitors now.

The uptime of the new culster of servers will be close to 99.9%, which is a highly reliable performance.

Could you explain briefly your basis for that estimate, and quantify "close to"?

I don't mean to be the eternal critic. I know you must all be working very hard, and it's a great relief that the server cluster is now available. I wonder, could you post regular server performance updates? It would be reassuring to have a figure to look at once a week or so.

MKlein · Postby **MKlein** » 09 Jun 2008

Could you explain briefly your basis for that estimate, and quantify "close to"?

Ah, 100 minus 3% of 3% ?

David R · Postby **David R** » 09 Jun 2008

It is actually fairly difficult to achieve "3 Nines" of uptime (99.9%). If our TS Support friends can achieve that I will be happy. According to my calculator, that is 45 seconds of downtime in 24 hours. I can live with that. This morning so far so good.

MKlein · Postby **MKlein** » 09 Jun 2008

It is actually fairly difficult to achieve "3 Nines" of uptime (99.9%). If our TS Support friends can achieve that I will be happy. According to my calculator, that is 45 seconds of downtime in 24 hours.

Well, I make it 86 seconds (24 x 60 x 60 x 0.001), but yes, that would be quite acceptable. Especially if those 86 seconds were at 0200 GMT, and not immediately following Non-Farm Payrolls...

Marina Pashkova · Postby **Marina Pashkova** » 10 Jun 2008

Hi guys,

99.9% of uptime is totally realistic. For example, when we didn't have too many customers using the server, it ran without any interruptions (even for a few seconds) for several weeks. This uptime level is even easier to achieve with the new cluster.

The major problem so far has been that, during highly active markets, there have been delays in real-time quotes. However, now, due to the changes in the way the servers process the requests (real-time quotes are assigned the top priority and are processed before any historical requests) and also due to the improvement in the overall server performance this problem has been solved.

The only thing that might cause unpleasant glitches is that FXCM sometimes stops to send quotes for up to 5 minutes. But this a problem on MCFX end and there's nothing we can do about it.

Also, if you notice any problems with the data on the new cluster, please let us know right away.

Regards.

David R · Postby **David R** » 10 Jun 2008

I think you meant this is a problem on FXCM's end

Thanks a bunch for the auto-trading. So nice to see my scripts trading all night without me (demo of course).

I have noticed a few things with the autotrading, but I will detail those in another posts.

Cheers, David

that FXCM sometimes stops to send quotes for up to 5 minutes. But this a problem on MCFX end

Marina Pashkova · Postby **Marina Pashkova** » 11 Jun 2008

I think you meant this is a problem on FXCM's end

You're right. My bad. And thank you for the positive feedback!

MKlein · Postby **MKlein** » 16 Jun 2008

Also, if you notice any problems with the data on the new cluster, please let us know right away.

GBPUSD is missing a chunk of data from 0900GMT through 2000GMT on the 12th of this month.

USDCAD has over 24 hours missing from the 9th to the 10th.

USDCHF and GBPJPY have gaps from 12th to 13th...

The minute/hour discrepancies in historical data are also much worse this week in every currency. While last week's data is generally good, hourly data prior to that is all out of whack.

Marina Pashkova · Postby **Marina Pashkova** » 16 Jun 2008

Hi everybody,

The gaps in the data from the new servers have been closed.

Let us know if you notice more gaps.

Thank you!

Regards,

Marina.

MKlein · Postby **MKlein** » 19 Jun 2008

The gaps in the data from the new servers have been closed.

Let us know if you notice more gaps.

Thanks Marina.

I'm seeing some strange behaviour from USDJPY and USDCHF this morning. In the shorter time scales (e.g. 5, 10, 15, 20, 30 minute) MCFX will download far fewer bars of data than requested if specified in "bars back" form. For example, if I ask for 400 5 minute bars, I only receive 16.

It's fine if I ask for "days back".

edit: in fact I was mistaken - there are problems in both formats.

Marina Pashkova · Postby **Marina Pashkova** » 19 Jun 2008

Dear MKlein,

Try installing the latest MCFX beta 2 which was released on June 17.
The problem isn't present in that version.

Regards.

MKlein · Postby **MKlein** » 19 Jun 2008

Thanks.

Would it be possible to post an announcement to this forum when new versions become available?

Marina Pashkova · Postby **Marina Pashkova** » 20 Jun 2008

Hi MKlein,

Of course. I'll be posting announcements whenever a new version is available.

Best regards.

MKlein · Postby **MKlein** » 25 Jun 2008

Thanks Marina.