Data loading speed improvement

Questions about MultiCharts and user contributed studies.
glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Data loading speed improvement

Postby glam_100 » 14 Nov 2010

Hi,

I've been an owner of multichart for a long time (5 years) and I'm still mostly using TS for my work because TS loads data a lot faster than MC. When loading from local cache data changing a chart symbol in TS rarely takes more than 5 seconds for 10 years of intraday data while MC would take 40 seconds for the same loading.

There are times when I test a strategy on say 15 min chart and would like to see how it performance on 5 min charts. While in TS I can switch the time interval and get a result in about 5-10 seconds (data loading time) in MC it would take a frustrating 1min plus. For me this is the single reason why I can't use MC on my work.

I'm surprise this issue hasn't been taken care of for such a long time since loading chart data is the first thing people need to do for any task and for people with a lot of charts it takes a lot longer to fire up all the symbols before you can do any work.

Is there any plan at all to improve the speed of data loading?

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 487 times

Re: Data loading speed improvement

Postby Dave Masalov » 15 Nov 2010

Dear glam_100,

Could you please tell me what is your data provider? Is it TS? If yes, it is not correct to compare the speed of internal data transfer in TS with the speed of MC getting data from TS servers. The speed of data loading depends on data source. TS servers respond to the requests from TS itself much more faster than to the data requests from outside (i.e. MC). Moreover, TS requests all the data in one time and MC sends multiple requests for different time intervals due to compatibility reasons (as MC is compatible with more than 20 data feeds and each one has its own format). This results in data requests from MC "hanging" in TS Download Scheduler. To speed up the process you need to set these requests to "as soon as possible" manually.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Hi Dave,

Here's the result for loading ASCII data (ES 1min 1997-2001)

1 min resolution:

TS - 23 seconds
MC - 3 min 6 seconds

That's 8 times slower for ASCII data. No TS data network of internet.

Attached is the file I used for testing.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Here's the files.
Attachments
ES_ASCII.part05.rar
(2 MiB) Downloaded 69 times
ES_ASCII.part04.rar
(2 MiB) Downloaded 69 times
ES_ASCII.part03.rar
(2 MiB) Downloaded 63 times
ES_ASCII.part02.rar
(2 MiB) Downloaded 61 times
ES_ASCII.part01.rar
(2 MiB) Downloaded 72 times

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Last file.
Attachments
ES_ASCII.part06.rar
(11.68 KiB) Downloaded 71 times

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 487 times

Re: Data loading speed improvement

Postby Dave Masalov » 18 Nov 2010

We have tested the data loading speed on W7 Intell Core2Duo 2.8 GHz, 3 Gb RAM, SATA HDD, no anti-virus software.

3rd Party ASCII Data source
Data request - 1 Min From: 11.09.1997, To: Today

MC 5.5 Release - 2 min 23 sec
MC 6.1 Release - 2 min 22 sec

TS 9.0 (Build 8505) - 1 min 4 sec

Not sure why you had 23 seconds. Please run the test again and post the configuration of your system as we did. Probably, you have plotted less data in TS or measured the speed differently.

As I have already written, TS requests all the data in one time and MC sends multiple requests for different time intervals due to compatibility reasons (as MC is compatible with more than 20 data feeds and each one has its own format). This is the reason of the speed difference that we have measured.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Re: Data loading speed improvement

Postby glam_100 » 18 Nov 2010

Here's a video of my 23 second time. It's exactly 23 second I've run it several times.

My configuration is Win XP Core Quad Q6600 overclocked to 3.7Ghz 2GB Ram SATA2 HDD x2 Raid0

TS version is 8.8 (Build 6124). The data file is the same file I sent you.

It could be possible that TS 9.0 (the one you test with) is slow as it is a release candidate build.

I really want to use MC for my daily work because MC has improved a lot over the years. The loading time is the only reason I can't. The difference is too big.
Attachments
Time.rar
(91.04 KiB) Downloaded 68 times

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 487 times

Re: Data loading speed improvement

Postby Dave Masalov » 19 Nov 2010

Dear glam_100,

In our test we have used the following session settings: Mon 0:00 - Mon 23:59; ... Sun 0:00 - Sun 23:59

When we retested the data loading speed with Mon 8:30 - Mon 15:15; ... Fri 8:30 - Sun 15:15, indeed, it took around 25 seconds in TS.

Please double check what session settings do you use in QuoteManager.

To make it faster in MultiCharts, you can also use ASCII Import instead of ASCII Mapping.

Your suggestion to increase the data loading speed has been forwarded to the developers.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Re: Data loading speed improvement

Postby glam_100 » 23 Nov 2010

Thanks Dave. I look forward to a faster MC. It's good for everyone.


Return to “MultiCharts”