Data loading speed improvement

Questions about MultiCharts and user contributed studies.
glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Data loading speed improvement

Postby glam_100 » 14 Nov 2010

Hi,

I've been an owner of multichart for a long time (5 years) and I'm still mostly using TS for my work because TS loads data a lot faster than MC. When loading from local cache data changing a chart symbol in TS rarely takes more than 5 seconds for 10 years of intraday data while MC would take 40 seconds for the same loading.

There are times when I test a strategy on say 15 min chart and would like to see how it performance on 5 min charts. While in TS I can switch the time interval and get a result in about 5-10 seconds (data loading time) in MC it would take a frustrating 1min plus. For me this is the single reason why I can't use MC on my work.

I'm surprise this issue hasn't been taken care of for such a long time since loading chart data is the first thing people need to do for any task and for people with a lot of charts it takes a lot longer to fire up all the symbols before you can do any work.

Is there any plan at all to improve the speed of data loading?

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 489 times

Re: Data loading speed improvement

Postby Dave Masalov » 15 Nov 2010

Dear glam_100,

Could you please tell me what is your data provider? Is it TS? If yes, it is not correct to compare the speed of internal data transfer in TS with the speed of MC getting data from TS servers. The speed of data loading depends on data source. TS servers respond to the requests from TS itself much more faster than to the data requests from outside (i.e. MC). Moreover, TS requests all the data in one time and MC sends multiple requests for different time intervals due to compatibility reasons (as MC is compatible with more than 20 data feeds and each one has its own format). This results in data requests from MC "hanging" in TS Download Scheduler. To speed up the process you need to set these requests to "as soon as possible" manually.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Hi Dave,

Here's the result for loading ASCII data (ES 1min 1997-2001)

1 min resolution:

TS - 23 seconds
MC - 3 min 6 seconds

That's 8 times slower for ASCII data. No TS data network of internet.

Attached is the file I used for testing.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Here's the files.
Attachments
ES_ASCII.part05.rar
(2 MiB) Downloaded 154 times
ES_ASCII.part04.rar
(2 MiB) Downloaded 153 times
ES_ASCII.part03.rar
(2 MiB) Downloaded 146 times
ES_ASCII.part02.rar
(2 MiB) Downloaded 139 times
ES_ASCII.part01.rar
(2 MiB) Downloaded 157 times

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Re: Data loading speed improvement

Postby glam_100 » 16 Nov 2010

Last file.
Attachments
ES_ASCII.part06.rar
(11.68 KiB) Downloaded 158 times

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 489 times

Re: Data loading speed improvement

Postby Dave Masalov » 18 Nov 2010

We have tested the data loading speed on W7 Intell Core2Duo 2.8 GHz, 3 Gb RAM, SATA HDD, no anti-virus software.

3rd Party ASCII Data source
Data request - 1 Min From: 11.09.1997, To: Today

MC 5.5 Release - 2 min 23 sec
MC 6.1 Release - 2 min 22 sec

TS 9.0 (Build 8505) - 1 min 4 sec

Not sure why you had 23 seconds. Please run the test again and post the configuration of your system as we did. Probably, you have plotted less data in TS or measured the speed differently.

As I have already written, TS requests all the data in one time and MC sends multiple requests for different time intervals due to compatibility reasons (as MC is compatible with more than 20 data feeds and each one has its own format). This is the reason of the speed difference that we have measured.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Re: Data loading speed improvement

Postby glam_100 » 18 Nov 2010

Here's a video of my 23 second time. It's exactly 23 second I've run it several times.

My configuration is Win XP Core Quad Q6600 overclocked to 3.7Ghz 2GB Ram SATA2 HDD x2 Raid0

TS version is 8.8 (Build 6124). The data file is the same file I sent you.

It could be possible that TS 9.0 (the one you test with) is slow as it is a release candidate build.

I really want to use MC for my daily work because MC has improved a lot over the years. The loading time is the only reason I can't. The difference is too big.
Attachments
Time.rar
(91.04 KiB) Downloaded 144 times

User avatar
Dave Masalov
Posts: 1712
Joined: 16 Apr 2010
Has thanked: 51 times
Been thanked: 489 times

Re: Data loading speed improvement

Postby Dave Masalov » 19 Nov 2010

Dear glam_100,

In our test we have used the following session settings: Mon 0:00 - Mon 23:59; ... Sun 0:00 - Sun 23:59

When we retested the data loading speed with Mon 8:30 - Mon 15:15; ... Fri 8:30 - Sun 15:15, indeed, it took around 25 seconds in TS.

Please double check what session settings do you use in QuoteManager.

To make it faster in MultiCharts, you can also use ASCII Import instead of ASCII Mapping.

Your suggestion to increase the data loading speed has been forwarded to the developers.

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 9 times

Re: Data loading speed improvement

Postby glam_100 » 23 Nov 2010

Thanks Dave. I look forward to a faster MC. It's good for everyone.

amadeu1
Posts: 67
Joined: 11 Feb 2008
Has thanked: 1 time
Been thanked: 6 times

Re: Data loading speed improvement

Postby amadeu1 » 24 Dec 2023

Was this ever really improved ? As the years went by people now load 20 years of 5 min data and the issue has just gotten worse I think . I have a workspace with 30 charts and each holding 15-20 years of 5 min data -- Takes approx 12 Minutes to load on my computer (session times 08:00-22:00. Makes no difference whether I use ascii mapping or direct ascii import of data. As historical data sets become bigger and bigger this is a real bottle neck for Multicharts.

I use Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz 3.00 GHz with 48 GB of RAM


As I am typing I am testing it without anti virus and malware program off-- now takes only one minute. Very good speed

I tried with excluding program files\TS Support folder in Norton and MalwareBytes and now works super fast --

HOWEVER .......When I shut off Multicharts completely and restart Multicharts it takes again 12 minutes to load all these charts . How can I overcome that ?

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Data loading speed improvement

Postby TJ » 25 Dec 2023

Simple solution -- keep your trading computer for trading only, then you do not need to use anti-virus.
Computers are so cheap these days, get another box for non-trading purposes. You can also surf the net with a tablet or even your phone.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: Data loading speed improvement

Postby Mark Brown » 10 Feb 2024

another problem that will never be fixed - i have complained about this for a decade.

mc does not really have it's on data server like ts does. it always relies on pulling data from the data vendor.

i have to laugh how many times this has been brought up sorry....


Return to “MultiCharts”