Pentium 4 CPU 3.20 GHz
Memory 1GB
Windows XP SP 2
When I plot a 1 min chart from 2007/1/1 to 2007/12/25, it loaded 88094 bars from the local storage, not from the download server. It took 3 min 30 seconds. Is this normal loading speed from the local storage when all the data is already stored? I guess not.
While loading data from the local strorage, "System" process from the TaskManager uses the CPU over 50%. This makes whole system slow down. Is this normal too? I guess not either.
Please comment on these phenomena.
Historical Data loading from local storage takes too long?
-
Fabrice Daniel
- Posts: 71
- Joined: 25 Aug 2007
- Been thanked: 2 times
This is the worse Multichart weakness. Good CPU usage (multi core) but very very very poor Database performance.
You could divide by 2 your data loading time excluding the Database Folder from your Antivirus real time scaning ... but it's a very small gain (loading time lowered from 3min to 1min30 sec ... very poor performance compared to AB which display the whole 5 years of 1 min datas ... and not only the most recent ... in less than 2 sec !!!) .
So until MC will take less than 5 sec to display 5 years of 1 min datas ... it will be unusable for serious everyday usage.
You could divide by 2 your data loading time excluding the Database Folder from your Antivirus real time scaning ... but it's a very small gain (loading time lowered from 3min to 1min30 sec ... very poor performance compared to AB which display the whole 5 years of 1 min datas ... and not only the most recent ... in less than 2 sec !!!) .
So until MC will take less than 5 sec to display 5 years of 1 min datas ... it will be unusable for serious everyday usage.
-
Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Hi,
The situation when loading 88,000 bars from history takes over 3 minutes is definitely not normal. I ran some tests according to which it takes 15 seconds to load 180,000 bars in the current 999.999 MC version. In the upcoming MC version it will take less than 5 seconds. In general, the upcoming version will load the data 3 times faster than the current version.The reasons why in your case data loading took over 3 minutes might be as follows:
1. When the data is loaded, two processes, MultiCharts and tsserver, exchange the information which can be intercepted by the antivirus program. This considerably slows down data loading. Make sure the antivirus program is turned off and try plotting the chart again.
2. Check if all of the requested data is stored in the database in the minute format. When minute data is requested, MultiCharts first tries to find the minute data. If there is no minute data in the database, MultiCharts checks the tick data and uses it to plot minute bars, which, of course, takes more time. AB, can't perform this kind of operations.
3. Check if you have mixed data in the database. If minute data alternate with tick data, data loading becomes considerably slower (again, because MultiCharts will be creating minute bars based on millions of ticks)
4. If all the above can't explain why it took so long to load the data in the case you're describing, please contact us through LiveChat/HelpDesk.
5. Fabrice, if I'm not mistaken 5 years of data would generally be used for backtesting, in which case instant data loading is not crucial. Could you please give examples when loading 5 years of data must be done instantaneoulsy.
Regards.
The situation when loading 88,000 bars from history takes over 3 minutes is definitely not normal. I ran some tests according to which it takes 15 seconds to load 180,000 bars in the current 999.999 MC version. In the upcoming MC version it will take less than 5 seconds. In general, the upcoming version will load the data 3 times faster than the current version.The reasons why in your case data loading took over 3 minutes might be as follows:
1. When the data is loaded, two processes, MultiCharts and tsserver, exchange the information which can be intercepted by the antivirus program. This considerably slows down data loading. Make sure the antivirus program is turned off and try plotting the chart again.
2. Check if all of the requested data is stored in the database in the minute format. When minute data is requested, MultiCharts first tries to find the minute data. If there is no minute data in the database, MultiCharts checks the tick data and uses it to plot minute bars, which, of course, takes more time. AB, can't perform this kind of operations.
3. Check if you have mixed data in the database. If minute data alternate with tick data, data loading becomes considerably slower (again, because MultiCharts will be creating minute bars based on millions of ticks)
4. If all the above can't explain why it took so long to load the data in the case you're describing, please contact us through LiveChat/HelpDesk.
5. Fabrice, if I'm not mistaken 5 years of data would generally be used for backtesting, in which case instant data loading is not crucial. Could you please give examples when loading 5 years of data must be done instantaneoulsy.
Regards.