Dear MultiCharts fans,
I am in the process of optimizing a strategy with Portfolio Trader. This is a big task, so every little bit of time savings I can make will help. I noticed a few things:
* initial loading of the data takes quite some time (I am simulating ~200 stocks, 5 minute charts, 10 years). Unfortunately, it seems that MultiCharts is using only a single core to load all these data, instead of using the 16 cores.
* when doing exhaustive optimization, all cores are nicely utilized.
* when doing genetic optimization, this doesn't seem to be the case:
- with the 5m charts mentioned above, I see 15 of my 16 cores fully utilized; Portfolio Trader seems to run 15 simulations in parallel. The last core may or may not be used for general housekeeping, at least I didn't see it ever being used for more than 50%.
- with daily charts, the simulations are about 1 minute each and I see a completely different pattern: None of the cores are utilized to 100% and overall CPU load is never more than 45%. This is actually not a good use of a fast machine. See the picture below.
* when doing backtesting, Portfolio Trader really doesn't seem to care for multiple cores. This is sad, as the individual strategies could be calculated in parallel, before the MM signal combines the results.
Now to my questions:
* is there anything that I can do to increase usage of the cores, especially during data loading and optimization?
* are there any improvements planned for future MultiCharts versions?
Thank you,
best regards, Felix
CPU Usage During Optimization
- fbertram
- Posts: 166
- Joined: 16 Oct 2014
- Location: Seattle, USA
- Has thanked: 36 times
- Been thanked: 76 times
- Contact:
CPU Usage During Optimization
- Attachments
-
- Screen Shot 2015-05-22 at 10.30.33 AM.png
- (185.73 KiB) Downloaded 14149 times
- JoshM
- Posts: 2195
- Joined: 20 May 2011
- Location: The Netherlands
- Has thanked: 1544 times
- Been thanked: 1565 times
- Contact:
Re: CPU Usage During Optimization
I can only guess from the outside looking in, but the feature request to have higher optimisation and backtesting speed by implementing new technology such as GPU calculation has been declined earlier this week:* are there any improvements planned for future MultiCharts versions?
MC-1735 - Hope MC could get much higher optimization/backtesting speed by using new technology such as GPU calculation.
- fbertram
- Posts: 166
- Joined: 16 Oct 2014
- Location: Seattle, USA
- Has thanked: 36 times
- Been thanked: 76 times
- Contact:
Re: CPU Usage During Optimization
well, I am not necessarily looking for GPU optimizations, these probably come with their very own set of challenges. All I am asking for is to use multi-cores more efficiently. I very well understand that for single strategies, that is not easy. For Portfolio Trader as well as optimization the situation is different though: there are a lot of things that *could* be done in parallel- but unfortunately aren't. So having a 16-core Xeon available actually doesn't help that much...
Cheers, Felix
Cheers, Felix
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: CPU Usage During Optimization
Hello Felix,
The whole MultiCharts application uses sequential calculations (charting, optimization, backtesting).
Video card does not affect MultiCharts performance as GPU cannot be utilized for sequential calculations (that is not possible to divide the data series into parts and do the calculations without referencing historical data).
The speed of strategies calculation/optimization is increased when using a multi-core CPU, as the calculations are divided into multiple flows. Multi-core CPU will also increase chart indicator calculation as the calculations are split between the cores (all studies from a single chart are calculated in a separate thread).
Multi-core CPU will not increase the backtesting speed as it is a linear calculation procedure and it cannot be divided into parts for calculation. Here is a good description from TJ.
If you want to force MultiCharts to utilize 16 cores for optimization you can import the attached reg key. Unzip the file, run it, confirm registry modification. The changes will come into effect after all MultiCharts processes are restarted.
The whole MultiCharts application uses sequential calculations (charting, optimization, backtesting).
Video card does not affect MultiCharts performance as GPU cannot be utilized for sequential calculations (that is not possible to divide the data series into parts and do the calculations without referencing historical data).
The speed of strategies calculation/optimization is increased when using a multi-core CPU, as the calculations are divided into multiple flows. Multi-core CPU will also increase chart indicator calculation as the calculations are split between the cores (all studies from a single chart are calculated in a separate thread).
Multi-core CPU will not increase the backtesting speed as it is a linear calculation procedure and it cannot be divided into parts for calculation. Here is a good description from TJ.
If you want to force MultiCharts to utilize 16 cores for optimization you can import the attached reg key. Unzip the file, run it, confirm registry modification. The changes will come into effect after all MultiCharts processes are restarted.
- Attachments
-
- 16_Threads_Optimization.zip
- (276 Bytes) Downloaded 502 times
- signalworks
- Posts: 65
- Joined: 06 Oct 2013
- Location: Germany.Solingen
- Has thanked: 23 times
- Been thanked: 2 times
- Contact:
Re: CPU Usage During Optimization
Dear MC-Team,
I use the MultiCharts64 Version 9.0 Release (Build 11210) on Win7/64Bit and have problems with using all cores while backtesting. I have installed your Registry-hack "All_Thread_ForOptimize.reg". This works fine, if MC is new started. But after some tests (I could not determine a definitely relation yet), only 5-6 thread are further used. When I close MC and re-install the registry-hack, it works fine again. Is there something, that overwrites the hack from the system OS perhaps?
Thanks.
regards,
Lutz
I use the MultiCharts64 Version 9.0 Release (Build 11210) on Win7/64Bit and have problems with using all cores while backtesting. I have installed your Registry-hack "All_Thread_ForOptimize.reg". This works fine, if MC is new started. But after some tests (I could not determine a definitely relation yet), only 5-6 thread are further used. When I close MC and re-install the registry-hack, it works fine again. Is there something, that overwrites the hack from the system OS perhaps?
Thanks.
regards,
Lutz
- Attachments
-
- cpu_03.PNG
- (14.53 KiB) Downloaded 13692 times
- TJ
- Posts: 7740
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1033 times
- Been thanked: 2221 times
Re: CPU Usage During Optimization
Are you backtesting or optimizing? You seemed to say both.Dear MC-Team,
I use the MultiCharts64 Version 9.0 Release (Build 11210) on Win7/64Bit and have problems with using all cores while backtesting. I have installed your Registry-hack "All_Thread_ForOptimize.reg". This works fine, if MC is new started. But after some tests (I could not determine a definitely relation yet), only 5-6 thread are further used. When I close MC and re-install the registry-hack, it works fine again. Is there something, that overwrites the hack from the system OS perhaps?
Thanks.
regards,
Lutz
[FAQ] Backtesting vs Optimization CPU Usage -- Single Core vs Multi-Core
See post #13
viewtopic.php?f=16&t=10811
- signalworks
- Posts: 65
- Joined: 06 Oct 2013
- Location: Germany.Solingen
- Has thanked: 23 times
- Been thanked: 2 times
- Contact:
Re: CPU Usage During Optimization
sorry, uncleared expressed, should called above: "while optimization".
Thanks.
Thanks.
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: CPU Usage During Optimization
Hello Lutz,Dear MC-Team,
I use the MultiCharts64 Version 9.0 Release (Build 11210) on Win7/64Bit and have problems with using all cores while backtesting. I have installed your Registry-hack "All_Thread_ForOptimize.reg". This works fine, if MC is new started. But after some tests (I could not determine a definitely relation yet), only 5-6 thread are further used. When I close MC and re-install the registry-hack, it works fine again. Is there something, that overwrites the hack from the system OS perhaps?
Thanks.
regards,
Lutz
Are you optimizing the same strategy on the same chart in all these tests?
- signalworks
- Posts: 65
- Joined: 06 Oct 2013
- Location: Germany.Solingen
- Has thanked: 23 times
- Been thanked: 2 times
- Contact:
Re: CPU Usage During Optimization
Hello Henry,
yes, same strategy in this case. But that is independent, with other strategies it is the same effect.
Maybe there is a hint with changing chartsettings or create a new one/new workspace. I will try something with it.
regards,
Lutz
yes, same strategy in this case. But that is independent, with other strategies it is the same effect.
Maybe there is a hint with changing chartsettings or create a new one/new workspace. I will try something with it.
regards,
Lutz
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: CPU Usage During Optimization
Hello Lutz,Hello Henry,
yes, same strategy in this case. But that is independent, with other strategies it is the same effect.
Maybe there is a hint with changing chartsettings or create a new one/new workspace. I will try something with it.
regards,
Lutz
Please send us (support@multicharts.com) the following information for analysis:
- workspace you are using;
- in QuoteManager select the symbols you are using, make a right click on the selection->Export data->Export instrument (with data). Send us the QMD export file for analysis;
- in PowerLanguage editor->File->Export->export the studies you are using for replicating this behavior; send us the export file;
- specify the version and build number of MultiCharts you are running (in MultiCharts go to Help tab-> About);
- instructions for replicating this behavior;
- screenshot of your system specs (Control Panel\System\General or right click on the Computer icon and select properties)
- screenshots or video demonstrating this case.
If the file size is >10 mb please upload it to any file sharing hosting and send us the download link.