Limited Multithreaded Optimization Performance

Questions about MultiCharts and user contributed studies.
Aston01
Posts: 21
Joined: 24 Oct 2013

Limited Multithreaded Optimization Performance

Postby Aston01 » 26 Nov 2013

I have spent the last couple of days working on trying to lower the length of time my optimizing takes by loading a copy of my work onto a more powerful system. What I ran across was a bit surprising to me an I was hoping someone could help me understand what is happening.

For reference I am portfolio testing a little over a years worth of ASCII mapped data across ~100 symbols on a 5 min bar interval with a consistent data 2 symbol. I made sure and ran the test without any 3rd party dlls to skew the results, so everything is basically standard EL code structure.

I stepped up from my normal Quad Core system w/8GB Ram I had been using to a 16 core Xeon system with 60GB Ram. What I found was that I could get the optimization to scale across the 32 threads, but peak CPU utilization was capped at about 65-67 %. What was even stranger was that it didn't really make much of a dent in the optimization time if at all.

I have attached a screenshot for reference. What's kind of weird is despite not maxing out the CPUs the utilization is oddly uniform in some ways. Any idea would could cause such an issue with scaling an already multithreaded application?

BTW - I also ran CineBench to make sure I wasn't having some type of hardware issue... no issue pegging all the threads at 100%
Attachments
32 vCPU Optimizing.PNG
(207.9 KiB) Downloaded 621 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limited Multithreaded Optimization Performance

Postby Henry MultiСharts » 27 Nov 2013

Hello Aston01,

The optimization load is split between the real cores, not virtual. Two threads from two virtual cores cannot occupy a single physical core at the same time. I.e. when one virtual thread pauses - the other takes it's CPU time.

Aston01
Posts: 21
Joined: 24 Oct 2013

Re: Limited Multithreaded Optimization Performance

Postby Aston01 » 27 Nov 2013

Hi Henry,

I understand what you are saying in principle regarding the concept of multithreading, but why would I get 100% utilization on my Quad Core machine and only 65-67% on a 16 Core system?

From a scalability stand point we are talking about the exact same thread to core ratio ... 4 Core w/ 8 Threads and 16 Core w/ 32 Threads. If I had taken a Quad core system and virtualized it to create 32 vCPUs then it would make sense that I was attempting to saturate pipelines that weren't physically there to saturate.

When I ran Cinebench on the two systems the Quad core achieved a 515 while the 16 Core came in at 2088, so from a sheer horsepower standpoint there seems to be enough to at least chip away at the optimization times some. Yet in practice this didn't prove to be the case.
Attachments
Quad Core Optimizing.PNG
(86.89 KiB) Downloaded 611 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limited Multithreaded Optimization Performance

Postby Henry MultiСharts » 29 Nov 2013

Aston01, please do the following:
1) unzip the attached file to your PC;
2) while all MultiCharts processes are closed double click on AllTracesOff64.reg and confirm registry key import - that will disable MultiCharts logging;
3) disable all output statements and file write processes in your strategies;
4) try backtesting prebuilt MACD or Bollinger Bands signals in genetic and exhaustive optimization modes - can you get 100% CPU usage for all virtual cores?
5) is there any difference between genetic and exhaustive optimization modes? please attach screenshots;
6) try running your strategy in genetic and exhaustive optimization modes - can you get 100% CPU usage for all virtual cores?
7) is there any difference between genetic and exhaustive optimization modes? please attach screenshots.
Attachments
AllTraces64bit.zip
(1.11 KiB) Downloaded 165 times

Aston01
Posts: 21
Joined: 24 Oct 2013

Re: Limited Multithreaded Optimization Performance

Postby Aston01 » 02 Dec 2013

Henry,

Just for clarification what is the purpose in disabling MC logging prior to testing again and do I need to reverse it some how afterwards ?

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limited Multithreaded Optimization Performance

Postby Henry MultiСharts » 03 Dec 2013

Disabling the logs is required to free up the threads that are logging MultiСharts activity so that all resources can be utilized by the optimization. That is recommended to turn the logs back on after the tests by importing the AllTradesOn64.reg file from the same zip file.

Aston01
Posts: 21
Joined: 24 Oct 2013

Re: Limited Multithreaded Optimization Performance

Postby Aston01 » 08 Dec 2013

Henry,

I finally got a moment to run the tests again on a 16 Core (32 vCPU) machine with logging disabled and grabbed some screenshots attached below. What I have noticed is I can hit 100% CPU usage on 5min SPY data running as the singular instrument and also multiple instances of the same data as separate instruments.

But there appears to be an issue hitting that same CPU usage on options data I work with. One of the differences may be due to the options not being as liquid, so less consistency in bar plotting compared to something very liquid like the SPY. Also due to volume not all options plot bars that cover the entire time frame i.e. (Call Strike 100 may cover Feb-Jun 2013 while Call Strike 125 covers April-Aug 2013).

Couple things of Note:
  • - This is all ASCII data formatted the exact same way
    - I get VERY little disk activity during optimization
    - RAM usage is 4.2-4.8GB (6-8%)
    - There seems to be negligible CPU difference between Exhaustive & Genetic Optimizing
    - What is odd is the exact same configuration w/ option files on a Quad Core machine = 100% CPU utilization
Attachments
Multiple Instrument Optimizing -Genetic.PNG
(154.67 KiB) Downloaded 597 times
Multiple Instrument Optimizing -Exhaustive.PNG
(144.6 KiB) Downloaded 595 times
Multiple Common Instrument Optimizing -Exhaustive.PNG
(299.98 KiB) Downloaded 597 times
Single Instrument Optimizing -Exhaustive.PNG
(243.24 KiB) Downloaded 593 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Limited Multithreaded Optimization Performance

Postby Henry MultiСharts » 11 Dec 2013

Aston01,

Please send me (support@multicharts.com) the following information for further investigation:
1) What exact version and build number of MultiCharts are you running? (in MultiCharts go to Help tab-> About);
2) Attach portfolio workspace you are using;
3) in QuoteManager select the symbols you are using, make a right click on any of them->Export data->Export instruments (with data). Send me the Qmd export file for analysis;
4) in PowerLanguage editor->File->Export->export with dependent functions the studies you are backtesting;

If the file size is >10 mb please upload it to any file sharing hosting and send me the download link.


Return to “MultiCharts”