I am running MC Version 5.0 Release (Build 2369) on Windows XP Pro SP3 on a quad-core Intel CPU. I noticed when using the genetic optimizer that for awhile at the beginning of an optimization run that all 4 CPUs are max'ed out during the run (looking at the Performance tab of the Windows task manager), but after several minutes only two of the CPUs are being used. In a specific run, the genetic optimizer starts out estimating the total run length will be 20 minutes, and it runs for ~5 minutes or so using all 4 CPU cores. But then after about 5 minutes only 2 CPU cores are being utilized and the optimizer's estimated total run length bumps up to ~28-29 minutes. Once it reaches this state of only using 2 CPU cores it remains that way until the optimizer run completes.
Is this expected behavior? Seems wrong not to be able to utilize all available CPU resources through the entire length of the optimizer run.
Thanks for any feedback on this issue.
Possible performance bug in Portfolio Backtester / Optimizer
-
- Posts: 24
- Joined: 21 Feb 2009
-
- Posts: 100
- Joined: 17 Jul 2009
- Location: Germany
I can absolutely confirm this. I am having a Quad Core CPU as well and for about the first 10 to 20% of Genetic Optimization all 4 Core´s are being used, then only 1 core and Estimated Time Left from this point does about quadrable. I´ve already reported this a while ago as a bug but never got an answer unfortunately:(
Hope this gets fixed.
Hope this gets fixed.
- TJ
- Posts: 7743
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1033 times
- Been thanked: 2222 times
-
- Posts: 100
- Joined: 17 Jul 2009
- Location: Germany
Attached are 2 screenshots where you can see that for the first few percent of genetic optimization all 4 Core´s are being used. Then after some percent, it drops down to only 25% load and estimated time left increases.
Interesting is: this doesn´t happen with Exhausive Search! There all 4 Core´s are fully used all the time.
Interesting is: this doesn´t happen with Exhausive Search! There all 4 Core´s are fully used all the time.
- Attachments
-
- opimizerproblem.GIF
- (100.51 KiB) Downloaded 736 times
-
- optimizerproblem2.GIF
- (100.41 KiB) Downloaded 735 times
-
- Posts: 100
- Joined: 17 Jul 2009
- Location: Germany
-
- Posts: 24
- Joined: 21 Feb 2009
My computer is completely off-the-shelf -- Dell with Intel Core™2 Q9400 Quad-Core (6MB L2 cache,2.66GHz,1333FSB), ATI Radeon HD 2400 PRO 128MB, 4GB Dual Channel DDR2 SDRAM....
Interesting that you haven't seen this with exhaustive optimizer -- I have not looked at that. I have strategies with numerous input variables so need the genetic optimizer to be able to search any sort of reasonable input ranges.
Interesting that you haven't seen this with exhaustive optimizer -- I have not looked at that. I have strategies with numerous input variables so need the genetic optimizer to be able to search any sort of reasonable input ranges.
-
- Posts: 100
- Joined: 17 Jul 2009
- Location: Germany
Yes, I also only use the Genetic Optimizer since my Neural Network Strategy has heaps of inputs. Anyway, you can try this on a smaller-input strategy and compare it. The "Exhausive Search" always uses all 4 Core´s from the start to the end of Optimization, only the "Genetic Optimizer" is having this problem. However, I also only need the Genetic Optimizer since I can´t run 1 trillion combinations in "Exhausive Search":) So hope they have a fix for that.
Cheers.
Cheers.
- Andrew Kirillov
- Posts: 1589
- Joined: 28 Jul 2005
- Has thanked: 2 times
- Been thanked: 31 times
- Contact:
-
- Posts: 100
- Joined: 17 Jul 2009
- Location: Germany
-
- Posts: 24
- Joined: 21 Feb 2009
Thanks Andrew for the confirmation this is a problem. I would also strongly urge you all to prioritize fixing this issue as most modern strategies have a reasonable number of inputs with input ranges which, in combination, far exceed what is possible to run using "exhaustive search". Genetic search is our only option.
Thanks again for your help and great support.
Thanks again for your help and great support.
- Andrew Kirillov
- Posts: 1589
- Joined: 28 Jul 2005
- Has thanked: 2 times
- Been thanked: 31 times
- Contact:
-
- Posts: 24
- Joined: 21 Feb 2009
Andrew-- I have not altered the default settings for the generic algorithm (screenshot of the default settings I am using is attached), so the problem I am having is with the "basic" optimization method. I have not tried the incremental optimization method.
Please confirm you can reproduce this problem with the "basic" genetic optimization as well.
Thanks.
Please confirm you can reproduce this problem with the "basic" genetic optimization as well.
Thanks.
- Attachments
-
- Genetic Algorithm Properties.jpg
- (45.33 KiB) Downloaded 736 times
- Andrew Kirillov
- Posts: 1589
- Joined: 28 Jul 2005
- Has thanked: 2 times
- Been thanked: 31 times
- Contact: