how to optimize with 2 cpu server (36 cores with 72 threads)  [SOLVED]

Questions about MultiCharts and user contributed studies.
guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

how to optimize with 2 cpu server (36 cores with 72 threads)

Postby guobinzhang2003 » 23 Jan 2015

Hi, dear multicharts techinical support,

I am now using a powerful server wih 2 cpu (e5-2699v3) which have totally 36 cores and 72 threads , but i can only find using 50% of the cpu load, which means the multicharts can only use 1 cpu for optimize, i would like to know how to use the full power of my server in optimizing.

Thanks and best regards,

Guobin
Attachments
cpu load.png
only 50% cpu load in optimizing on my server with 2 cpus
(159.47 KiB) Downloaded 8076 times

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 23 Jan 2015

Hello guobinzhang2003,

We have performed tests only on 16 cores. Unfortunately we cannot guarantee it will use 100% if you use more than 16 cores.

Please perform a test: launch exhaustive optimization for the default MACD LE and MACD SE signals. What is the load?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 23 Jan 2015

Hello Andrew Multicharts,

Thanks for the quick response, I did the test for the MACD SE, the result is the same - cpu load is maximum 50%. it seems that Multicharts optimize process can only use all of the resources in one CPU, Even if another CPU is known by Windows, but multicharts can not recognize it.

Hope the software developer can improve it in lastest hotfix or next version.

Thanks

Guobin

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 23 Jan 2015

I did the test for the MACD SE, the result is the same
Sorry, was that exactly the Exhaustive optimization?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 23 Jan 2015

Exactly, "Exhaustive", the first option of optimize.
I am now using Multicharts 8.8 Chinese version
I think the software development team should know whether MC support 2 CPU or not.
Thanks.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 27 Jan 2015

Hello,

Please follow the next steps:
  1. Close MultiCharts and all its parts
  2. Make sure all MC processes are ended in Windows Task Manager. Here you can find the list of all MC processes: https://www.multicharts.com/trading-sof ... _Processes
  3. Download a file from here (make a right-click on the link and choose "save file as...":
  4. Run the file
  5. If the issue is reproduced, please let me know.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 27 Jan 2015

Sorry, the file can not be downloaded.

"Object not found"

Guobin

Yves BENOIT
Posts: 30
Joined: 17 Jul 2009
Has thanked: 5 times
Been thanked: 10 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Yves BENOIT » 28 Jan 2015

For information,
2 years ago, I did several tests with a 4 CPU x 16 cores from HP.
with same results: very low cpu usage.
We had to perform the last test in an HP certified lab.
An AMD high tech specialist finally brought the explanation:
every RAM slot has to be populated. If not, CPU usage is bounded, for some (stupid) unknown technical reason.
So instead of putting 4 x 8 GB (1 per CPU), we put 16 x 2GB (4slots/cpu x4cpu), and it worked: we obtained 100% CPU usage. It was not the global quantity of Ram, which matters, but the physical presence of any Ram in each slot. Actually, probably 8GB total RAM would have been enough.

So Multicharts ( 8 beta3) was not in cause.

Please note that the speed gain between another computer with 1 CPU x 16 cores and this one 4 CPU x 16 cores was less than 3 times… :(
It means that, probably, a 1 CPU with 64 cores would do much better.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 28 Jan 2015

Sorry, the file can not be downloaded.

"Object not found"

Guobin
Please follow the instruction once again, i have updated it.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Hi, Andrew,

I try again and again on several pc, but still not get the link.
or maybe you send it to me the file by email: guobinzhang2003@qq.com

***************************************************
Not Found

The requested URL /u/26340639/Soft/DO NOT DELETE THIS FOLDER/Threads_For_Optimization/All_Thread_ForOptimize.reg was not found on this server.

**************************************************************************

Guobin


guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Sorry, Andrew, I can not see the right display.
i don't know if it is my pc's problem, but there are 3 pc on my desk, all of them are same.
please see my attached file.
Attachments
no display.png
no right display
(42.96 KiB) Downloaded 8080 times

Yves BENOIT
Posts: 30
Joined: 17 Jul 2009
Has thanked: 5 times
Been thanked: 10 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Yves BENOIT » 28 Jan 2015

Hehe, if in China, it's quite possible they block Dropbox access.
Trying to communicate with a friend who just arrived in Dubai university, we discovered similar problems. Impossible also to use Teamviewer… :)

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Hi, Yves BENOIT,

Thanks for the information.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 28 Jan 2015

Sorry, Andrew, I can not see the right display.
i don't know if it is my pc's problem, but there are 3 pc on my desk, all of them are same.
please see my attached file.
The file and the screenshots are from my DropBox account. The are blocked on your end.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Thanks, Andrew, I am asking my friend to open the link outside China.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Hi, Andrew, I found a similar file name in this forum, is it the same file?

viewtopic.php?f=19&t=46322&p=103483&hil ... ds#p103483

if it is the same, I have tried it and it doesn't work.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 28 Jan 2015

Hi, Andrew, I found a similar file name in this forum, is it the same file?

viewtopic.php?f=19&t=46322&p=103483&hil ... ds#p103483

if it is the same, I have tried it and it doesn't work.
What file from the archive did you run?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

Hi, Yves BENOIT,

I thought about your case two years ago, it is very interesting.
In my case, when I bought this server, I did the CPU load test, all of 36 cores with 72 threads working on 100% load , I can see it very clear in task manager.
the software is for testing the temperature of CPU.
So it means another software can use 100% of CPU, Multicharts should also work and have nothing to do with number of RAM bars in this case.
Am I right ?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

hi, Andrew,

I run the file of "All_Thread_ForOptimize.reg"

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby TJ » 28 Jan 2015

Hi, Yves BENOIT,

I thought about your case two years ago, it is very interesting.
In my case, when I bought this server, I did the CPU load test, all of 36 cores with 72 threads working on 100% load , I can see it very clear in task manager.
the software is for testing the temperature of CPU.
So it means another software can use 100% of CPU, Multicharts should also work and have nothing to do with number of RAM bars in this case.
Am I right ?
RAM bar distribution is a hardware design in the motherboard. It is not a software issue.

txls
Posts: 76
Joined: 17 Nov 2010
Has thanked: 36 times
Been thanked: 24 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby txls » 28 Jan 2015

I'm guessing this cpu load thing has got smth to do with Hyperthreading and the fact that those are not all actually cores.

And from what I understand, MC can now use all threads on Intel cpus, is that right?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 28 Jan 2015

In this case, I see only 18 cores (36 threads) are using for optimizing , and the CPU load is 50%, so I guess MC only recognize only 1 CPU. And I try several times, sometimes, the first half of cores were used. In another time, the second half of cores were used.
please see attached picture - the cpu load in 2 different time
Attachments
cpu load.png
cpu load in 2 different time
(94.99 KiB) Downloaded 8059 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby TJ » 28 Jan 2015

In this case, I see only 18 cores (36 threads) are using for optimizing , and the CPU load is 50%, so I guess MC only recognize only 1 CPU. And I try several times, sometimes, the first half of cores were used. In another time, the second half of cores were used.
please see attached picture - the cpu load in 2 different time
What are you optimizing? Maybe your load is not enough?

Are you testing as requested by by Andrew? ie BOTH signals together?
Have you tried adding a few more signal to the pot?

I remember one user tested MultiCharts with a 4 CPU server, and he reported all CPUs were used. I don't know how many threads he was using though.

ps. which windows version you are using?


Hello guobinzhang2003,
:
Please perform a test: launch exhaustive optimization for the default MACD LE and MACD SE signals. What is the load?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 29 Jan 2015

Hello, TJ

I followed the suggestion of Andrew to use MACD LE, MACD LE & MACD SE together and other complicated signal by myself, the result is the same. Only 36 threads of total 72 threads are used by MC for optimizing. it means 50% CPU load always.

Please see attached file.


If I start another MC program, and do another optimizing, then the 2nd MC will use the remain 36 threads. Then 2 MC can use 100% cpu load for optimizing.

My Windows version is Windows7 64 bit sp1 Chinese version.
Attachments
cpu load0.png
(197.59 KiB) Downloaded 8047 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby TJ » 29 Jan 2015

::
My Windows version is Windows7 64 bit sp1 Chinese version.
Which version? Pro? Enterprise?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 29 Jan 2015

Enterprise!, And I also use AIDA64 to do the CPU testing, please find attached picture.
It can use 100% CPU.
Attachments
AIDA64 testing.png
AIDA64 testing
(265.14 KiB) Downloaded 8053 times

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 29 Jan 2015

Please follow the next steps:
  1. Close MultiCharts and all its parts
  2. Make sure all MC processes are ended in Windows Task Manager. Here you can find the list of all MC processes: https://www.multicharts.com/trading-sof ... _Processes
  3. Download the attached archive.
  4. Unzip the archive.
  5. Run the file from the archive.
  6. If the issue is reproduced, please let me know.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 29 Jan 2015

Hi, Andrew,

Thanks for the update, I just download the file and restart my server, checked the all of the MC processes and run the file.

But unfortunitily , the result is the same. Only one CPU was recoginized, and 36 threads of 72 can be used, and CPU load is still 50%.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 29 Jan 2015

Sorry, wrong file :-)
This one should help.
Attachments
72 threads.zip
(250 Bytes) Downloaded 341 times

Yves BENOIT
Posts: 30
Joined: 17 Jul 2009
Has thanked: 5 times
Been thanked: 10 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Yves BENOIT » 29 Jan 2015

Hello guobinzhang,
Aida or temperature software are low level softwares. Some parts might have been written in machine language (specific routines designed for specific processors). When testing 2 years ago on this HP 4 proc x16 cores , standard testing routines would also be able to provoke 100% load.
But Normally compiled C program max load was 50%.

I still wonder if you are not in the same situation as we were. The bug we observed was also a very low level, inside the multiprocessor managing task, probably hard coded ; it was ignored by HP top representative engineers in Paris: they spent with us several months to arrive to the conclusion that I stated. They sent us the server back & forth for testing for free. Lots of efforts. A clever engineer from processor company gave us the clue:
Finally, limitation to 50 % load was per process. When all slots were equally populated, this weird limitation disappeared.
TJ is right, it was hard coded, independent of software.

I don't want to interfere, but are there empty Ram slots in your server?
can you try to put equal RAM in each slot?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 29 Jan 2015

Sorry Andrew, I updated the download file and test several times, but the problem remains

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 29 Jan 2015

Hello, Yves BENOIT,

Thank you very much for the input, I think you are right after so much effort had made.
I will try your suggestion for testing, increase the ram bars and see the result.

Hello, Andrew and TJ,

Also thank you so much for all of the effort you made. If this is the system bug, then it is difficult for us to solve it.

I will inform you after I got the result.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 29 Jan 2015

Sorry Andrew, I updated the download file and test several times, but the problem remains
If the issue still persists, please come to our live chat during working hours (6:30 am – 2:45 pm EST) to let our operators connect to your computer remotely and help you.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 02 Feb 2015

Hello, Andrew Multicharts,Yves BENOIT,txls and TJ,

Firstly thanks for your help in the case.

In this two days, I did some test, and finally got some result (maybe not the best one).

Firstly, I followed the hint from Yves BENOIT, and insert ram bar to all of the ram slot.
but it seems not like the case of him, the problem remains - only 36 threads (total is 72 threads in the system) working and the CPU load is still 50%.

Then I followed the hint of txls, switch off HT option in mainboard, then during the optimizing, I can only see 36 threads (That is the total thread can be seen by system). But CPU load became to 100%.

And then I compared the speed of optimazing with one strategy, the result is the first case using 45 min, and in second case was 30 min.

So the conclusion is if switching of HT options on mainboard will reduce the optimizing time 30% and use 100% CPU, otherwise the 2-cpu system can only use 50% of CPU load.

I don't know if it is the best solution.

Later I will do the same test in 1-cpu system again.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 02 Feb 2015

Firstly, I followed the hint from Yves BENOIT, and insert ram bar to all of the ram slot.
but it seems not like the case of him, the problem remains - only 36 threads (total is 72 threads in the system) working and the CPU load is still 50%.
Sorry, could you confirm something to avoid confusion?
  1. You have 2 separate CPUs in your computer, don't you?
  2. What is the number of cores (not threads) of CPU #1?
  3. What is the number of cores (not threads) of CPU #2?
  4. Do both CPUs have hyper threading technology?

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 02 Feb 2015

Yes,

1. my computer have 2 seperate CPU (intel E5-2699v3)
2. each CPU has 18 cores, so total 36 cores in my computer, 72 threads (with hyper threading)
3. both CPUs have hyper threading technology before I switch them off.
4. After I switch off the HT, the computer have 36 threads in the system.

txls
Posts: 76
Joined: 17 Nov 2010
Has thanked: 36 times
Been thanked: 24 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby txls » 02 Feb 2015

That is actually not unheard of. Some software just benchmarks better with HT off. If you start to add tasks to a system already running optimization, that's where you're supposed to benefit from HT.

http://forums.anandtech.com/showthread.php?t=2325367 just an example of an HT-related discussion.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 02 Feb 2015

I have tried to compare optimizing time w and w/o HT option in 1 cpu PC, the result is with HT option the speed is faster. But in 2 CPU system, with HT switch on ,it seems MC only use 1 CPU to optimize so the speed is slower. I also found in some other testing software, when they compare the performance wiht 2 CPU system, HT switch is always off. And in other forum , I also found they mentioned SQLserver software can only have 50% CPU load in 2 CPU system. some people suggested to switch off HT.

orion
Posts: 250
Joined: 01 Oct 2014
Has thanked: 65 times
Been thanked: 104 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby orion » 02 Feb 2015

The best way to get more throughput for backtest and optimization is to buy more software licenses from MC and use cheaper desktop processors rather than server class processor with 18 cores.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre  [SOLVED]

Postby Andrew MultiCharts » 06 Feb 2015

Guobinzhang2003,

I have talked to our main developer and he confirmed that the file should tell your system to use exactly 72 threads. Unfortunately we cannot help you if the system doesn't use maximum of them. As you have checked it yourself, it does work without HT.

guobinzhang2003
Posts: 23
Joined: 23 Jan 2015
Has thanked: 13 times
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby guobinzhang2003 » 07 Feb 2015

Hello, Andrew Multicharts,

Would you please check with MC developer if the file of 72 threads also fit for the MC8.8 Chinese version or not ? Thanks

Gaempi
Posts: 60
Joined: 27 Jul 2010
Location: Switzerland
Has thanked: 3 times
Been thanked: 8 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Gaempi » 09 Feb 2015

Hello

I'm working with two workstations, one with 2 x 6 cores (24 threads) and one with 2 x 10 cores (40 threads). So i have a lot of experiences with optimization and developing strategies. I see the % use of the cores depending of the number of the inputs. For using 100% of the cores you have to take more than 3 inputs for optimizing the strategy. If i take 4 and more inputs for genetic optimizing i have normally an usage of near 100% of all cores. if i optimize one or two inputs i see 35% up to 75% of the cores (threads) are used. Only in this case not alle cores are computing.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: how to optimize with 2 cpu server (36 cores with 72 thre

Postby Andrew MultiCharts » 20 Feb 2015

Hello, Andrew Multicharts,

Would you please check with MC developer if the file of 72 threads also fit for the MC8.8 Chinese version or not ? Thanks
In this case it doesn't matter if it is regular or Chinese version of MC.

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 12 Sep 2017

once number of logical cpu is more than 64, the cpu utilization will get problem, even in version 11

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby Henry MultiСharts » 14 Sep 2017

Hello zhiguang,

Please describe the specific issue you have.

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 14 Sep 2017

The MC optimizer uses only half of the core if number of logic cpu is more than 64

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby TJ » 14 Sep 2017

Hyperthreads are not real threads.

Threadrippers are not real threads.

They do not add performance gains to optimizations.

Yves BENOIT
Posts: 30
Joined: 17 Jul 2009
Has thanked: 5 times
Been thanked: 10 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby Yves BENOIT » 14 Sep 2017

Hello zhiguang,
How many RAM slots do you have?
How are they populated (number of GB/slot), one by one in detail?
what CPU trademark and reference?
Yves Benoit

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 14 Sep 2017

mc11.jpg
(246.07 KiB) Downloaded 6842 times

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 14 Sep 2017

mc8.jpg
(196.47 KiB) Downloaded 6841 times

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 14 Sep 2017

mc11-64.jpg
(197.73 KiB) Downloaded 6841 times

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 14 Sep 2017

I have done a lot of tests, including different physical machines and virtual machine of Hyper-V and different windows operating system (including Windows Server 2016 DC), mc64 (until version 11) optimization can only use up to 64 logical processors, this problem needs to be solved.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby Henry MultiСharts » 19 Sep 2017

zhiguang,

Please follow these steps:
  1. Close MultiCharts and all its processes
  2. Make sure all MC processes are ended in Windows Task Manager. Here you can find the list of all MC processes: https://www.multicharts.com/trading-sof ... _Processes
  3. Download the attached archive.
  4. Unzip the archive.
  5. Run the file from the archive.
  6. Run optimization.
  7. let us know if it solved the issue or not.
Attachments
72 threads.zip
(250 Bytes) Downloaded 253 times

zhiguang
Posts: 11
Joined: 14 Apr 2013
Been thanked: 2 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby zhiguang » 19 Sep 2017

This doesn't solve the problem

bomberone1
Posts: 310
Joined: 02 Nov 2010
Has thanked: 26 times
Been thanked: 23 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby bomberone1 » 04 Feb 2018

Hi,
could you confirm that we could use all cores and all 20 threads of these 2 cpus?

1
CPU is:
Intel i9-7900X CPU 2066, 3.3 GHz, Skylake
10 cores, 20 threads
Motherboard:
ASUS PRIME X299-A Scheda Madre, Socket 2066 ATX, Dual M.2, USB 3.1

2
CPU:
AMD Ryzen Threadripper 1950X 3.4GHz 32MB L3 processor -
16 core , 32 threads
Motherboard:
Gigabyte X399 AORUS Gaming 7 AMD X399 ATX motherboard -

What do you like?

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby Smoky » 05 Feb 2018

the second sure !

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby TJ » 05 Feb 2018

Threadripper is not real thread.

When you are doing optimization, you need to use real threads.
This is just basic computer logic. There are no shortcuts. There are no free lunches.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: how to optimize with 2 cpu server (36 cores with 72 threads)

Postby wilkinsw » 05 Feb 2018

Hi,

Here is a link to a new forum on the use of MCs genetic algorithm with multiple cores:

viewtopic.php?t=51229

I have a 40 core dedicated server that does indeed use all 100% of CPU resource when I load it up with at least a couple of optimizations. With only one optimization all cores are active but slightly less than 100%.

Using the workspace:

https://drive.google.com/file/d/16LJTKp ... sp=sharing

and data:

https://drive.google.com/file/d/1qZwDfO ... sp=sharing

If a run a exhaustative single optim I can bash out 5000 sims in about 22s.

If I open 4 identical WSs and run optimizations that drops to 11s.


Return to “MultiCharts”