Optimisation and Multi-Core
Optimisation and Multi-Core
Dear Sirs,
What is the maximum of cores that the upcoming version of MC support?
Any update of the 64bits version?
Many thanks,
What is the maximum of cores that the upcoming version of MC support?
Any update of the 64bits version?
Many thanks,
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
I notice with two cores and the current beta (2.1.928.1050) that computation seems to depend on MCActivex.exe.
With two MC instances with about 3 workspaces each, I find that only 50% of the CPU gets used (one core) and all the time is spent in MCActiveX.exe.
Any way to break that up so that MCActiveX.exe isn't a bottleneck?
With two MC instances with about 3 workspaces each, I find that only 50% of the CPU gets used (one core) and all the time is spent in MCActiveX.exe.
Any way to break that up so that MCActiveX.exe isn't a bottleneck?
MC can you guys comment on this ?I notice with two cores and the current beta (2.1.928.1050) that computation seems to depend on MCActivex.exe.
With two MC instances with about 3 workspaces each, I find that only 50% of the CPU gets used (one core) and all the time is spent in MCActiveX.exe.
Any way to break that up so that MCActiveX.exe isn't a bottleneck?
In this case, it looks like charting is not taking advantage of the other CPU....and this contradicts what Andrew said earlier here
forum.tssupport.com/viewtopic.php?t=4200
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Hello,
The way MultiCharts works with multiple cores (which has been proved by our tests) is that every chart window uses a different thread which is assigned to one of the cores.
We suggest to run the following test.
Create a chart and apply many indicators to ensure a large workload for CPU.
Then create another chart and apply lots of indicators to it as well.
And after check CPU: two cores should be employed at that point.
If this is not the case on your machine, please let us know.
The way MultiCharts works with multiple cores (which has been proved by our tests) is that every chart window uses a different thread which is assigned to one of the cores.
We suggest to run the following test.
Create a chart and apply many indicators to ensure a large workload for CPU.
Then create another chart and apply lots of indicators to it as well.
And after check CPU: two cores should be employed at that point.
If this is not the case on your machine, please let us know.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Dear Marina,
Thank you very much for your reply.
I just read about a year ago that you might develop a 64bits version of MC.
I believe I would notice substantial performance gains in my nested loops by taking full advantage of 16 GPRS, and in certain type of optimisations.
But it is not that important.
A 64bits version might give you a marketing/competitive edge as well, but only for a limited number of enthusiast customers.
Disregard that, I am deeply interested by your multicore functionality: what would be, in your programming team's opinion, the ultimate multicore processor for optimisation?
A few :
Dual Core Intel Itanium 2 processor 9050?
Intel® Core™2 Extreme Quad-Core Processor?
Quad-Core Intel® Xeon® Processor 5300?
or an AMD setup because they are better in floatingpoint calculations?
Many thanks,
Thank you very much for your reply.
I just read about a year ago that you might develop a 64bits version of MC.
I believe I would notice substantial performance gains in my nested loops by taking full advantage of 16 GPRS, and in certain type of optimisations.
But it is not that important.
A 64bits version might give you a marketing/competitive edge as well, but only for a limited number of enthusiast customers.
Disregard that, I am deeply interested by your multicore functionality: what would be, in your programming team's opinion, the ultimate multicore processor for optimisation?
A few :
Dual Core Intel Itanium 2 processor 9050?
Intel® Core™2 Extreme Quad-Core Processor?
Quad-Core Intel® Xeon® Processor 5300?
or an AMD setup because they are better in floatingpoint calculations?
Many thanks,
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
I cannot give you a detailed answer because I do not know the exact charactersistics of each and every processor.
The rule of thumb is: the more cores there are the higher the overall efficiency is. If you can get a two-core processor it would be good. Quad core would be better. A motherboard with two quadcores would be better still.
The rule of thumb is: the more cores there are the higher the overall efficiency is. If you can get a two-core processor it would be good. Quad core would be better. A motherboard with two quadcores would be better still.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
The answer I posted earlier was not my personal opinion but that of our engineers.
They have not run tests with the processors you mentioned that is why they wanted to be cautious as to what we say on this matter. Also, when targeting the software to work with multiple cores we did it in such a way that the optimization will not depend on a particular CPU architecture. But the general principle is the greater the number of cores the faster the program works.
Also, we have received Intel Partner certificate which soon will be published on our website.
They have not run tests with the processors you mentioned that is why they wanted to be cautious as to what we say on this matter. Also, when targeting the software to work with multiple cores we did it in such a way that the optimization will not depend on a particular CPU architecture. But the general principle is the greater the number of cores the faster the program works.
Also, we have received Intel Partner certificate which soon will be published on our website.
Dear Marina,
I understand.
Please keep me posted if there is any suggestions on this, or if your team decides to run some benchmarks one day.
Also, it might be an idea to create a Minimum, Recommended and Optimal Hardware Requirements for MC (I could not find the information on the website).
Thank you for your kind support,
I understand.
Please keep me posted if there is any suggestions on this, or if your team decides to run some benchmarks one day.
Also, it might be an idea to create a Minimum, Recommended and Optimal Hardware Requirements for MC (I could not find the information on the website).
Thank you for your kind support,
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
This is what I have done. But the bottleneck seems to be, as I said, in MCActiveX.exe.Hello,
The way MultiCharts works with multiple cores (which has been proved by our tests) is that every chart window uses a different thread which is assigned to one of the cores.
We suggest to run the following test.
Create a chart and apply many indicators to ensure a large workload for CPU.
Then create another chart and apply lots of indicators to it as well.
And after check CPU: two cores should be employed at that point.
If this is not the case on your machine, please let us know.
Perhaps it is not the graphical rendering but the processing of market data that is the bottleneck?
I notice that that as the market gets faster, the slower charts can't keep up with the faster charts. The display of the slower charts just sort of "Freezes" and then catches up after some amount of time. The faster charts keep up better. They have a much smaller amount of history that they are displaying, which might be related. Somehow, the longer charts with more bars can't keep up and the CPU usage sits at 50% (one core).
Is it possible that this has something to do with Vista 64 bit?
Or possibly with using TS (3896) along with the "save data" flag in QuoteManager?
It is pretty clear that both cores are not being utilized fully.
This suggests that you try a test using charts with few bars and lots of bars in the same workspace. Then see what happens as the market data rate climbs to the point that they can't keep up and observe the CPU utilization.
The principal advantage is that the 64 bit instruction set has double the number of processor registers which allows faster program execution.At the moment we are not developing the 64bit version.
Could I ask why it is so important for you to have 64version?
The secondary advantages are:
1 Better floating point instructions.
2 Ability to have essentially unlimited amounts of virtual memory.
All that said, a stable MC is much more important than a faster MC.
I have written at length in the past over general performance of 'basic' functions (receiving & storing data, retrieving it and displaying it). I wont repeat my observations and results of empirical testing.
The bottlenecks for these function are likely to be the places that wont benefit from multiple threaded processing (storing and retrieving data & BLITing pixels onto the screen). While TSS should be applauded for taking into account multiple cores the most significant performance gains for basic tasks are likely to achieved by profiling areas of code that are bottlenecks and always will be bottlenecks.
For example a recent bug I have noticed has allowed me to observe chart scaling and positioning in operation. (when an indicator with a margin is at the bottom of the screen it 'flickers up and down' as the chart is re-scaled...looks like the first try at scaling is wrong causing the chart to jump). It made me wonder if the scaling routine is in the multithreaded portion (where it could safely be) or in the display part of the code which I guess wont benefit from multiprocessing. Also from actually being able to see re-scaling working it appears as if the scaling algorithm may be being called more often than it need be.
In short and more generally - code optimisation > more cores.
Cheers,
Nick.
The bottlenecks for these function are likely to be the places that wont benefit from multiple threaded processing (storing and retrieving data & BLITing pixels onto the screen). While TSS should be applauded for taking into account multiple cores the most significant performance gains for basic tasks are likely to achieved by profiling areas of code that are bottlenecks and always will be bottlenecks.
For example a recent bug I have noticed has allowed me to observe chart scaling and positioning in operation. (when an indicator with a margin is at the bottom of the screen it 'flickers up and down' as the chart is re-scaled...looks like the first try at scaling is wrong causing the chart to jump). It made me wonder if the scaling routine is in the multithreaded portion (where it could safely be) or in the display part of the code which I guess wont benefit from multiprocessing. Also from actually being able to see re-scaling working it appears as if the scaling algorithm may be being called more often than it need be.
In short and more generally - code optimisation > more cores.
Cheers,
Nick.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Still, I would like to know if you ran the test I suggested trying and what were the results? We need some documented proof (e.g. workspaces) of the problem you are describing. When you are talking about one core of the CPU working with the 50% workload: how many workspaces were open? How many charts were plotted? We need to know exactly what was going on. Could you please post or e-mail (mpashkova@tssupport.com) all the worskpaces that were open?This is what I have done. But the bottleneck seems to be, as I said, in MCActiveX.exe.Hello,
The way MultiCharts works with multiple cores (which has been proved by our tests) is that every chart window uses a different thread which is assigned to one of the cores.
We suggest to run the following test.
Create a chart and apply many indicators to ensure a large workload for CPU.
Then create another chart and apply lots of indicators to it as well.
And after check CPU: two cores should be employed at that point.
If this is not the case on your machine, please let us know.
Perhaps it is not the graphical rendering but the processing of market data that is the bottleneck?
I notice that that as the market gets faster, the slower charts can't keep up with the faster charts. The display of the slower charts just sort of "Freezes" and then catches up after some amount of time. The faster charts keep up better. They have a much smaller amount of history that they are displaying, which might be related. Somehow, the longer charts with more bars can't keep up and the CPU usage sits at 50% (one core).
Is it possible that this has something to do with Vista 64 bit?
Or possibly with using TS (3896) along with the "save data" flag in QuoteManager?
It is pretty clear that both cores are not being utilized fully.
This suggests that you try a test using charts with few bars and lots of bars in the same workspace. Then see what happens as the market data rate climbs to the point that they can't keep up and observe the CPU utilization.
We could also have a look at the issue via HelpDesk to see how the program behaves on your machine. You can contact us through LiveChat, briefly remind us what the issue is and you will be prompted to run HelpDesk.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Indeed, introducing 64bit will expand the size of available memory. However, MultiCharts does not use that much memory and switching to 64bit version is not vital for improving performance.The principal advantage is that the 64 bit instruction set has double the number of processor registers which allows faster program execution.At the moment we are not developing the 64bit version.
Could I ask why it is so important for you to have 64version?
The secondary advantages are:
1 Better floating point instructions.
2 Ability to have essentially unlimited amounts of virtual memory.
All that said, a stable MC is much more important than a faster MC.
As for gain in speed, it is not that great to make that much of a difference.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
Hello Nick,I have written at length in the past over general performance of 'basic' functions (receiving & storing data, retrieving it and displaying it). I wont repeat my observations and results of empirical testing.
The bottlenecks for these function are likely to be the places that wont benefit from multiple threaded processing (storing and retrieving data & BLITing pixels onto the screen). While TSS should be applauded for taking into account multiple cores the most significant performance gains for basic tasks are likely to achieved by profiling areas of code that are bottlenecks and always will be bottlenecks.
For example a recent bug I have noticed has allowed me to observe chart scaling and positioning in operation. (when an indicator with a margin is at the bottom of the screen it 'flickers up and down' as the chart is re-scaled...looks like the first try at scaling is wrong causing the chart to jump). It made me wonder if the scaling routine is in the multithreaded portion (where it could safely be) or in the display part of the code which I guess wont benefit from multiprocessing. Also from actually being able to see re-scaling working it appears as if the scaling algorithm may be being called more often than it need be.
In short and more generally - code optimisation > more cores.
Cheers,
Nick.
You are right: there are always bottlenecks and we do know that some of our solutions are not optimal and we are constantly working to improve them.
However, one needs to differentiate between a bottleneck and the limits inherent in a particular problem. For example, even the optimal algorithm for finding a solution takes time.
Also, talking about bottlenecks one needs to have a look at a particular case in question. We only emply highly qualified engineers who have a considerable experience in the field. However, the number of possible situations is enormous and there are some that our programmers never come across while developing the product. For example, a user had a problem when a drawing containing an error lead to 100% CPU usage. We got connected to his machine and fixed the problem.
So when you are talking about bottlenecks related to codes, we would like to know what exactly you are talking about.
I think configuration I had is similar to the test you suggested.Still, I would like to know if you ran the test I suggested trying and what were the results? We need some documented proof (e.g. workspaces) of the problem you are describing. When you are talking about one core of the CPU working with the 50% workload: how many workspaces were open? How many charts were plotted? We need to know exactly what was going on. Could you please post or e-mail (mpashkova@tssupport.com) all the worskpaces that were open?
It can be demonstrated with two instances of MC each of which has one workspace.
The key is that the market data needs to be updated quickly as on market open.
Say one with emini ES and one with ER2 or NQ. I will upload workspaces for
those.
That is very difficult since the problem happens in fast markets. My experienceWe could also have a look at the issue via HelpDesk to see how the program behaves on your machine. You can contact us through LiveChat, briefly remind us what the issue is and you will be prompted to run HelpDesk.
with the help desk is that it is very network intensive and would compete with
the market data. Your people have also been frustrated with the very large monitors that I have.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007
We can make an appointment by PM.From what you are saying, it sounds like we still need to have a look at what is going on on your computer through HelpDesk.
Do you think you could choose time when the market is not very active and HelpDesk would not interfere with your trading?
But first, I would like it if you could try what I'm talking about during market open. It only shows up when data rates are high.
- Marina Pashkova
- Posts: 2758
- Joined: 27 Jul 2007