Massive memory usage on MC.net
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
Massive memory usage on MC.net
Hi,
I've been using multicharts.net overnight with 5 workspaces (each workspace roughly have 4 charts) monitoring about 14 different instruments using auto trading. I write my own custom signals and functions.
After leaving them on for a night, the memory usage of Multicharts.exe now is 850 MB! and it keep growing. I'm just curious if there's people having same problem. I'm just trying to figure out if this is a known issue of multicharts.net or this is caused by my own custom stuff.
About my own custom stuff, I don't use any custom code for storage or anything like that (I don't even use any custom library nor do I use any array). I just use standard classes provided by MC.net (e.g. VariableSeries, RSI, AvgTrueRange, etc).
Or, is there any trick to reduce memory usage that I should know?
Thanks,
Felix
I've been using multicharts.net overnight with 5 workspaces (each workspace roughly have 4 charts) monitoring about 14 different instruments using auto trading. I write my own custom signals and functions.
After leaving them on for a night, the memory usage of Multicharts.exe now is 850 MB! and it keep growing. I'm just curious if there's people having same problem. I'm just trying to figure out if this is a known issue of multicharts.net or this is caused by my own custom stuff.
About my own custom stuff, I don't use any custom code for storage or anything like that (I don't even use any custom library nor do I use any array). I just use standard classes provided by MC.net (e.g. VariableSeries, RSI, AvgTrueRange, etc).
Or, is there any trick to reduce memory usage that I should know?
Thanks,
Felix
Re: Massive memory usage on MC.net
Hi there,
I'm no expert in MC.NET by any stretch, but as a start I'd make sure your number of days loaded for each chart is kept low. See if you can get away without using a variableseries type and instead use more simple and typed storage. If you don't need things calculated every tick then disable that, just in case you are making too many new objects too frequently.
That's about all I can suggest at this stage and hope it helps somewhat.
With kind regards,
MK
I'm no expert in MC.NET by any stretch, but as a start I'd make sure your number of days loaded for each chart is kept low. See if you can get away without using a variableseries type and instead use more simple and typed storage. If you don't need things calculated every tick then disable that, just in case you are making too many new objects too frequently.
That's about all I can suggest at this stage and hope it helps somewhat.
With kind regards,
MK
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
The amount of memory in use is highly dependent on your setup. Realtime data is being received, indicators are being calculated and plotted, signals are being generated and orders are being sent to the broker. All of this takes memory and that is natural. The longer you run the platform-the more memory it utilizes.
felixtjung, what was the memory usage once you started MultiCharts and all of the charts were backfilled, indicators plotted, signals calculated? What data resolution do you use (ticks, minutes, days)?
felixtjung, what was the memory usage once you started MultiCharts and all of the charts were backfilled, indicators plotted, signals calculated? What data resolution do you use (ticks, minutes, days)?
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
Re: Massive memory usage on MC.net
Hi Henry,
each chart has following setup:
* 1-hour data resolution
* display 500 bars
* There's no indicator plotted
* only has one signal object
Once MC is ready with auto-trading turned on, the memory usages 207 MB which is still acceptable. The problem is, it grow larger and larger indefinitely. (After I restarted it last night, now it reaches 1 GB memory usage).
As bluejack, pointed out, the beta has memory usage fix. I'll give it a shot and let you guys know how it goes.
Thanks,
Felix
each chart has following setup:
* 1-hour data resolution
* display 500 bars
* There's no indicator plotted
* only has one signal object
Once MC is ready with auto-trading turned on, the memory usages 207 MB which is still acceptable. The problem is, it grow larger and larger indefinitely. (After I restarted it last night, now it reaches 1 GB memory usage).
As bluejack, pointed out, the beta has memory usage fix. I'll give it a shot and let you guys know how it goes.
Thanks,
Felix
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
Re: Massive memory usage on MC.net
Ok,
I've been using MC 8 beta 1. Here's the result:
1. After updating all of my signals class are broken due to change in API. It's fine because the API looks cleaner. Good job guys!
2. Starting MC with all charts with autotrading turned on starts with 147 MB. Perhaps due to the change of the code and the MC improvement that you guys made.
3. I keep it running for 8 hours, and after I checked it, Multicharts.exe uses 438 MB! It still keeps growing.
So, whatever the problem is/are, still present in MC8. It might have something to do with my code or with MC, perhaps the combination of both. I've re-checked my signal code, there's no custom dynamic storage (e.g. List, array or growable data structure) exists in any of my signal code except VariableSeries<double> / standard indicators (e.g. RSI, Lowest, Highest).
I'm not sure if you guys have consider of doing this, perhaps, if you limit the data kept inside those objects to the max number of bar available on chart / max number look back bar of the strategy / whatever that can be used to limit the number of data kept in each ISeries, might solves the problem if that is the problem.
Cheer,
Felix
I've been using MC 8 beta 1. Here's the result:
1. After updating all of my signals class are broken due to change in API. It's fine because the API looks cleaner. Good job guys!
2. Starting MC with all charts with autotrading turned on starts with 147 MB. Perhaps due to the change of the code and the MC improvement that you guys made.
3. I keep it running for 8 hours, and after I checked it, Multicharts.exe uses 438 MB! It still keeps growing.
So, whatever the problem is/are, still present in MC8. It might have something to do with my code or with MC, perhaps the combination of both. I've re-checked my signal code, there's no custom dynamic storage (e.g. List, array or growable data structure) exists in any of my signal code except VariableSeries<double> / standard indicators (e.g. RSI, Lowest, Highest).
I'm not sure if you guys have consider of doing this, perhaps, if you limit the data kept inside those objects to the max number of bar available on chart / max number look back bar of the strategy / whatever that can be used to limit the number of data kept in each ISeries, might solves the problem if that is the problem.
Cheer,
Felix
Re: Massive memory usage on MC.net
Hi,
I left MC.net Beta 8.1 running for like 5 hours now and reached 440.8 MB usage. I have only a DOM window, a 5 min chart going back 20 days and one simple indicator that plots the chart. On regular MC I have m5 chart going back 2 years, m15, m30, h1, h4 and daily charts going back as much as the broker allows (LMAX), all of them with a complex indicator using multiple arrays and drawing but the usage is only 367.7 MB in the same 5 hours.
I have never watched this as I am still learning to code for .NET but I can definitely confirm the memory usage problem.
I forgot to mention that on regular MC on H4 and Daily I am using MB Trading for historical data combined with LMAX realtime.
Both MC's are 64 bit latest release.
Hope this helps.
I left MC.net Beta 8.1 running for like 5 hours now and reached 440.8 MB usage. I have only a DOM window, a 5 min chart going back 20 days and one simple indicator that plots the chart. On regular MC I have m5 chart going back 2 years, m15, m30, h1, h4 and daily charts going back as much as the broker allows (LMAX), all of them with a complex indicator using multiple arrays and drawing but the usage is only 367.7 MB in the same 5 hours.
I have never watched this as I am still learning to code for .NET but I can definitely confirm the memory usage problem.
I forgot to mention that on regular MC on H4 and Daily I am using MB Trading for historical data combined with LMAX realtime.
Both MC's are 64 bit latest release.
Hope this helps.
-
- Posts: 40
- Joined: 28 Aug 2012
- Location: Vancouver, Canada
- Has thanked: 8 times
- Been thanked: 1 time
Re: Massive memory usage on MC.net
I've been monitoring my MC and noticed that it reaches 2 GB in no time! There's a heavy leak somewhere. Hope they fix this or this is going to be a huge problem.
Re: Massive memory usage on MC.net
I must say that the debugger was "hanging" in VS2010 according to what I understood and for some reason MC.net was taking up more memory. After Henry fixed the issue it doesn't seem to use up more than 50 MB compared to 400 this morning with the same workspace (had to compile from PL Editor and reload in VS for the debugger to work).
I'm not sure if this is of any help to you, I thought I should share the experience.
I'm not sure if this is of any help to you, I thought I should share the experience.
Re: Massive memory usage on MC.net
Hello,
I run two workspaces on MC.Net x64 : one of them with 16 one hour charts connected to IB, the other has ~20 charts connected to IQFeeD, no indicators except couple moving averages. Started early in the morning with 147MB running all day without restart, at the end of the day still the same 147MB usage. Windows 7 Ultimate SP1, MC.Net 8.1 beta.
Tested for two days.
I run two workspaces on MC.Net x64 : one of them with 16 one hour charts connected to IB, the other has ~20 charts connected to IQFeeD, no indicators except couple moving averages. Started early in the morning with 147MB running all day without restart, at the end of the day still the same 147MB usage. Windows 7 Ultimate SP1, MC.Net 8.1 beta.
Tested for two days.
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
Re: Massive memory usage on MC.net
I've tried to remove "most" VariableSeries and all stateful indicators (e.g. AverageFC, LowestFC, HighestFC, etc) as much as possible.
I run it after 8 hours until this point. It started off with 133 MB memory usage, now (after 8 hours running) the memory usage is 151 MB. It's still acceptable for "me", so I won't make a big fuss about it.
But, it might still be a big problem for other users. My hunch is, that's something to do with "ISeries data structures", keeping track of enormous amount of data without pruning oldest data. Another guess is it's something to do with keeping track unnecessary data (e.g. series of tick data even though it's no longer needed by higher time resolution such as 5 min chart).
It seems the only memory usage problematic program is Multicharts.exe only. Perhaps, if you guys could memory profile this program, you might be able to see clearly what happens.
I hope that helps.
I run it after 8 hours until this point. It started off with 133 MB memory usage, now (after 8 hours running) the memory usage is 151 MB. It's still acceptable for "me", so I won't make a big fuss about it.
But, it might still be a big problem for other users. My hunch is, that's something to do with "ISeries data structures", keeping track of enormous amount of data without pruning oldest data. Another guess is it's something to do with keeping track unnecessary data (e.g. series of tick data even though it's no longer needed by higher time resolution such as 5 min chart).
It seems the only memory usage problematic program is Multicharts.exe only. Perhaps, if you guys could memory profile this program, you might be able to see clearly what happens.
I hope that helps.
-
- Posts: 40
- Joined: 28 Aug 2012
- Location: Vancouver, Canada
- Has thanked: 8 times
- Been thanked: 1 time
Re: Massive memory usage on MC.net
Yes, it is the MultiCharts64.exe that's memory hungry. On the same note, I'm sure MC developers will come up with a solution so we don't have to keep speculating as to what's causing the problem.
This application seems to be capable of handling complex codes and hopefully we won't need to sacrifice our ideas.
This application seems to be capable of handling complex codes and hopefully we won't need to sacrifice our ideas.
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
We are running some tests in order to reproduce this behavior in our environment.
Re: Massive memory usage on MC.net
What does tsServer process do actually? Can anyone give more details about it, looks like this process incresed in memory after some time, but I have no time to watch it closely, just noticed that is doubled after about 5 hours of running. Multicharts64.exe has no problems at all on my system, its exatly the same size as it was after all day/ half day of using MC.
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
Which data feed and broker do you use?I've been monitoring my MC and noticed that it reaches 2 GB in no time! There's a heavy leak somewhere. Hope they fix this or this is going to be a huge problem.
How many workspaces, charts, symbols, bars, indicators, signals do you have? Is auto trading turned on? Are you using prebuilt studies or custom ones? Please attach a screenshot of your system properties:
For Windows Vista/7 you can find this information here: Control Panel\System and Security\System or right click on the Computer icon and select properties.
For Windows XP you can find this information here: Control Panel\System\General or right click on the Computer icon and select properties.
Please make a screenshot of this window.
You can also come to our live chat Monday-Friday 6:30 am - 4 pm EST to demonstrate the issue remotely and we will collect all required for analysis information.
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
tsServer.exe is in charge of loading data and building bars from lower timeframes (ex. points built from ticks). It loads data, processes it, sends to MC or Portfolio and then releases the memory.What does tsServer process do actually? Can anyone give more details about it, looks like this process incresed in memory after some time, but I have no time to watch it closely, just noticed that is doubled after about 5 hours of running. Multicharts64.exe has no problems at all on my system, its exatly the same size as it was after all day/ half day of using MC.
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
We did some testing for 4 days in our environment and memory usage has not been increased.
We have information from Holya's PC and will be running more tests in order to replicate the behaviour with his setup.
We have information from Holya's PC and will be running more tests in order to replicate the behaviour with his setup.
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
No performance issues have been reported after update to MC .Net 8.1 beta 1
-
- Posts: 24
- Joined: 06 Apr 2011
- Has thanked: 5 times
- Been thanked: 2 times
Re: Massive memory usage on MC.net
How does this problem is marked as "[SOLVED]" when I constantly get this problem all the time!? I've been constantly restarting MC.net every few hours to get around this massive memory usage. I'm using latest MC 8.1 Beta 1 (Build 6048). The problem still persists.No performance issues have been reported after update to MC .Net 8.1 beta 1
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Massive memory usage on MC.net
You need to trace Virtual memory usage of Multicharts components on your PC.How does this problem is marked as "[SOLVED]" when I constantly get this problem all the time!? I've been constantly restarting MC.net every few hours to get around this massive memory usage. I'm using latest MC 8.1 Beta 1 (Build 6048). The problem still persists.
Please compare virtual memory once you start MultiCharts and after running it a certain period of time.
Install Process Explorer http://technet.microsoft.com/en-us/sysi ... 96653.aspx
Go to: View->Select columns->Process memory->check "Virtual Size".
Then you need to sort the processes by the Virtual memory column.
Please send me your results for comparison (support@multicharts.com).
If you are experiencing crashes then you need to send us the logs and dumps for analysis.
After the crash appears, without restarting MultiCharts, please collect the logs manually and send to me for analysis.
Here is how to do that:
https://www.multicharts.com/trading-sof ... harts_Logs