I know it's not helpful to post this without debugging logs, but I wanted to share this information with other users in case they have same problems. It happens if you don't restart MC once a day and get a disconnection on historical market data farm on InteractiveBrokers. MC gets into memory leak mode and runs out of memory...
MC 64-bit - 9.1.12010
Multicharts - Memory Leak
-
- Posts: 79
- Joined: 28 Mar 2013
- Has thanked: 3 times
- Been thanked: 4 times
- Contact:
Re: Multicharts - Memory Leak
I receive a memory leak and low memory warning after or during long optimization runs.
For example just finished a 9 day genetic test and windows says to close MC. This happens on a repeatable basis for long tests. I'd really need MC to work on a test for several weeks so this is an issue for me.
For example just finished a 9 day genetic test and windows says to close MC. This happens on a repeatable basis for long tests. I'd really need MC to work on a test for several weeks so this is an issue for me.
- Attachments
-
- Screen Shot 2016-02-21 at 9.12.53 AM.png
- (105.25 KiB) Downloaded 1043 times
- TJ
- Posts: 7752
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1034 times
- Been thanked: 2228 times
Re: Multicharts - Memory Leak
How large is your optimization?I receive a memory leak and low memory warning after or during long optimization runs.
For example just finished a 9 day genetic test and windows says to close MC. This happens on a repeatable basis for long tests. I'd really need MC to work on a test for several weeks so this is an issue for me.
How long is the history?
How many runs?
Were you running other programs? Browsers? email? calendar? Messengers?
Re: Multicharts - Memory Leak
How large is your optimization?: 256,217 simulations
How long is the history?: Using data1 and data2 (ascii). 5 minute bars, 5 years of history.
How many runs?: Population 17081, Generations 14946
Were you running other programs? Browsers? email? calendar? Messengers?: No nothing is open, I close all that I can and run the opt after a fresh reboot.
I have tried adding more ram (from 8gb to 16gb) but on the 7th day or so the ram is quickly gobbled up in the next days.
How long is the history?: Using data1 and data2 (ascii). 5 minute bars, 5 years of history.
How many runs?: Population 17081, Generations 14946
Were you running other programs? Browsers? email? calendar? Messengers?: No nothing is open, I close all that I can and run the opt after a fresh reboot.
I have tried adding more ram (from 8gb to 16gb) but on the 7th day or so the ram is quickly gobbled up in the next days.
Re: Multicharts - Memory Leak
Now I remember running into this issue in the past.
Currently running a large test and what I think is happening is that the committed memory creeps up and then the hard drive starts to run out of space then it crashes.
Often stopping MC before the crash still results in losing optimization results, the memory doesn't get fixed until a reboot.
Creeping up about 2-3Gb a day.
Currently running a large test and what I think is happening is that the committed memory creeps up and then the hard drive starts to run out of space then it crashes.
Often stopping MC before the crash still results in losing optimization results, the memory doesn't get fixed until a reboot.
Creeping up about 2-3Gb a day.
- Attachments
-
- Screen Shot 2016-02-26 at 11.05.02 AM.png
- (99.17 KiB) Downloaded 1038 times
Re: Multicharts - Memory Leak
Shane I've found the same thing.
If I don't shutdown and reboot the instance once per day with the Portfolio Trader, that it causes all kinds of problems with live trading. Most specifically, it won't trade. Even if I restart Portfolio Trader and TWS and go into Autotrading, it won't process any incoming data or send orders to IB.
In the portfolio trader I'm running 30 symbols, 2 minute bars on data1, daily on data2. If I do more than a 2 year back test, I'll get low memory warnings. This is a c4.large instance with 4GB. The instance is only running Portfolio Trader and TWS.
With 16gb (c4.2xlarge AWS), I've been able to run 400 symbols on 2 minute bars, 2 years back successfully. I get memory errors however it doesn't crash. Any more than that and I'll likely get the BSOD somewhere during the backtest.
If I don't shutdown and reboot the instance once per day with the Portfolio Trader, that it causes all kinds of problems with live trading. Most specifically, it won't trade. Even if I restart Portfolio Trader and TWS and go into Autotrading, it won't process any incoming data or send orders to IB.
In the portfolio trader I'm running 30 symbols, 2 minute bars on data1, daily on data2. If I do more than a 2 year back test, I'll get low memory warnings. This is a c4.large instance with 4GB. The instance is only running Portfolio Trader and TWS.
With 16gb (c4.2xlarge AWS), I've been able to run 400 symbols on 2 minute bars, 2 years back successfully. I get memory errors however it doesn't crash. Any more than that and I'll likely get the BSOD somewhere during the backtest.
Re: Multicharts - Memory Leak
That sounds unfortunate. I'm going to keep troubleshooting and see if I can figure out a way to prevent this.
I'm needing to run optimizations for 60 days straight.
Will post any findings here.
I'm needing to run optimizations for 60 days straight.
Will post any findings here.