Big Backtest Memory Problems

I have a problem with running big backtests, my RAM runs out. (16GB)

My question is if i can run the backtest over 3000 symbols and 1 year of backtesting at once without it running into memory problems?

I see now that if i have multiple strategies (that i don't use at the same time) that the RAM usage is already higher of the Portfolio Trader. This seems like a bug.
If i run the Portfolio Trader with only one strategy then the RAM usage is lower.

Please fix this

Hi Rien,

please provide a little more detail about the time resolution you are using. In the absence of that information, I can only guess.

But let's do the math here real quick:
* assume end-of-day bars
* 3,000 instruments x 1 bar / day = 3,000 bars / day
* 252 days / year x 3,000 bars / day = 756,000 bars / year
* 6 x 64 bit / bar x 756,000 bars / year = 18,144,000 bytes / year = 18 MB / year

So, end-of-day bars should fit into memory quite easily. If you are using 1 minute bars instead of daily bars, this will be 390 times as much, or about 7GB. Depending on how complex your strategy is, how many indicators you use, and what else is running on that machine, 16GB might be a little tight.

Hope this helps,
cheers, Felix

Hello, rien!

As we discussed this in Live Chat there are several things you can do:
- decrease the data range,
- decrease the number of symbols,
- distribute your strategies/symbols between several portfolio workspaces.

As for this:

I see now that if i have multiple strategies (that i don't use at the same time) that the RAM usage is already higher of the Portfolio Trader. This seems like a bug.
If i run the Portfolio Trader with only one strategy then the RAM usage is lower.

Do you mean that you have 2 workspaces: one that contains only 1 strategy/symbol and another that has several strategies, only one of which (the same strategy/symbol combination as on workspace #1) is active and backtested, and these workspaces are giving you different RAM load? Please verify that all strategy settings are identical in both cases.