+1 888 340 6572

# Monte Carlo produces incorrect results for many signals and setups

Description

https://www.multicharts.com/discussion/viewtopic.php?f=1&t=50422&p=128094#p128094

The Monte Carlo sim is discounting the correlation effects derived from having multiple signals with positions at the same time, which is wrong. Please see the above link.

The fix is to add radio buttons that allow the user to chose the sampling dataset: trade list OR daily p&ls

That way Multicharts will have a monte carlo sim that can produce correct figures for all but the most basic of signals. More importantly it will correct the production of very misleading and dangerous outputs in its current form (which is why it is a bug and not a feature request).

Steps to reproduce this issue

Simple way to demo it:

Apply a strategy to PT and collect the 5% drawdown metric. Then clone the strategy so you now have 4 in total. Again, collect the 5% drawdown metric. You'll see that the latter metric, divided by 4, will be significantly less that the former metric.

#1
MultiCharts Support
Jun 01, 2018 - 15:03
Hello wilkinsw,
There is no way to "add the function to shuffle daily p&l and not trade list" as the resolutions higher than 1 day may not have enough data points for shuffling.
Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution?
#2
Will
Jun 04, 05:32, in reply to comment #1
"resolutions higher than 1 day may not have enough data points for shuffling"
Can't you simply mark to market daily as is done in the performance report?

"Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution? "

I don't know, have you tested it? "Smoother line" was quite the opposite of what I'm after.

Please look again at "steps to reproduce this issue". All MC has to do is correct the simulator (or add a user defined switch) that stops the false improvement of the drawdown metric when cloning a signal. The simulator is currently nonsense unless you are only using one signal.

I suggest a time bucket solution so that buckets are shuffled, which will mean cloned signals (using 100% cloned signals reproduces the correlation issue at its extreme) will correctly include the negative effects of correlation on the drawdown metric.

I've built a monte carlo sim in excel that uses daily values (I had to, because Multicharts' is so very wrong). I suggest MC test solutions that solve the problem in "Steps to reproduce this issue" OR place a disclaimer in the Wiki that the monte carlo simulator is for single signals only!

Thanks.

#3
Will
Jan 11, 2019 - 20:25

https://www.multicharts.com/discussion/viewtopic.php?t=50422&p=129969#p129969

Annualising the results is also essential else can't make side by side comparisons (and is the norm for monte carlo).

To Annualise: create a fictitious year= Total trades / Trade date range (years) .

e.g. 1000 trades / 10 years = 100 trades = 1 year.

Create X* 1 years via resampling (probably with replacement). X probably >= 100000.

Output percentiles.

#4
MultiCharts Support
Jan 15, 2019 - 08:03
William, I'm afraid we are not getting your idea. Please provide an example of the calculations in Excel.
#5
Will
Jan 15, 08:37, in reply to comment #4
I'll email you my whole sheet. It may take some time for your to understand it though (as it was built only for my eyes).

I'll also have another bash here in describing how I run monte carlo sims. I'm presuming you're stuck on how to annualise??

Currently Multicharts outputs percentiles from their monte carlo sims. But they do so without the context of time.

If I have performance report derived from a signal that started trading in 1/1/2003 (chart start date + maxbarsback) and ends 14/1/2019, I have a report for 16.04 years of trading.

I then want to construct a "typical year" trading. If my performance report has 1150 trades, I do 1150 divided by 16.04 which gives me: 72 trades in an average year.

So, I now want to to randomly sample (with replacement) my performance report and stop once I've reached 72 samples** . I then collect performance metrics of interest. I then repeat this 100,000 times. I finally output the percentiles for the metrics that were collected.

• sample could equal trades or could equal daily P&L (I specifically use the daily mark to market P&L in the daily performance of Multicharts' performance report). I recommend the definition of "sample" be user definable in Multicharts as simply using trades will only produce reliable results on one signal at a time (and nonsense for portfolios).

History
Issue basics
• Type of issue
Bug report
• Category
Not determined
• Targeted for
MultiCharts Future Releases
• Status
Confirmed
• Priority
Not determined
User pain
• Type of bug
Not triaged
• Likelihood
Not triaged
• Effect
Not triaged
People involved
• Posted by
Will
• Owned by
Not owned by anyone
• Assigned to
Not assigned to anyone
• Subscribers
3 subscriber(s)
Times and dates
• Posted at
• Last updated
Issue details
• Reproducability
Not determined
Commits (0)
There are no code checkins for this issue