+1 888 340 6572
MultiCharts Project Management
previous_open_issue.png
Go to the previous open issue
previous_issue.png
Go to the previous issue (open or closed)
star_faded.png
Please log in to bookmark issues
bug_report_small.png
Open Bug report MC-2397

Monte Carlo produces incorrect results for many signals and setups

action_vote_minus_faded.png
4
Votes
action_vote_plus_faded.png
next_issue.png
Go to the next issue (open or closed)
next_open_issue.png
Go to the next open issue
Description

https://www.multicharts.com/discussion/viewtopic.php?f=1&t=50422&p=128094#p128094

The Monte Carlo sim is discounting the correlation effects derived from having multiple signals with positions at the same time, which is wrong. Please see the above link.

The fix is to add radio buttons that allow the user to chose the sampling dataset: trade list OR daily p&ls

That way Multicharts will have a monte carlo sim that can produce correct figures for all but the most basic of signals. More importantly it will correct the production of very misleading and dangerous outputs in its current form (which is why it is a bug and not a feature request).

Steps to reproduce this issue

Simple way to demo it:

Apply a strategy to PT and collect the 5% drawdown metric. Then clone the strategy so you now have 4 in total. Again, collect the 5% drawdown metric. You'll see that the latter metric, divided by 4, will be significantly less that the former metric.

Comments (5)
#1
user-offline.png  MultiCharts Support (MultiCharts)
Jun 01, 2018 - 09:03
Hello wilkinsw,
There is no way to "add the function to shuffle daily p&l and not trade list" as the resolutions higher than 1 day may not have enough data points for shuffling.
Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution?
#2
user-offline.png  Will (wilkinsw)
icon_reply.pngJun 03, 23:32, in reply to comment #1
"resolutions higher than 1 day may not have enough data points for shuffling"
Can't you simply mark to market daily as is done in the performance report?

"Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution? "

I don't know, have you tested it? "Smoother line" was quite the opposite of what I'm after.

Please look again at "steps to reproduce this issue". All MC has to do is correct the simulator (or add a user defined switch) that stops the false improvement of the drawdown metric when cloning a signal. The simulator is currently nonsense unless you are only using one signal.

I suggest a time bucket solution so that buckets are shuffled, which will mean cloned signals (using 100% cloned signals reproduces the correlation issue at its extreme) will correctly include the negative effects of correlation on the drawdown metric.

I've built a monte carlo sim in excel that uses daily values (I had to, because Multicharts' is so very wrong). I suggest MC test solutions that solve the problem in "Steps to reproduce this issue" OR place a disclaimer in the Wiki that the monte carlo simulator is for single signals only!


Thanks.




#3
user-offline.png  Will (wilkinsw)
Jan 11, 2019 - 14:25

https://www.multicharts.com/discussion/viewtopic.php?t=50422&p=129969#p129969

Annualising the results is also essential else can't make side by side comparisons (and is the norm for monte carlo).

To Annualise: create a fictitious year= Total trades / Trade date range (years) .

e.g. 1000 trades / 10 years = 100 trades = 1 year.

Create X* 1 years via resampling (probably with replacement). X probably >= 100000.

Output percentiles.

#4
user-offline.png  MultiCharts Support (MultiCharts)
Jan 15, 2019 - 02:03
William, I'm afraid we are not getting your idea. Please provide an example of the calculations in Excel.
#5
user-offline.png  Will (wilkinsw)
icon_reply.pngJan 15, 02:37, in reply to comment #4
I'll email you my whole sheet. It may take some time for your to understand it though (as it was built only for my eyes).

I'll also have another bash here in describing how I run monte carlo sims. I'm presuming you're stuck on how to annualise??

Currently Multicharts outputs percentiles from their monte carlo sims. But they do so without the context of time.

If I have performance report derived from a signal that started trading in 1/1/2003 (chart start date + maxbarsback) and ends 14/1/2019, I have a report for 16.04 years of trading.

I then want to construct a "typical year" trading. If my performance report has 1150 trades, I do 1150 divided by 16.04 which gives me: 72 trades in an average year.

So, I now want to to randomly sample (with replacement) my performance report and stop once I've reached 72 samples** . I then collect performance metrics of interest. I then repeat this 100,000 times. I finally output the percentiles for the metrics that were collected.

    • sample could equal trades or could equal daily P&L (I specifically use the daily mark to market P&L in the daily performance of Multicharts' performance report). I recommend the definition of "sample" be user definable in Multicharts as simply using trades will only produce reliable results on one signal at a time (and nonsense for portfolios).


History
Issue basics
  • Type of issue
    Bug report
  • Category
    Not determined
  • Targeted for
    MultiCharts 13
  • Status
    Confirmed
  • Priority
    Not determined
User pain
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
Affected by this issue (0)
There are no items
People involved
Times and dates
  • Posted at
  • Last updated
Issue details
  • Reproducability
    Not determined
Attachments (0)
There is nothing attached to this issue
Commits (0)
There are no code checkins for this issue
Duplicate issues (0)
This issue does not have any duplicates