+1 888 340 6572
MultiCharts Project Management
previous_open_issue.png
Go to the previous open issue
previous_issue.png
Go to the previous issue (open or closed)
star_faded.png
Please log in to bookmark issues
bug_report_small.png
Open Bug report MC-2397

Monte Carlo produces incorrect results for many signals and setups

action_vote_minus_faded.png
4
Votes
action_vote_plus_faded.png
next_issue.png
Go to the next issue (open or closed)
next_open_issue.png
Go to the next open issue
Description

https://www.multicharts.com/discussion/viewtopic.php?f=1&t=50422&p=128094#p128094

The Monte Carlo sim is discounting the correlation effects derived from having multiple signals with positions at the same time, which is wrong. Please see the above link.

The fix is to add radio buttons that allow the user to chose the sampling dataset: trade list OR daily p&ls

That way Multicharts will have a monte carlo sim that can produce correct figures for all but the most basic of signals. More importantly it will correct the production of very misleading and dangerous outputs in its current form (which is why it is a bug and not a feature request).

Steps to reproduce this issue

Simple way to demo it:

Apply a strategy to PT and collect the 5% drawdown metric. Then clone the strategy so you now have 4 in total. Again, collect the 5% drawdown metric. You'll see that the latter metric, divided by 4, will be significantly less that the former metric.

Comments (2)
#1
user-offline.png  Alex MultiCharts (Alex MultiCharts)
Jun 01, 2018 - 08:03
Hello wilkinsw,
There is no way to "add the function to shuffle daily p&l and not trade list" as the resolutions higher than 1 day may not have enough data points for shuffling.
Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution?
#2
user-offline.png  Will (wilkinsw)
icon_reply.pngJun 03, 22:32, in reply to comment #1
"resolutions higher than 1 day may not have enough data points for shuffling"
Can't you simply mark to market daily as is done in the performance report?

"Currently the Monte Carlo simulation is calculated based on the Equity Curve Close to Close.
That is possible to use the Equity Curve Detailed - using the “per bar” equity values should eliminate the symbol correlation effect and provide a smoother line. Do you consider it a working solution? "

I don't know, have you tested it? "Smoother line" was quite the opposite of what I'm after.

Please look again at "steps to reproduce this issue". All MC has to do is correct the simulator (or add a user defined switch) that stops the false improvement of the drawdown metric when cloning a signal. The simulator is currently nonsense unless you are only using one signal.

I suggest a time bucket solution so that buckets are shuffled, which will mean cloned signals (using 100% cloned signals reproduces the correlation issue at its extreme) will correctly include the negative effects of correlation on the drawdown metric.

I've built a monte carlo sim in excel that uses daily values (I had to, because Multicharts' is so very wrong). I suggest MC test solutions that solve the problem in "Steps to reproduce this issue" OR place a disclaimer in the Wiki that the monte carlo simulator is for single signals only!


Thanks.




History
Issue basics
  • Type of issue
    Bug report
  • Category
    Not determined
  • Targeted for
    MultiCharts 13
  • Status
    Confirmed
  • Priority
    Not determined
User pain
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
Affected by this issue (0)
There are no items
People involved
Times and dates
  • Posted at
  • Last updated
Issue details
  • Reproducability
    Not determined
Attachments (0)
There is nothing attached to this issue
Commits (0)
There are no code checkins for this issue
Duplicate issues (0)
This issue does not have any duplicates