New Monte Carlo simulator

Questions about MultiCharts and user contributed studies.
wilkinsw
Posts: 299
Joined: 21 Apr 2013
Has thanked: 75 times
Been thanked: 49 times

New Monte Carlo simulator

Postby wilkinsw » 05 Jan 2017

Hi,

I've been playing around with the new MC (monte carlo) simulator.

When running on individual signals applied to one instrument, it seems fine. As an example I typically find the 5 percentile worse case drawdown to be about 2-3 times the backtest drawdown. That feels right to me.

However, when I run the MC on the portfolio level, it seems to produce very optimistic 5 percentile drawdown figures. As an example, maybe only 1.25 times the backtest drawdown.

To give more background on the portfolio I'm using: 1 instrument with 3 highly correlated but different signals. The signals are all the product of optimisation using a fitness function featuring max drawdown as a criteria. So how can the 5 percentile drawdown figure, from MC resampling be so close to the backtest base drawdown figure? In fact it shows the base backtest drawdown figure as itself being just below the 20 percentile! That can't be right when max drawdown has been optimised?? You'd expect is to be at least the 50 percentile?

Please correct me if I'm wrong, but I believe what is happening is the following........

The MC simulator is reshuffling trades without any regard for if the trades happened at the same time or not. Therefore, when running multiple signals the simulator is actually helping to artificially smooth out run ups and drawdowns, skewing results optimistically. It is basically using the correlation of signals and instruments in a portfolio and artificially introducing diversification through reshuffling.

For example, if I simply created a portfolio of identical signals with different order names I would end up with a MC median drawdown figure that is very very optimistic and much smaller than if I had just used one signal with a larger order size.

Suggestions:

I think the Monte Carlo simulator needs redesigning. I'd almost argue it's dangerous in it's current form as it promotes under capitalisation.
Might I suggest that the Monte Carlo sim has a user defined button introduced to allow the choice of what is being resampled: Trades or Daily P&L?

Allowing the user to define "daily P&L" as the dataseries to be resampled allows the preservation of correlation between signals and instruments (that can/will lead to a worse/more realistic drawdown).

Other users: please chip in with your thoughts.
These users thanked the author wilkinsw for the post:
tcat

User avatar
Henry MultiСharts
Posts: 8503
Joined: 25 Aug 2011
Has thanked: 1213 times
Been thanked: 2731 times

Re: New Monte Carlo simulator

Postby Henry MultiСharts » 06 Jan 2017

Hello wilkinsw,

Thanks for your feedback. We will discuss it with our engineers.

User avatar
Angelina MultiСharts
Posts: 260
Joined: 28 Dec 2016
Has thanked: 28 times
Been thanked: 66 times

Re: New Monte Carlo simulator

Postby Angelina MultiСharts » 24 Jan 2017

wilkinsw,

This case has been forwarded to development team for analysis.

wilkinsw
Posts: 299
Joined: 21 Apr 2013
Has thanked: 75 times
Been thanked: 49 times

Re: New Monte Carlo simulator

Postby wilkinsw » 01 Feb 2017

Very much looking forward to it getting released.

Currently the Monte Carlo sim only produces relevant data for a single signal, that can only hold one position at a time.

I'm hoping my suggestion of providing an option to resample "daily P&L," instead of trade list, will be a very simple one to implement?

It would then mean you could run simulations on large, complex portfolios.

Can MC provide me with an eta on this fix?

If it's going to take a while then I'll look at purchasing an excel solution instead.
These users thanked the author wilkinsw for the post:
NothingDone

pablo gonzalez vidal
Posts: 33
Joined: 25 Aug 2007
Has thanked: 4 times
Been thanked: 1 time

Re: New Monte Carlo simulator

Postby pablo gonzalez vidal » 10 Nov 2017

Is it possible the shuffled trades monte-carlo analysis is using more of one time the same trade? I´ve been observed it and there is a big difference between MC Monte-Carlos shuffled analysis and the same analysis done with other software like Market System Analyzer. If the trades are used once there woudn't have any difference.

On the other hand the distriution analysis coincides with MSA monte-carlo analysis. It´s like they were changed.

Can someone confirm this observation??
Last edited by pablo gonzalez vidal on 10 Nov 2017, edited 1 time in total.

wilkinsw
Posts: 299
Joined: 21 Apr 2013
Has thanked: 75 times
Been thanked: 49 times

Re: New Monte Carlo simulator

Postby wilkinsw » 10 Nov 2017

pablo gonzalez vidal wrote:Is it possible the shuffled trades monte-carlo analysis is using more of one time the same trade? I´ve been observed it and there is a big difference between MC Monte-Carlos shuffled analysis and the same analysis done with other software like Market System Analyzer. If the trades are used once there woudn't have any difference.

Can someone confirm this observation??


Sampling with replacement is the methodology used. So yes, the same trade can be used twice. Nothing wrong with that IMHO


Multicharts:

When are you adding the feature to shuffle daily P&Ls and not individual trades? I raised this flaw ages ago now.

In its current form, if I had a signal that buys 4 seperate clips of euroFX on a Monday and sells on a Friday..........

The Monte Carlo sim will artificially spread out those 4 clips, improve consistency and reduce risk in the process.

It really is a major limitation and potentially a dangerous one. Please please please add the function to shuffle daily p&l and not trade list.

pablo gonzalez vidal
Posts: 33
Joined: 25 Aug 2007
Has thanked: 4 times
Been thanked: 1 time

Re: New Monte Carlo simulator

Postby pablo gonzalez vidal » 10 Nov 2017

wilkinsw wrote:
pablo gonzalez vidal wrote:Is it possible the shuffled trades monte-carlo analysis is using more of one time the same trade? I´ve been observed it and there is a big difference between MC Monte-Carlos shuffled analysis and the same analysis done with other software like Market System Analyzer. If the trades are used once there woudn't have any difference.

Can someone confirm this observation??


Sampling with replacement is the methodology used. So yes, the same trade can be used twice. Nothing wrong with that IMHO




Captura.JPG
That isn't what they say in the MC guide
Captura.JPG (62.08 KiB) Viewed 60 times

wilkinsw
Posts: 299
Joined: 21 Apr 2013
Has thanked: 75 times
Been thanked: 49 times

Re: New Monte Carlo simulator

Postby wilkinsw » 10 Nov 2017

Hi Pablo,

Apologies, I should've read that you were talking about shuffled trades and not distribution analysis.

I ran a test for you. A signal that generates lots of 1 lot entries and exits. But on one day only.... generates orders for 1000000 lots.

If the shuffled curves were indeed replacing as you suspect (even though MC state it is not) then you'd expect worse case drawdowns much much larger than baseline. Which I didn't find. Worst case was about the same as baseline. Ie we never generated a curve that sampled the massive trades more frequently than baseline....which would appear to be representative of sampling without replacement.

demo of no replacement.PNG
demo of no replacement.PNG (174.46 KiB) Viewed 52 times


If you disagree, I welcome evidence.
These users thanked the author wilkinsw for the post:
pablo gonzalez vidal


Return to “MultiCharts”