Backtest results discrepancies

Has anyone noticed discrepancies in backtesting results between EL code running in MC and other popular backtesting platforms? I was evaluating a new system code generator from Price Action Lab and I noticed slight differences between the EL code results running in MC with that running on another popular platform. Initially I thought this was due to cancellation of simultaneous long and short signals but apparently that was not the problem.

Does anyone know if MC determines whether a stop is hit before the target or vice versa based on relative location of open price to high and/or low of a bar? I think this was done in TS but I am not sure if it still being used. I went over dozens of trades manually and I was not able to find the few trades that cause the discrepancy.Note that when the trade sample is small I do not see any differences between MC and the results of other platform. This is only happening when the backtest involves hundreds of trades and a manual check is hard. Any ideas will be appreciated.

Are you using the new code generation tool that was released for PAL this week or the standard one?

Either way, you should turn on the Bar Magnifier and set the appropriate backtesting assumptions. It is not too surprising that you are finding discrepancies. I find MC to be the least optimistic of the trading tools I have tested. This is a good thing in my opinion.

As far as PAL is concerned, I don't find the code generation very useful. You are much better off taking the raw patterns and starting from scratch. Other than that, PAL is a pretty useful tool.

Are you using the new code generation tool that was released for PAL this week or the standard one?

Thanks. I am using the new tool. I had a license in the past that expired and when I inquired to renew it I was offered that in exchange for providing some feedback. I'm not an expert coder and this feature helps.

Either way, you should turn on the Bar Magnifier and set the appropriate backtesting assumptions. It is not too surprising that you are finding discrepancies. I find MC to be the least optimistic of the trading tools I have tested. This is a good thing in my opinion.

Thanks for this suggestion. I agree least optimistic is a good think. You do not want to be too optimistic with backtests.

As far as PAL is concerned, I don't find the code generation very useful. You are much better off taking the raw patterns and starting from scratch. Other than that, PAL is a pretty useful tool.

I was initially confused by your reference to the <raw patterns> but then I looked at the PAL manual I found out that this refers to code for the patterns without entry and exit conditions, risk management etc. I think this is a good suggestion for learning how to code systems.

If you are using this tool and you can share a few tips that would be of great help. I'm trying to get some systems in ways similar to those employed in the blog (ex. in here http://bit.ly/1Hahts3).

There are many ways to use PAL but it depends on your trading style and what you want to achieve. I use it on intraday data which is not the way Mike Harris recommends. It also depends on how many systems you intend to build and manage.

I use Rapidminer to store and split in sample and out of sample data and the resulting raw patterns are kept in a SQL database. I use Codesmith Generator to pull patterns out of the sql database into Easylanguage templates which then get compiled. This was the best workflow that I could come up with to make the process as automated and error free as possible.

There are many ways to use PAL but it depends on your trading style and what you want to achieve. I use it on intraday data which is not the way Mike Harris recommends. It also depends on how many systems you intend to build and manage

When I first got my license I used the scan function and that worked well. Now I want to use it to develop systems for position trading. I am not into intraday trading for now but I may get into it later.

I use Rapidminer to store and split in sample and out of sample data and the resulting raw patterns are kept in a SQL database. I use Codesmith Generator to pull patterns out of the sql database into Easylanguage templates which then get compiled. This was the best workflow that I could come up with to make the process as automated and error free as possible.

You are way too advance for me but it is always good to get an idea of what others do. For me this is way down the road. Do you have a tech support team or you manage all that yourself? Sounds like hedge funds could use your services.

Thank you for your insightful comments and suggestions.

Lol, this is all very easy to manage but it is essential to organize yourself. If you want, you can contact me via PM and I will help you. If you want to automate your trading this way then you will need to do everything you can to avoid trading random patterns.

Alex

Thank you Alex. I'm trying to catch up with Price Action Lab new version and when I'm ready I'll ask you for some help with using Codesmith generator to create systems in MC from PAL output. This route sounds quite interesting.

One last thing: In trying to deal with the random patterns I look for larger sample size. I'm also experimenting with the portfolio backtest tool of PAL but applying it only on the out-of-sample data. What is your opinion of this?

Thanks again.

Alex, I hope you're still around and doing well. It's been two years already, time flies. Your help in using PAL was invaluable. I've done well. I'm now using the DLPAL version with success. I've started testing simple ML models taking advantage of new functionality but my Python skills are not at sufficient level yet. My question is about backtest validation on intraday data. Have you done any? How do PAL backtest results fair when using say 5 minute or one minute data?

I saw an article in PAL blog (http://bit.ly/2ssJMFL) where test in another platform with one minute data doubled returns instead of expected decrease. Is it possible that PAL backtests are too conservative estimates as per that article? If that is true, is it important in your opinion to also test systems developed with daily data also with intraday data in another platform rather than solely relying on PAL backtests, which I was doing ?