MultiCharts Easter Sale has jumped in! Up to 50% off Explore offers
+1 888 340 6572
MultiCharts Project Management
previous_open_issue.png
Go to the previous open issue
previous_issue.png
Go to the previous issue (open or closed)
star_faded.png
Please log in to bookmark issues
feature_request_small.png
Open Feature request MC-1026

Walk-Forward Logic is Flawed

action_vote_minus_faded.png
16
Votes
action_vote_plus_faded.png
next_issue.png
Go to the next issue (open or closed)
next_open_issue.png
Go to the next open issue
Description

I don't believe Multichart's walk-forward logic is correct. As has been confirmed, Multicharts is using the best optimization inputs for each test. This, in itself, is not acceptable since it would include the most outliers with the most curve fitting, leading to poor results. It is critical that the Multicharts development team either writes an algorithm to automatically find clusters of similar inputs or allows the user to select which inputs to use for each test (possibly with the aid of 3D visualization). Tradestation offers a similar feature, however it is completely useless since TS does not use "extended" backtesting. I think this is a VERY important issue that needs to be addressed and corrected ASAP.

Steps to reproduce this issue

Logic issue, not a bug, per se.

Comments (6)
#0
user-offline.png  JoshM (Jura)
Jun 27, 2012 - 08:19

+1; totally agree.

#0
icon_locked.png
user-offline.png  GB (Id2)
Dec 14, 2012 - 16:54
This is huge, basically making walk forward un usable. Which is a real shame. I bought this assuming it would work and i have found the same thing. I would also like to look at each output for each period. So i can evalute myself. Basically this thing is worthless. I cant believe i spent money for a lifetime license and have this problem.
#0
user-offline.png  MeiHua1 (MeiHua1)
Dec 14, 2012 - 16:59

The way that the optimizer works, as well as the out put is totally un usable. I have been doing research on why it only outputs the best result and how it generated it. Its basically worthless. I have to use completely other software because of this. Actually this is a reason I want to switch from multicharts. You can not develop trading systems with out this. Which is what multicharts is for. If this is not changed quickly then I will be leaving.

#0
icon_locked.png
user-offline.png  GB (Id2)
Dec 14, 2012 - 20:59

Since my original post, I have determined that MC DOES NOT backtest properly, even if you use so-called "extended testing" with bid and ask data.  Since MC doesn't capture millisecond data, any backtested results will be useless; w/o sub second data, there is no way to know the price action as a bar is created AND how a strategy would behave.  To verify this, all you have to do is create a chart real time with strategy signals and then copy the same chart to a new workbook and let MC generate hypothetical trades; you will see immediately that not only are fills different, but trades are as well!  And I obtained these minor differences with IOG off, using MC's stock crossover MA modules!  Once you turn IOG on you can completely disregard the results as pure fantasy.  My experience shows that even a few ticks can make the diffference between success and failure for a day trading system.  So please be aware of the these issues before you blow out your account...

#1
user-offline.png  Zheka (Zheka)
Apr 06, 2017 - 18:26

A great solution would be to extend the "custom fitness function" functionality to WFO, exposing all performance variables available in MC (incl. e.g. maxposprofit). And if it were possible to have such functions accept input parameters and make those optimizable - that would be a "dream-come-true"!

#2
user-offline.png  MultiCharts Support (MultiCharts)
Jan 31, 2018 - 15:22
Hello Z-Man,

Please provide a detailed description of the functionality you are looking for. What exactly is incorrect in MultiCharts and TradeStation WFO? How exactly do you want to determine/define the proper inputs for WFO? Have you tried using the Custom optimization criterion in MultiCharts to find the desired inputs? Are you referring to cluster analysis in TS? If yes - what do you want to see in 3d visualization? Which parameters do you want to change/switch?
History
Issue basics
  • Type of issue
    Feature request
  • Category
    Usability
  • Targeted for
    Not determined
  • Status
    Need Details
User pain
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
Affected by this issue (3)
People involved
Times and dates
  • Posted at
  • Last updated
Issue details
  • Resolution
    Not determined
  • Severity
    Critical
Attachments (0)
There is nothing attached to this issue
Commits (0)
There are no code checkins for this issue
Duplicate issues (0)
This issue does not have any duplicates