OPTIMIZATION AND BACKTESTING BUGS

Questions about MultiCharts and user contributed studies.
User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

OPTIMIZATION AND BACKTESTING BUGS

Postby Marina Pashkova » 25 Aug 2008

Hi everybody,

We keep working on the official MultiCharts release.

Please, report the bugs in backtesting and optimization that you have encountered so far and that need to be addressed here. Also, please try to be as specific in your comments as possible. When reporting a bug, please provide

1. a detailed description
2. screenshots with the problem areas marked

We might also request additional infomation if the above isn't enough.

Thank you for your input.
Last edited by Marina Pashkova on 16 Oct 2008, edited 2 times in total.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 25 Aug 2008

Marina,

Thank you for admitting with this post that there might be a problem with the backtester :D

Guys,

Let's come up with reports on the Backtester so that TSS could make the product we all use better. If we miss this opportunity to do it now, we might never get another chance like this.

Thank you.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 25 Aug 2008

OK,

This is my first and minor report on the Backtester (more to come later when I find time). Version MC 3.1.1353.400

The problem is: when I apply strategy signals on the chart and then backtest in the Backtester the same strategy with the same signals' inputs with the same data then the first signals on the chart are not the same as first signals in the list of trades from a performance report.

Some trades are missing in the chart and same are additionally added. I attach two screenshots that explain this issue. To make it short: there are different trades on the chart and yet different trades in the backtester's list of trades.

I will be able to demonstrate this bug live to customer support after I report additional bugs.
Attachments
Chart.jpg
(174.67 KiB) Downloaded 2292 times
List of Trades.jpg
(216.81 KiB) Downloaded 2297 times

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 26 Aug 2008

Hi Tresor,

The differences may be accounted for by the different MaxBarsBack value set for the strategy in the portfolio as compared to the MaxBarsBack value set for the strategy in the chart. Also, maybe the capital that you specified in portfolio is not sufficient to take all the trades.

The fastest way to find out what is wrong will be to contact our customer support via LiveChat.

Regards.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 27 Aug 2008

Hi Tresor,

The differences may be accounted for by the different MaxBarsBack value set for the strategy in the portfolio as compared to the MaxBarsBack value set for the strategy in the chart. Also, maybe the capital that you specified in portfolio is not sufficient to take all the trades.

The fastest way to find out what is wrong will be to contact our customer support via LiveChat.

Regards.
Hello Marina,

I checked your suggestion and indded the mismatch was caused by the different MaxBarsBack value set for the strategy in the portfolio as compared to the MaxBarsBack value set for the strategy in the chart.

Sorry for troubling you.

Regards

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 28 Aug 2008

Hi everybody,

It is quite sad more than 60 hours after Marina started this there are ONLY 187 views and Tresor is the LONE (non TS Support) participant.

I would have participated fully but right now I am do not have a trading computer -- hence no screen shots, Excel files to attach. I am changing my computer and the new one shall be delivered in another 48 hours.

Khalid

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 28 Aug 2008

Since we have a limited number of volunteers contributing to this thread I decided to once again raise the issue of Max Intraday Drawdown to make sure this issue is on the list for fixing.

I attach 3 screenshots of a strategy that (i) I first optimized and after selecting the settings that I liked best (ii) I backtested the strategy. The strategy was run on 3.5 year data and generated 530 trades:

1 Optimization Report.jpg - I selected the inputs that resulted in Net Profit of $7340 and Max Intraday Drawdown of $7131

2 Performance Report Profit.jpg - I backtested the above inputs; the profit of $7340 was reflected perfectly in Performance Report

3 Performance Report Equity Curve.jpg - as you can see the equity curve smoothly reaches the profit of $7340 but the equity drawdowns are smaller than $7131. In fact Max Aggregated Equity Drawdowns over a number of days (with consecutive losers) do not exceed $600.

To the best of my believe Performance Report displays Equity Drawdown properly (Aggregated) but the Optimization Report miscalculates the Intraday Dradowns (they are unlogically huge).

I do believe that the Optimization Report calculates the Max Intraday Drawdowns according to a very specific definition that the engineers were given (and for sure they coded it perfectly), but clearly this definition is neither logical nor is close to:

''Displays the greatest loss drawdown, from the previous highest equity run-up, bar to bar looking across all trades, during the specified period. If a new bar equity run-up high occurs, the low equity value is reset to 0 so that the next maximum drawdown can be calculated from that point.<br>You can roughly see this value on a detailed equity curve graph by looking from the highest peaks to the lowest peaks moving forward.''


To get an idea of how unlogical is the logic of Max Intraday Drawdown just compare the two values in 1 Optimization Report.jpg:

- Gross Loss over the whole 3.5 year period = $7823
- Max Intraday Drawdown at some point during 3.5 year period = $7131

To sum it up the way Max Intraday Drawdown is calculated and displayed makes no practical sense to a trader. No good analysis can be made based on this. The logic of Max Intraday Drawdown should be changed or MID itself would need to be replaced with something else. Or MID would need to be removed from the Optimization Report as it is misleading.

Could a trader with an analytical mind support me and explain this issue in a programming jargon please?

Regards

Please see the screenshots in the following order: FROM BOTTOM TO TOP
Attachments
1 Optimization Report.jpg
(290.07 KiB) Downloaded 2293 times
2 Performance Report Profit.jpg
(100.18 KiB) Downloaded 2288 times
3 Performance Report Equity Curve.jpg
(167.3 KiB) Downloaded 2284 times

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 28 Aug 2008

Tresor,

If possible, please attach the EXCEL report.

Thank you.

Khalid

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 28 Aug 2008

Tresor,

If possible, please attach the EXCEL report.

Thank you.

Khalid
Sure,

I attach the Optimization Report (csv converted to xls; the line painted red was backtested) + Performance Report. The strategy was tested on 13 minute bars.

Regards
Attachments
Report_2008-08-29_02-13-18.xls
(56.5 KiB) Downloaded 189 times
Portfolio Performance Report.xls
(561.5 KiB) Downloaded 199 times

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 29 Aug 2008

This post will be about Margin calculation. If I am not mistaken most of us trade futures contracts and want to optimize and backtest strategies for futures.

At the moment MC does not support margin calculation. There is one blank box which says ''Margin'' but it is actually not taken into calculation by Portfolio Backtester.

I think It is high time to make backtesting / optimization suitable for futures.

Generally, there are two types of Margins. A broker may want the margin as:
(i) fixed amount of dollars, or
(ii) percentage of the contract value.

''Fixed amount of dollars'' is easy. Below I show how to calculate the Margin as a percentage of the contract value (12.5% margin):

points on the chart: 3700 x 25 BPV = $/Euro/other currency 92,500 (value of the contract) x 12.5% = $/Euro 11,562.5 margin.

I attach a screenshot to show how to create boxes for two types of Margins in QM.

Regards
Attachments
Margin calculation.jpg
(94.86 KiB) Downloaded 2284 times

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 29 Aug 2008

Another issue that is known to all but seems to have been neglected are reversals. MC thinks there is not sufficient cash in the account and one has to artificially double one's sccount to be able to backtest any reversal strategy.

Without this being fixed no proper calculation on return on account / size of the account / etc can be made and no reliable analytical conclusions can be drawn.

Please fix it.

Regards

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 29 Aug 2008

MC has won the battle with its competitors on number of cores on which startegies can be optimized. But at the same time MC is losing a battle on ease of use of its bactester.

Please watch this tutorial http://www.tradersstudio.com/Tutorials/ ... fault.aspx

and implement Chart - Performance Report synchronizing. With such a feature MC user would easily see if his / her strategy is performing better in choppy / trending / other market condictions. Implementing this feature would put MC again on track with its competitors as far as optimizing goes and would make traders' lives much easier.

Regards
Attachments
Chart - Performance Report Synchronizing.jpg
(136.39 KiB) Downloaded 2283 times

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 29 Aug 2008

After considering the implementing of the Synchronizing feature you may also want to make one step further and watch this video tuturial on algorthmic trading with MATLAB: http://www.mathworks.com/company/events ... 7&p2=50649

It is about creating strategies and optimizing them the way big hedge funds do. Seeing this tutorial requires 30 sec registering, but it is worth it.

The Optimization Report by MATLAB not only shows the best input parametres like lengths but it can also show you best resolution for the strategy (5 / 15 / 20 minutes).

I attach a screenshot of such 3D presentation. Please compare it to MC 3D presentation (which sometimes is illegible to read) :(

Regards
Attachments
3D Matlab Optimization.jpg
(88.09 KiB) Downloaded 2300 times
3D MC.jpg
(26.22 KiB) Downloaded 2306 times

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 29 Aug 2008

Since we have a limited number of volunteers contributing to this thread I decided to once again raise the issue of Max Intraday Drawdown to make sure this issue is on the list for fixing.

I attach 3 screenshots of a strategy that (i) I first optimized and after selecting the settings that I liked best (ii) I backtested the strategy. The strategy was run on 3.5 year data and generated 530 trades:

1 Optimization Report.jpg - I selected the inputs that resulted in Net Profit of $7340 and Max Intraday Drawdown of $7131

2 Performance Report Profit.jpg - I backtested the above inputs; the profit of $7340 was reflected perfectly in Performance Report

3 Performance Report Equity Curve.jpg - as you can see the equity curve smoothly reaches the profit of $7340 but the equity drawdowns are smaller than $7131. In fact Max Aggregated Equity Drawdowns over a number of days (with consecutive losers) do not exceed $600.

To the best of my believe Performance Report displays Equity Drawdown properly (Aggregated) but the Optimization Report miscalculates the Intraday Dradowns (they are unlogically huge).

I do believe that the Optimization Report calculates the Max Intraday Drawdowns according to a very specific definition that the engineers were given (and for sure they coded it perfectly), but clearly this definition is neither logical nor is close to:

''Displays the greatest loss drawdown, from the previous highest equity run-up, bar to bar looking across all trades, during the specified period. If a new bar equity run-up high occurs, the low equity value is reset to 0 so that the next maximum drawdown can be calculated from that point.<br>You can roughly see this value on a detailed equity curve graph by looking from the highest peaks to the lowest peaks moving forward.''


To get an idea of how unlogical is the logic of Max Intraday Drawdown just compare the two values in 1 Optimization Report.jpg:

- Gross Loss over the whole 3.5 year period = $7823
- Max Intraday Drawdown at some point during 3.5 year period = $7131

To sum it up the way Max Intraday Drawdown is calculated and displayed makes no practical sense to a trader. No good analysis can be made based on this. The logic of Max Intraday Drawdown should be changed or MID itself would need to be replaced with something else. Or MID would need to be removed from the Optimization Report as it is misleading.

Could a trader with an analytical mind support me and explain this issue in a programming jargon please?

Regards

Please see the screenshots in the following order: FROM BOTTOM TO TOP
Hi Tresor,

Intraday Drawdown IS calculated incorrectly in Optimization. We have already fixed this issue for the regular optimization and are working on the fix in the portfolio module. The fix will be available in the upcoming beta 2 version of MC.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 29 Aug 2008


Intraday Drawdown IS calculated incorrectly in Optimization. We have already fixed this issue for the regular optimization and are working on the fix in the portfolio module. The fix will be available in the upcoming beta 2 version of MC.
Marina thank you, a member of this forum has PMed and raised the issue of ''Return on Drawdown percentage'' which is supposedly miscalculated, as well:

''Another issue is the "Return on Drawdown percentage" which is negative when you have a profitable system. For example, if drawdown is 10 and your account ending value is 20, then return on drawdown should be 100%. "((20 - 10)/10) * 100 = 100". However MC shows it as -100% because MC reason that drawdown is a negative value and calculated it as: "((20 - 10)/-10) * 100 = -100".

This means if you have a losing system and your account value is -20 in the end, MC will say your return on the account is +100%, which doesn't make sense at all. ''

Can your development guys check it?

Regards

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 30 Aug 2008


Intraday Drawdown IS calculated incorrectly in Optimization. We have already fixed this issue for the regular optimization and are working on the fix in the portfolio module. The fix will be available in the upcoming beta 2 version of MC.
Marina thank you, a member of this forum has PMed and raised the issue of ''Return on Drawdown percentage'' which is supposedly miscalculated, as well:

''Another issue is the "Return on Drawdown percentage" which is negative when you have a profitable system. For example, if drawdown is 10 and your account ending value is 20, then return on drawdown should be 100%. "((20 - 10)/10) * 100 = 100". However MC shows it as -100% because MC reason that drawdown is a negative value and calculated it as: "((20 - 10)/-10) * 100 = -100".

This means if you have a losing system and your account value is -20 in the end, MC will say your return on the account is +100%, which doesn't make sense at all. ''

Can your development guys check it?

Regards
Tresor,
Could you make a screenshot of the "Return on Drawdown percentage" ratio?
Our guys have analyzed the problem and they found a few ratios that work the way you describe, but we don't have the "Return on Drawdown percentage" measure. I want to double check that we are talking about the same thing, so please make a snapshot.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 30 Aug 2008

Tresor,
Could you make a screenshot of the "Return on Drawdown percentage" ratio?
Our guys have analyzed the problem and they found a few ratios that work the way you describe, but we don't have the "Return on Drawdown percentage" measure. I want to double check that we are talking about the same thing, so please make a snapshot.
Andrew,

My bad, sorry, I just pasted a message I got from one guy from this forum and did not check if this is a real issue or a bogus one. Please accept my apologies for this.

As we are approaching the issue of statistics, please read the post that I will post in 5 minutes with some ideas.

Regards

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 30 Aug 2008

I am a futures contract trader. When one trades a leveraged instrument the most important measure of a given strategy's performance is the linearity of the equity curve rather than the Net Profit.

A strategy that makes $100 million (on paper) in 2 years but has many severe equity drowdowns may in a final effect be worst than a strategy that makes only $ 5 million with small drawdowns.

Please see an equity curve of almost perfect strategy (Equity linearity close to 1.jpg) that I have from kreslik.com

I would like the Optimization Report to include some additional statistics:
- Equity linearity (where 1 would be a perfect linearity and 0 would represent the worst choppines of the equity line);
- Linearity / Net profit ratio - so that one could level out between linearity measure and Net profit
- Max Portfolio Close to Close Drawdown (forgot to show this one on the screenshot)
- Recovery time from Max Portfolio Close to Close Drawdown
- Sharpe ratio
- other statistics that MC users may be willing to post here

Please have a look at the linearity issue disscussed here: http://kreslik.com/forums/viewtopic.php ... pe&start=0

You and MC users may also want to have a look at this product (which is MC compatible) and provides a number of data performance metrics that could be inspirational for Portfolio Backtester' further development: http://www.meyersanalytics.com/pwfo.php

It would be also okay if one could define the sorting criteria before the optimization starts, like shown on the screenshot Equity line statistics.jpg

Regards
Attachments
Equity linearity close to 1.gif
(13.07 KiB) Downloaded 2294 times
Equity line statistics.jpg
(203.79 KiB) Downloaded 2324 times
Last edited by Tresor on 30 Aug 2008, edited 2 times in total.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 30 Aug 2008

I thought we are focusing on bugs?

The above are enhancement request/suggestions. They are important, but we should not confuse the pressing issues at hand. Otherwise the mixed emotions will get us very frustrated.

Maybe a new thread on feature suggestion is more appropriate?

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 30 Aug 2008

I thought we are focusing on bugs?

The above are enhancement request/suggestions. They are important, but we should not confuse the pressing issues at hand. Otherwise the mixed emotions will get us very frustrated.

Maybe a new thread on feature suggestion is more appropriate?
Hi TJ,

For many reasons it is better to have all issues in one thread. One of such reasons can be to make TSS aware of what users want with regard to the optimization / backtesting tool.

I asked Marina to start this thread so that users could report bugs and make suggestions on development.

No one is going to get frustrated (at leat not me) beacuse unlike other threads on optimization, this one is deprived of emotions.

Regards

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 01 Sep 2008

In this post I would like to summarize the input that I made in this thread so far and share some thoughts with you:

MC is promoted on TSS’s main website as a professional platform featuring (i) professional charting, (ii) advanced analytics and (iii) trading strategy optimization and backtesting.

There is no doubt that MC is superior in charting and analytics capabilities than any other similarly priced competing product!!! You were given congrats many times on different internet forums. We, the customers, like MC’s charting and its advanced analytics.

The catch is though that while in terms of charting MC is years ahead of similarly priced products, it unfortunately fails to compete in terms of optimization / backtesting. I hope we can all agree that a product that exaggerates drawdowns by as much as 10 times (now admitted by TSS - thank you for this) and does not allow for reversals, simply cannot compare to products that can do the above properly. MC users addressed this issue many times in many forums.

I think the most urgent for Backtester is to:
(i) get rid of the two already described bugs (drawdowns’ miscalculation, inability to reverse position)
(ii) include Margin calculation


The two above are the NECESSARY prerequisites to consider optimizing / backtesting being at least satisfactory and will keep MC users away from grumbling about backtesting / optimizing for the next year or two in this and other forums. Not a big price for keeping customers happy, is it?

There should be no compromise whether to fix it or not, as without fixing it the Bactester remains only a toy. Please do not hesitate to shift your programmers working on another bell and whistle for charting and get them making Backtester work properly.

TSS, please fix the above. Failing to do so would in fact mean that the customers will keep on using MC solely for charting and that all the efforts you put to Backtester’s R&D and its coding went down the drain. These are only two issues :) Shouldn’t be difficult. PLEASE provide ETA for this.

In previous posts I proposed additional features that you might consider. If I were to prioritize them, I would list them in the following order (from most to least wanted):

1. Additional analytics in Optimization report to include:
- Equity linearity
- Linearity / Net Profit Ratio
- Max Portfolio Close to Close Drawdown
- Recovery Time from Max Portfolio Close to Close Drawdown
- Sharpe Ratio

2. Chart – Performance Report Sychronizing similar to Tradersstudio

3. Resolution Optimization similar to MATLAB


Marina and Andrew,

Please have a brainstorming in your company and decide whether features described in points 1 – 3 are going to help traders thus addvaluing your product. I personally think that implementing these features would in a way level out the imbalance between the feature-rich charting and analytics and feature-poor optimization / bactesting. And please let us know your decision. No ETA needed here, no ‘’we will consider’’, just consider and say simple YES or NO.

Thank you

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 03 Sep 2008

Dear Tresor,

For now, we are planning to concentrate on resolving the following issues:

1. Make sure drawdown is calculated correctly
2. Fix the bug with position reversal to make sure you don't need to have the double capital size to reverse
3. Add margin calculation (and as a consequence make it possible to backtest futures)
4.Provide information on how idividual symbols are traded within portfolio

Once we have fixed those, we will start considerting the features that you listed above.

Thank you.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 03 Sep 2008

Dear Tresor,

For now, we are planning to concentrate on resolving the following issues:

1. Make sure drawdown is calculated correctly
2. Fix the bug with position reversal to make sure you don't need to have the double capital size to reverse
3. Add margin calculation (and as a consequence make it possible to backtest futures)
4.Provide information on how idividual symbols are traded within portfolio

Once we have fixed those, we will start considerting the features that you listed above.

Thank you.
Marina,

THANK YOU. THAT'S EXACTLY WHAT I WANTED TO HEAR :D

oakshadows
Posts: 33
Joined: 01 May 2007

Postby oakshadows » 10 Sep 2008

Version 4.0 Beta 2 (Build 1573) of MC miscalculates the slippage and commission in backtesting. The amount you enter is not the amount deducted from the trade in the historical results. (Yes; I took into account "per trade" vs "per share/contract" and yes, I took into account that the amount assigned in the Signal Properties is per side.)

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 12 Sep 2008

Hi All MultiCharts Stakeholders,

The MultiCharts 4.0 Beta 2 (Build 1573) Back-Testing Strategy Performance Report is -- at least on my computer -- a disaster.

There are so many comments to make I shall not be ready to post the Excel file until Sunday.

Khalid

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 12 Sep 2008

Gee,

I was just seconds away from clicking ''Next'' on ''Welcome to the MultiCharts 4.0.1573.202'' Setup Wizard''.

Khalid, I'll wait for your report.

Regards

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 14 Sep 2008

In deference to certain wishes I shall not post my comments on the Version 4.0 Beta 2 (Build 1573) Back-Testing Strategy Performance Report here.

Instead, I shall send these directly to TS Support.

To their credit, TS Support team has at least the courage to listen to criticism and do whatever they can to remedy matters.

Thank God for that.

Khalid

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 14 Sep 2008

In deference to certain wishes I shall not post my comments on the Version 4.0 Beta 2 (Build 1573) Back-Testing Strategy Performance Report here.

Instead, I shall send these directly to TS Support.

To their credit, TS Support team has at least the courage to listen to criticism and do whatever they can to remedy matters.

Thank God for that.

Khalid
Khalid,

Noone has criticised you in this thread. This thread is run in good faith. PLEASE post your report on the backtester here. Making the report publicly open may result in other users of this forum become more active and your findings lead to other findings that will contribute to an improvement of MC. Some of us sent lots of e-mails to TSS, which proved non-effective. This thread is meant for such reports.

Regards

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 15 Sep 2008

The sole purpose of posting problems on this forum is to share them, get feedback, and thus facilitate product improvement.

I do not mind criticism, I can handle being proven wrong; this forum has a number of my posts admitting fault.

But I have no desire to get entangled with the self-appointed forum police.

Here onwards, I would rather talk to TS Support one to one.

I am sure they will listen.

Khalid

khalaad
Posts: 323
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 64 times
Been thanked: 57 times

Postby khalaad » 18 Sep 2008

Tresor,
Some of us sent lots of e-mails to TSS, which proved non-effective.
There is reason to hopeful your very valuable contributions here shall become effective sooner rather than later.

Khalid

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 25 Sep 2008

Version 4.0 Beta 2 (Build 1573) of MC miscalculates the slippage and commission in backtesting. The amount you enter is not the amount deducted from the trade in the historical results. (Yes; I took into account "per trade" vs "per share/contract" and yes, I took into account that the amount assigned in the Signal Properties is per side.)
Hi oakshadows,

There was a bug related to the slippage and commission size being multiplied by the BigPointValue. The bug has been fixed in the 4.0 beta 3 version of MultiCharts. I would recommend upgrading and trying beta 3. If the discrepancies are still there, we would need the following information for analysis:

1. your workspace
2. your strategy
3. strategy properties
4. performance report
5. clarifications on what values and how are different from what the expected ones

Thank you.

Regards.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 02 Oct 2008

Hello Marina,

I noticed that optimization reports differ when one runs optimization through Portfolio Backtester and through MC. The optimization report generated through MC has more features like profit factor, win / loss ratio, return on account, etc.

On the other hand, the optimization run through MC lacks period settings.

Would it be possible to have both reports the same?


Regards

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 03 Oct 2008

Hello Tresor,
I noticed that optimization reports differ when one runs optimization through Portfolio Backtester and through MC. The optimization report generated through MC has more features like profit factor, win / loss ratio, return on account, etc.
Portfolio Optimization Report lacks those parameters that are based on individual trades. The reason for that is that so far we have not come to a conclusion what exactly should be considered a trade within portfolio. Once it's done those extra parameters will be added.

As for the period analysis, I am not quite sure I understand what exactly you mean. Maybe you are talking about the backtesting report?

Thank you.

Best regards.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 04 Oct 2008


As for the period analysis, I am not quite sure I understand what exactly you mean. Maybe you are talking about the backtesting report?

.
I mean that in Portfolio Backtester one can determine periods for whch one wants to optimize / backtest a strategy , e.g. optimize from 1st January 2007 till now or optimize for last N bars, while you cannot do so when optimizing via MC.

Regards

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 04 Oct 2008

Marina,

In the latest beta version the BPV (which is set in QuoteManager) is perfectly reflected in the Backtester's calculations, just the way it should be.

Were you able, in the newest beta, to incorporate also Margin (also set via QouteManager) in the Backtester's calculations?

Regards

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 09 Oct 2008


I mean that in Portfolio Backtester one can determine periods for whch one wants to optimize / backtest a strategy , e.g. optimize from 1st January 2007 till now or optimize for last N bars, while you cannot do so when optimizing via MC.
Hi Tresor,

What you are describing is the same thing as requesting a specified number of bars or the period over which the charts will be plotted. This is the data on which MC runs optimization.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 09 Oct 2008

Marina,

Were you able, in the newest beta, to incorporate also Margin (also set via QouteManager) in the Backtester's calculations?

Regards
Margins will be used in MultiCharts 4.1 beta (this is a beta following the upcoming 4.0 release).

Moreover, that version will also have Potential Loss as one of the strategy settings. The potential loss will be either set explicitly in the strategy settings or calculated from a script. The potential loss will be taken into consideration to estimate how many contracts can be bought.

Regards.

Tresor
Posts: 1104
Joined: 29 Mar 2008
Has thanked: 12 times
Been thanked: 53 times

Postby Tresor » 15 Oct 2008

Thanks Marina


Return to “MultiCharts”