Portfolio Backtester with different backtest result

Questions about MultiCharts and user contributed studies.
henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Portfolio Backtester with different backtest result

Postby henrywl » 19 Aug 2012

I have encounter problem with the backtester with the same setting, and same strategy but different result from one day to the other. I am using the IB's data feed Historical data, and even I saved the portfolio and open it from other computer on the other day, some results are the same and some are different. Is that normal??

User avatar
StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Portfolio Backtester with different backtest result

Postby StefanoSF » 20 Aug 2012

I'm experiencing the same exact thing, with some results being 50% off the previous results.

evdl
Posts: 396
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 84 times
Been thanked: 122 times

Re: Portfolio Backtester with different backtest result

Postby evdl » 20 Aug 2012

If you save the portfolio and open it another day, the data period is also possibly changed to the current day. Instead of the period you used before. That can make a difference.

And if you open it on another computer, you have to make sure that the historical data is the same on all computers. With other data you will get other results.

henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Re: Portfolio Backtester with different backtest result

Postby henrywl » 20 Aug 2012

Well I am using Historical future data instead of current future data. so all the data received from IB should be the same. But result are still different.. (very different)...

Since I assume the calculation are based on close prices.. so they indicator calculation should be same all the time unless the data received from IB change from time to time for the historical data.

evdl
Posts: 396
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 84 times
Been thanked: 122 times

Re: Portfolio Backtester with different backtest result

Postby evdl » 20 Aug 2012

There can be a difference in historical data of IB that you collected yourself during realtime (in the quotemanager) and the historical data you get for example if you reload a chart (snapshot of realtime data, depending on your resolution settings). But to be sure you have the same data you can do a reload of the chart with the period you want. Then it will get the historical data from IB again.

And with the data period I mean, that if you select "from 14-08-2012 to 20-08-2012" today to backtest and you save this. The next day if you open it, it will have the data range of 14-08-2012 to 21-08-2012. (see picture)

In the instrument settings you can select "trade", "ask" and "bid". If you selected trade you will get the close depending on your resolution.
Attachments
portfolio.png
portfolio.png (65.35 KiB) Viewed 459 times
These users thanked the author evdl for the post:
JoshM

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 557 times

Re: Portfolio Backtester with different backtest result

Postby Andrew MultiCharts » 20 Aug 2012

Portfolio Backtester should show the same results as MultiCharts if:
    1. Data is the same (data source, resolution, data range, quote field and sessions).
    2. Signals are the same.
    3. Strategy Properties are the same (right click on "Strategy 1" -> Show Properties...).
    4. Set "Max Potential Loss" to 100% in Potfolio Settings.
    5. Set Initial Portfolio Capital to something big, like 1000000000 (Initial Capital is not taken into consideration in MultiCharts but is taken in Portfolio Backtester)

henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Re: Portfolio Backtester with different backtest result

Postby henrywl » 20 Aug 2012

Thanks, well actually all the things mention above have been repeatedly set everytime I try to setup the backtester. But somehow the result are still different between computers. I try it back with the original saved portfolio computer and it return the same result.

That keeps me wondering the test result are being correct or not, since with the same strategy and same autotrade setting return the same result no matter where you deploy it as long as with MC.

The test result sometimes varies from a +ve return to a -ve lose.

Thx.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 557 times

Re: Portfolio Backtester with different backtest result

Postby Andrew MultiCharts » 20 Aug 2012

Something in settings must be definitely wrong. If you can't figure it out, you can come to our live chat to ask our operators to help you with this. Remote access to your computer will be required.

henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Re: Portfolio Backtester with different backtest result

Postby henrywl » 20 Aug 2012

Actually, I just come across there's might be a problem, my computer accidently shutdown instantly, and I reload my saved portfolio I just did on Aug 20th, with the From date let say 29/5/2012 to 29/5/2012 on May's Future and it works prefectly fine. But I now reload the portfolio on Aug 21st with the same setting, there's an error complaining "From Date cannot be later than the To Date" Dialogue.

Is this causing the error of loading data from different time zone?? Thx

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 557 times

Re: Portfolio Backtester with different backtest result

Postby Andrew MultiCharts » 20 Aug 2012

Obviously when your computer shut down the "from" date became later than your "to" date, that makes no sense. The dates should be reset manually.

henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Re: Portfolio Backtester with different backtest result

Postby henrywl » 20 Aug 2012

Well, But I am using the May's future with the From 29/5/2012 to 29/5/2012 from IB. And this setting is prefectly fine few hrs ago. Now I try to run the optimizer or the backtest with the same setting, it complains the "From Date cannot be later than the To Date" Error

henrywl
Posts: 26
Joined: 28 Jun 2012
Has thanked: 2 times
Been thanked: 1 time

Re: Portfolio Backtester with different backtest result

Postby henrywl » 20 Aug 2012

henrywl wrote:Well, But I am using the May's future with the From 29/5/2012 to 29/5/2012 from IB. And this setting is prefectly fine few hrs ago. Now I try to run the optimizer or the backtest with the same setting, it complains the "From Date cannot be later than the To Date" Error


And if I reopen a new portfolio with the same settings, for the From 29/5/2012 to 29/5/2012. Then it works without the complains "From Date cannot be later than the To Date"

Many Thx.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 557 times

Re: Portfolio Backtester with different backtest result

Postby Andrew MultiCharts » 21 Aug 2012

Could you provide exact steps to reproduce the situation so i could descuss it with engineers?


Return to “MultiCharts”