Pair Trading in Backtesting Not Working as Documented  [SOLVED]

Questions about MultiCharts .NET and user contributed studies.
User avatar
orad
Posts: 121
Joined: 14 Nov 2012
Has thanked: 50 times
Been thanked: 20 times

Pair Trading in Backtesting Not Working as Documented

Postby orad » 11 Oct 2018

Hi,

On the wiki about Pair Trading in Backtesting it says that in Portfolio Trader, strategies run sequentially.
When you have GV on first series, you should be able to get those GV when script is calculated on next data series. The calculation of the script is sequential. It means that the script is first calculated on bar 1 of data A and it generates a sell order (as an example) and passes a value through global variable. Then the script is calculated on bar 1 of data B where it reads the global variable and generates identical buy order. Though everything happens sequentially, in fact the sell and buy orders are generated on the bars with the same timestamp (if data A and data B have identical series).
After a lot of tries, I could not get it working and there is always a time difference between bars calculating across the strategies in the portfolio. You can try it with the following code.

Code: Select all

namespace PowerLanguage.Strategy
{
public class ParallelSignal : SignalObject
{
public ParallelSignal(object ctx) : base(ctx) { }

protected override void StartCalc()
{
Output.WriteLine(Bars.Info.Name + " started");
}

protected override void CalcBar()
{
Output.WriteLine(Bars.Info.Name + " calculated bar number " + Bars.CurrentBar);
}

}
}
Save the ParallelSignal and compile it, then in Portfolio Trader setup three Strategies that look like below under the Portfolio Tree:
  • Strategy 1
    • Instruments
      • symbol A
    • Signals
      • ParallelSignal
  • Strategy 2
    • Instruments
      • symbol B
    • Signals
      • ParallelSignal
  • Strategy 3
    • Instruments
      • symbol C
    • Signals
      • ParallelSignal
The output will look like this:

Code: Select all

SYMBOL_A started
SYMBOL_B started
SYMBOL_C started
SYMBOL_A calculated bar number 1
SYMBOL_A calculated bar number 2
SYMBOL_B calculated bar number 1
SYMBOL_A calculated bar number 3
SYMBOL_B calculated bar number 2
SYMBOL_A calculated bar number 4
SYMBOL_B calculated bar number 3
SYMBOL_A calculated bar number 5
SYMBOL_B calculated bar number 4
SYMBOL_A calculated bar number 6
SYMBOL_B calculated bar number 5
SYMBOL_A calculated bar number 7
SYMBOL_B calculated bar number 6
SYMBOL_A calculated bar number 8
SYMBOL_B calculated bar number 7
SYMBOL_A calculated bar number 9
SYMBOL_A calculated bar number 10
SYMBOL_B calculated bar number 8
SYMBOL_A calculated bar number 11
SYMBOL_B calculated bar number 9
SYMBOL_A calculated bar number 12
SYMBOL_B calculated bar number 10
SYMBOL_A calculated bar number 13
SYMBOL_B calculated bar number 11
SYMBOL_A calculated bar number 14
SYMBOL_B calculated bar number 12
SYMBOL_A calculated bar number 15
SYMBOL_B calculated bar number 13
SYMBOL_A calculated bar number 16
SYMBOL_B calculated bar number 14
SYMBOL_C calculated bar number 1
SYMBOL_A calculated bar number 17
SYMBOL_B calculated bar number 15
SYMBOL_C calculated bar number 2
SYMBOL_A calculated bar number 18
SYMBOL_A calculated bar number 19
SYMBOL_B calculated bar number 16
SYMBOL_C calculated bar number 3
SYMBOL_A calculated bar number 20
SYMBOL_B calculated bar number 17
SYMBOL_C calculated bar number 4
SYMBOL_A calculated bar number 21
SYMBOL_B calculated bar number 18
SYMBOL_C calculated bar number 5
SYMBOL_A calculated bar number 22
SYMBOL_B calculated bar number 19
SYMBOL_C calculated bar number 6
SYMBOL_A calculated bar number 23
SYMBOL_B calculated bar number 20
SYMBOL_C calculated bar number 7
SYMBOL_A calculated bar number 24
SYMBOL_B calculated bar number 21
SYMBOL_A calculated bar number 25
SYMBOL_B calculated bar number 22
SYMBOL_C calculated bar number 8
SYMBOL_A calculated bar number 26
SYMBOL_B calculated bar number 23
SYMBOL_C calculated bar number 9
SYMBOL_A calculated bar number 27
SYMBOL_B calculated bar number 24
SYMBOL_C calculated bar number 10
SYMBOL_A calculated bar number 28
SYMBOL_B calculated bar number 25
SYMBOL_C calculated bar number 11
SYMBOL_A calculated bar number 29
SYMBOL_B calculated bar number 26
SYMBOL_A calculated bar number 30
SYMBOL_B calculated bar number 27
SYMBOL_C calculated bar number 12
SYMBOL_A calculated bar number 31
SYMBOL_B calculated bar number 28
SYMBOL_C calculated bar number 13
SYMBOL_A calculated bar number 32
SYMBOL_B calculated bar number 29
SYMBOL_C calculated bar number 14
SYMBOL_A calculated bar number 33
SYMBOL_B calculated bar number 30
SYMBOL_C calculated bar number 15
SYMBOL_A calculated bar number 34
SYMBOL_B calculated bar number 31
SYMBOL_C calculated bar number 16
SYMBOL_A calculated bar number 35
SYMBOL_B calculated bar number 32
SYMBOL_C calculated bar number 17
SYMBOL_A calculated bar number 36
SYMBOL_B calculated bar number 33
SYMBOL_A calculated bar number 37
SYMBOL_B calculated bar number 34
SYMBOL_C calculated bar number 18
SYMBOL_A calculated bar number 38
SYMBOL_B calculated bar number 35
SYMBOL_C calculated bar number 19
SYMBOL_A calculated bar number 39
SYMBOL_B calculated bar number 36
SYMBOL_C calculated bar number 20
SYMBOL_A calculated bar number 40
SYMBOL_B calculated bar number 37
SYMBOL_A calculated bar number 41
SYMBOL_B calculated bar number 38
SYMBOL_C calculated bar number 21
SYMBOL_A calculated bar number 42
SYMBOL_B calculated bar number 39
SYMBOL_C calculated bar number 22
SYMBOL_A calculated bar number 43
SYMBOL_B calculated bar number 40
SYMBOL_A calculated bar number 44
SYMBOL_B calculated bar number 41
SYMBOL_A calculated bar number 45
SYMBOL_B calculated bar number 42
SYMBOL_C calculated bar number 23
SYMBOL_A calculated bar number 46
SYMBOL_B calculated bar number 43
SYMBOL_C calculated bar number 24
SYMBOL_A calculated bar number 47
SYMBOL_B calculated bar number 44
SYMBOL_C calculated bar number 25
SYMBOL_A calculated bar number 48
SYMBOL_B calculated bar number 45
SYMBOL_A calculated bar number 49
SYMBOL_B calculated bar number 46
SYMBOL_A calculated bar number 50
SYMBOL_B calculated bar number 47
SYMBOL_C calculated bar number 26
SYMBOL_A calculated bar number 51
SYMBOL_B calculated bar number 48
SYMBOL_C calculated bar number 27
SYMBOL_A calculated bar number 52
SYMBOL_B calculated bar number 49
SYMBOL_C calculated bar number 28
SYMBOL_A calculated bar number 53
SYMBOL_B calculated bar number 50
SYMBOL_A calculated bar number 54
SYMBOL_B calculated bar number 51
SYMBOL_C calculated bar number 29
SYMBOL_A calculated bar number 55
SYMBOL_B calculated bar number 52
SYMBOL_A calculated bar number 56
SYMBOL_B calculated bar number 53
SYMBOL_C calculated bar number 30
SYMBOL_A calculated bar number 57
SYMBOL_B calculated bar number 54
SYMBOL_A calculated bar number 58
SYMBOL_B calculated bar number 55
SYMBOL_A calculated bar number 59
SYMBOL_C calculated bar number 31
SYMBOL_A calculated bar number 60
That is clearly not running in sequence. I really hope that this is some mistake on my part and not a limitation of the platform. :roll:

How can I make it run as documented? Please help.

Thanks!
Last edited by orad on 11 Oct 2018, edited 1 time in total.

User avatar
orad
Posts: 121
Joined: 14 Nov 2012
Has thanked: 50 times
Been thanked: 20 times

Re: Pair Trading in Backtesting Not Working as Documented

Postby orad » 11 Oct 2018

Update 1:

I noticed when adding multiple signals under the same strategy, they indeed run sequentially. But all of them have access only to the same instrument on Data1 and it cannot be used for pair trading.

Update 2:

I tried adding the thread number to the logs with Thread.CurrentThread.ManagedThreadId and it shows all the signals are running on the same thread. I was assuming that in Portfolio Trader the strategies run on separate threads. If that was the case, then I could probably make a signal wait by sleeping the thread until a "MasterSignal" tells it to continue to the next bar. But if all signals on all the strategies run on the same thread, sleeping the thread would pause everything. I'm still wondering how to make this work. Please help!

User avatar
Anna MultiCharts
Posts: 560
Joined: 14 Jul 2017
Has thanked: 42 times
Been thanked: 140 times

Re: Pair Trading in Backtesting Not Working as Documented

Postby Anna MultiCharts » 12 Oct 2018

Hello, orad!

We tested your case on 3 symbols (all 1 minute resolution), and received the following result:

Code: Select all

AUD/JPY started
EUR/USD started
CHF/JPY started
AUD/JPY calculated bar number 1
EUR/USD calculated bar number 1
CHF/JPY calculated bar number 1
AUD/JPY calculated bar number 2
EUR/USD calculated bar number 2
CHF/JPY calculated bar number 2
AUD/JPY calculated bar number 3
EUR/USD calculated bar number 3
CHF/JPY calculated bar number 3
AUD/JPY calculated bar number 4
EUR/USD calculated bar number 4
CHF/JPY calculated bar number 4
The backtest is done as described in our Wiki.

Please provide your strategy and overall portfolio settings as screenshots and specify if you’re running back/forward testing or live trading (in this case please also attach Strategy Properties -> Autotrading settings) in portfolio.

User avatar
orad
Posts: 121
Joined: 14 Nov 2012
Has thanked: 50 times
Been thanked: 20 times

Re: Pair Trading in Backtesting Not Working as Documented

Postby orad » 12 Oct 2018

Hi Anna,

Thanks for looking into this. I tried it again on another installation of MultiCharts.NET on another computer and at first glance, all the 3 signals were running sequentially like what you got. But when I cloned the strategy a few more times (total of 8 strategies) then it started to show the race condition. I think when the processor is fast enough and data feeds are not too heavy the strategies happen to run in sequence, but that is not deterministic. On another run you may have one or more signals go out of sync with the others.

For testing this, I tried 8 stock instruments. I used this slightly modified Signal code that also shows the thread number:

Code: Select all

using System.Threading;

namespace PowerLanguage.Strategy
{
public class ParallelSignal : SignalObject
{
public ParallelSignal(object ctx) : base(ctx) { }

protected override void StartCalc()
{
Output.WriteLine("{0} started on thread {1}",
Bars.Info.Name, Thread.CurrentThread.ManagedThreadId);
}

protected override void CalcBar()
{
Output.WriteLine("{0} running on thread {1} calculated bar number {2}",
Bars.Info.Name, Thread.CurrentThread.ManagedThreadId, Bars.CurrentBar);
}

}
}
Output:

Code: Select all

AAPL started on thread 13
MSFT started on thread 13
AMZN started on thread 13
FB started on thread 13
GOOG started on thread 13
INTC started on thread 13
AMD started on thread 13
NVDA started on thread 13
AAPL running on thread 13 calculated bar number 1
MSFT running on thread 13 calculated bar number 1
AMZN running on thread 13 calculated bar number 1
GOOG running on thread 13 calculated bar number 1
INTC running on thread 13 calculated bar number 1
AMD running on thread 13 calculated bar number 1
NVDA running on thread 13 calculated bar number 1
AAPL running on thread 13 calculated bar number 2
MSFT running on thread 13 calculated bar number 2
AMZN running on thread 13 calculated bar number 2
GOOG running on thread 13 calculated bar number 2
INTC running on thread 13 calculated bar number 2
AMD running on thread 13 calculated bar number 2
NVDA running on thread 13 calculated bar number 2
.
.
.
AAPL running on thread 13 calculated bar number 388
MSFT running on thread 13 calculated bar number 388
AMZN running on thread 13 calculated bar number 388
GOOG running on thread 13 calculated bar number 388
INTC running on thread 13 calculated bar number 388
AMD running on thread 13 calculated bar number 388
NVDA running on thread 13 calculated bar number 388
AAPL running on thread 13 calculated bar number 389
MSFT running on thread 13 calculated bar number 389
AMZN running on thread 13 calculated bar number 389
FB running on thread 13 calculated bar number 1
GOOG running on thread 13 calculated bar number 389
INTC running on thread 13 calculated bar number 389
AMD running on thread 13 calculated bar number 389
NVDA running on thread 13 calculated bar number 389
AAPL running on thread 13 calculated bar number 390
MSFT running on thread 13 calculated bar number 390
AMZN running on thread 13 calculated bar number 390
FB running on thread 13 calculated bar number 2
GOOG running on thread 13 calculated bar number 390
INTC running on thread 13 calculated bar number 390
AMD running on thread 13 calculated bar number 390
NVDA running on thread 13 calculated bar number 390
Notice in the logs that FB calculates bar 1 when other symbols are on bar 389. All signals running on the same thread.

In my previous test, I used a group of BITFINEX symbols in 1-minute resolution. If you add to the number of strategies at a higher resolution of the instruments, I think you will be able to reproduce the issue. I'm trying this only for backtesting.

Thanks!
Attachments
ParallelSignals.png
(82 KiB) Downloaded 647 times

User avatar
orad
Posts: 121
Joined: 14 Nov 2012
Has thanked: 50 times
Been thanked: 20 times

Re: Pair Trading in Backtesting Not Working as Documented

Postby orad » 14 Oct 2018

Notice in the logs that FB calculates bar 1 when other symbols are on bar 389. All signals running on the same thread.
There is another big problem related to this: When the strategy with Data1=FB calculates bar 1, Bars.CloseValue on that strategy returns close value at bar 1 as expected, but getting close value on other data streams (BarsOfData(2).CloseValue, BarsOfData(3).CloseValue, etc) returns close value of those data streams at bar 389! This makes pair trading totally unreliable on MC :cry:

User avatar
orad
Posts: 121
Joined: 14 Nov 2012
Has thanked: 50 times
Been thanked: 20 times

Re: Pair Trading in Backtesting Not Working as Documented  [SOLVED]

Postby orad » 18 Oct 2018

Solution: I had a support session with MultiCharts and it looks like the problem was due to incomplete data, and that I was trying backtesting in offline mode for the data to load faster, so it didn't try to load complete data.

We added the charts for the two instruments in one window and plotted Bars.CurrentBar on them to see where bar numbers went out of sync.

Quoting from Anna:
The calculation of the strategy gets out of synch because of missing bars on each of the data series. So, for example, when there's bar 90 on BTC, there's still bar 31 on ETH. The data is incomplete, therefore the calculations are not sequential

If on the main data series bars are present, and on the additional data series it's missing, strategy will be operating with the current bar of the main data series

If you see such inconsistency further, please make sure that the data series used are not missing bars as this is the reason for the behavior you're facing.

You can compare them as we did on the charts and see the completeness of the data there
Thanks MC for the great support!


Return to “MultiCharts .NET”