problem with data2

Questions about MultiCharts and user contributed studies.
Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

problem with data2

Postby Fabrice Daniel » 18 May 2009

Hi,

There is a problem with some indicators calculation from data2.

I initially had a strange behaviour with a system, so I wrote a very basic system printing the indicator value based on data2.

Code: Select all

if Date=1090416 then
print (Time, " ", close of data2:0:5, " ",AverageFC (close of data2,100):0:5);
Simply open a chart on EURUSD with data1 = 1Hour chart and data2=Daily chart.

Apply this system to the main chart (1Hour EURUSD).

The resulting log is the following :

0.00 1.32117 9.20004
100.00 1.32117 9.20031
200.00 1.32117 9.20057
300.00 1.32117 9.20083
400.00 1.32117 9.20110
500.00 1.32117 9.20136

So we get an average with a value > 9 and modified on each 1 Hour bar.

When I replace AverageFC by XAverage :

0.00 1.32117 1.32737
100.00 1.32117 1.32725
200.00 1.32117 1.32713
300.00 1.32117 1.32701
400.00 1.32117 1.32690
500.00 1.32117 1.32678
600.00 1.32117 1.32667

The theorical value on my chart is 1.32472. These values can slighly differ depending on your data source but here the result is wrong compared to the same indicator added to the Daily chart and the data is modified on each 1 hour bar.

And finaly when I replace XAverage by Average:

0.00 1.32117 1.31707
100.00 1.32117 1.31707
200.00 1.32117 1.31707
300.00 1.32117 1.31707
400.00 1.32117 1.31707
500.00 1.32117 1.31707
600.00 1.32117 1.31707
700.00 1.32117 1.31707

The Average value is exactly the same as in the chart and doesn't change on every 1 hour bar. It's exactly as expected.

This explain why my system using AverageFC never took long trades after 3 months.

Of course I compared the same results with TS 8.6. In TS everythying works as expected (AverageFC and XAverage) with the good MA value not changing on every 1hour bar.

here is an example of AverageFC on TS 8.6:

0.00 1.32260 1.31720
100.00 1.32260 1.31720
200.00 1.32260 1.31720
300.00 1.32260 1.31720
400.00 1.32260 1.31720
500.00 1.32260 1.31720
600.00 1.32260 1.31720

This is a very important blocking issue for all of us using trend confirmation or filters based on different time frames.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 18 May 2009

the print statement can be finicky if it is required to call a function.

Try convert all the variables to text before inserting them into the print statement.

e.g

var1 = text(Time);
var2 = text(close of data2:0:5);
var3 = text(AverageFC (close of data2,100):0:5);

print(var1, " ", var2, " ", var3);

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 19 May 2009

The same code works fine in TS and I did it only because one of my systems stops to take long positions after the first 3 months on a 5 years backtest. So I don't think this is a "print" issue.

This system uses the position of close compared to the averageFC applied on data 2 (close above or below AverageFC) to take long only or short only signals. The AverageFC value I found after printing perfectly explain this situation.

However I will test your suggestion but even if this suggest using a temporary variable can make it works, it's only a workaround, not a bug fix.

And in any case even if a workaround exist, many existing and protected EL scripts, including vendors trading systems, could fail and you will not be able to apply any workaround on it.

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 20 May 2009

Ok so using temporary variables doesn't work (of course).

I think I found an explanation but I have to verify it.

Everybody know that AverageFC save CPU not using a summation loop on every bar but adding the most recent bar value and simultaneously substracting the oldest one to the current average (only the initial calculation use a loop).

Code: Select all

var0 = var0[1] + PriceValue - PriceValue[Len] ;
Many indicators use the same kind of calculation: e.g: Exponential Moving average.

The results I got shows that the "close of data2" is added or substracted on every 1hour bar. This could explain why the value based on Close Of Data2 change on every 1Hour bar for XAverage or AverageFC and not for Average which compute the full summation on every 1hour bar (so hiding the problem).

It looks like a common "intrabarpersist" problem not handled by Power Language (it's not exactly the same problem but we get the same kind of results). Here data1 is the "intrabar" for data2 and the intrabarpersist doesn't concern a variable but a function call. Power Language must call any function based on data2 only 1 time per data2 modification or find a way to not allow an indicator based on data2 to be modified 24 times per day if data2 didn't change.

It's easy to know how TS handle this problem adding some logs on AverageFC and SummationFC functions (N calls per data2 with the same data on every call ? or One call per data2 occurence ?).

I will test it but a reply from TSSupport should be better if they already have the solution :wink: .

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 20 May 2009

i remember reading solution like this, don't remember the exact syntax, or don't know if this applies, but you can try:

AverageFC (close of data2,100) of data2;

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 20 May 2009

Thanks TJ, I will try it in the subway :wink: (yes MC works fine on a Netbook).

In any case it's a workaround, not the expected behaviour.

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 20 May 2009

OK after testing this workaround doesn't work.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 21 May 2009

[1] MultiCharts has different values for Average and AverageFC functions, because they have different algorithms:
Average uses Summation function which simply processes bars one by one and sums them up.
AverageFC uses SummationFC function which references its previous values. When used on non –basis data series it can show different values.
[2] In TS 8.6 if you create a user function and use AverageFC text you can see that its values will be different from Average. AverageFC is built and its algorithm is not equal to the text.
[3] We checked the following script (as suggested by TJ):
print (
time,
" AvgFC=",AverageFC (c of data2,Length):0:10 ,
" AvgFC of data2=",AverageFC (c,Length)of data2 :0:10,
" Avg=",Average (c of data2,Length):0:10 );

And we have this output:
--------------------------------------
1000.00 AvgFC=0.9629400000 AvgFC of data2=1.3527450000 Avg=1.3527450000
1100.00 AvgFC=0.9590350000 AvgFC of data2=1.3527450000 Avg=1.3527450000
1200.00 AvgFC=0.9551300000 AvgFC of data2=1.3527450000 Avg=1.3527450000
1300.00 AvgFC=0.9512250000 AvgFC of data2=1.3527450000 Avg=1.3527450000
1400.00 AvgFC=0.9473200000 AvgFC of data2=1.3527450000 Avg=1.3527450000
--------------------------------------
AverageFC (c,Length)of data2
matches
Average (c of data2,Length)

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 21 May 2009

* Sorry, I was writing my message when you replied - here is what I have found.


I have create copy of AverageFC and SummationFC to put my own logs into TS and MC.

This is the log code into my own FXSummationFC.

Code: Select all

Sum = Sum[1] + Price - Price[Length] ;
Print (Date," ",Time," In SummationFC Price : ",Price:0:5, " Sum: ", Sum:0:5, " Sum[1]: ", Sum[1]:0:5, " Price [Lenght] : ", Price[Length]:0:5);
Here is the result for TS on the 20/05/2009:

1090505.00 2259.00 In SummationFC Price : 1.33291 Sum: 13.20733 Sum[1]: 13.16884 Price [Lenght] : 1.29442
1090506.00 2259.00 In SummationFC Price : 1.33315 Sum: 13.24000 Sum[1]: 13.20733 Price [Lenght] : 1.30048
1090507.00 2259.00 In SummationFC Price : 1.33882 Sum: 13.26441 Sum[1]: 13.24000 Price [Lenght] : 1.31441
1090508.00 2259.00 In SummationFC Price : 1.36389 Sum: 13.30425 Sum[1]: 13.26441 Price [Lenght] : 1.32405
1090511.00 2259.00 In SummationFC Price : 1.35806 Sum: 13.35883 Sum[1]: 13.30425 Price [Lenght] : 1.30348
1090512.00 2259.00 In SummationFC Price : 1.36489 Sum: 13.40891 Sum[1]: 13.35883 Price [Lenght] : 1.31481
1090513.00 2259.00 In SummationFC Price : 1.35993 Sum: 13.44172 Sum[1]: 13.40891 Price [Lenght] : 1.32712
1090514.00 2259.00 In SummationFC Price : 1.36390 Sum: 13.48288 Sum[1]: 13.44172 Price [Lenght] : 1.32274
1090515.00 2259.00 In SummationFC Price : 1.34940 Sum: 13.50548 Sum[1]: 13.48288 Price [Lenght] : 1.32680
1090518.00 2259.00 In SummationFC Price : 1.35609 Sum: 13.52104 Sum[1]: 13.50548 Price [Lenght] : 1.34053
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
0.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
100.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
200.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
300.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
400.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291
500.00 1.36308 1.35512
1090519.00 2259.00 In SummationFC Price : 1.36308 Sum: 13.55121 Sum[1]: 13.52104 Price [Lenght] : 1.33291

-------------------------------------------------------------------------------

And here is the same result for MC (small variations on the Price is only due to the ref hour for daily change - 22h59 into TS and 00:00 into MC, it's not significant).

1090519.00 1500.00 In SummationFC Price : 1.35557 Sum: 95.93048 Sum[1]: 95.91650 Price [Lenght] : 1.34159
1090519.00 1600.00 In SummationFC Price : 1.35557 Sum: 95.94446 Sum[1]: 95.93048 Price [Lenght] : 1.34159
1090519.00 1700.00 In SummationFC Price : 1.35557 Sum: 95.95844 Sum[1]: 95.94446 Price [Lenght] : 1.34159
1090519.00 1800.00 In SummationFC Price : 1.35557 Sum: 95.97242 Sum[1]: 95.95844 Price [Lenght] : 1.34159
1090519.00 1900.00 In SummationFC Price : 1.35557 Sum: 95.98640 Sum[1]: 95.97242 Price [Lenght] : 1.34159
1090519.00 2000.00 In SummationFC Price : 1.35557 Sum: 96.00038 Sum[1]: 95.98640 Price [Lenght] : 1.34159
1090519.00 2100.00 In SummationFC Price : 1.35557 Sum: 96.01436 Sum[1]: 96.00038 Price [Lenght] : 1.34159
1090519.00 2200.00 In SummationFC Price : 1.35557 Sum: 96.02834 Sum[1]: 96.01436 Price [Lenght] : 1.34159
1090519.00 2300.00 In SummationFC Price : 1.35557 Sum: 96.04232 Sum[1]: 96.02834 Price [Lenght] : 1.34159
1090520.00 0.00 In SummationFC Price : 1.36273 Sum: 96.07398 Sum[1]: 96.04232 Price [Lenght] : 1.33107
0.00 1.36273 9.60740
1090520.00 100.00 In SummationFC Price : 1.36273 Sum: 96.10564 Sum[1]: 96.07398 Price [Lenght] : 1.33107
100.00 1.36273 9.61056
1090520.00 200.00 In SummationFC Price : 1.36273 Sum: 96.13730 Sum[1]: 96.10564 Price [Lenght] : 1.33107
200.00 1.36273 9.61373
1090520.00 300.00 In SummationFC Price : 1.36273 Sum: 96.16896 Sum[1]: 96.13730 Price [Lenght] : 1.33107
300.00 1.36273 9.61690
1090520.00 400.00 In SummationFC Price : 1.36273 Sum: 96.20062 Sum[1]: 96.16896 Price [Lenght] : 1.33107
400.00 1.36273 9.62006
1090520.00 500.00 In SummationFC Price : 1.36273 Sum: 96.23228 Sum[1]: 96.20062 Price [Lenght] : 1.33107
500.00 1.36273 9.62323
1090520.00 600.00 In SummationFC Price : 1.36273 Sum: 96.26394 Sum[1]: 96.23228 Price [Lenght] : 1.33107


---------------------------------------------------------------

So the results shows that the difference is on how Sum [1] reference is managed:

In TS, Sum [1] uses the data of the previous data2 bar, not of the previous data1 bar. So even if TS compute SummationFC on each data1 bar, he do it on the same setof data, so he always get the same result on each 1 hour bar.

In MC, Sum [1] uses thedata of the previous data1 bar (1 hour), so Sum is modified on every data1 bar.

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 21 May 2009

Thank you Andrew, I have only tested the suggestion of TJ :

Code: Select all

AverageFC (close of data2,100) of data2;
but not:

Code: Select all

AverageFC (close,100) of data2;
I do it now and keeps you informed in a few minutes.

In any case, the problem remain for protected existing EL from vendors.

Two solutions:

1. You can make it fully TS compatible

2. You keep the current behaviour (if there is no other hidden impact) and write a special note on the help (into Data dictionary entry). So the vendor and your customer will be able to adapt their codes. If only the syntax is slightly different but eveything works fine, I think it will not be a problem.

In any case if there is a way to make it work, no problem. It's only a question of documentation.

About the doc I suggest you think about a special chapter : "EL and PL" with a table summarizing up what is different and usages samples EL and equivalent PL.

It could be very helpful and time saving.

Fabrice
http://www.addictfx.com

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 21 May 2009

Ok Andrew, It works, very good news for my systems :D

I have tested it on AverageFC, XAverage and Hull Moving Average.

Thank you very much.

User avatar
Andrew Kirillov
Posts: 1589
Joined: 28 Jul 2005
Has thanked: 2 times
Been thanked: 31 times
Contact:

Postby Andrew Kirillov » 22 May 2009

You are welcome.


Return to “MultiCharts”