Issues with average function under multiple data series  [SOLVED]

I have been checking on the mechanics of the array function and found some possible issues with the function when using multiple data series. I compare the results of simple average calculation using array function and the build-in average function. This is the code I used for testing on a 5-second chart with multiple data series:

input:printswitch(0);
var:array_average_1(0), function_average_1(0), array_average_2(0), function_average_2(0),array_average_3(0), function_average_3(0),
counter(0);
array:ma_1[10](0),ma_2[10](0),ma_3[10](0);


counter=counter+1;
ma_1[counter]=c of data1;
array_average_1=AverageArray(ma_1,10);
function_average_1=average(c,10);

ma_2[counter]=c of data4;
array_average_2=AverageArray(ma_2,10);
function_average_2=average(c of data4,10);

ma_3[counter]=c of data5;
array_average_3=AverageArray(ma_3,10);
function_average_3=average(c of data5,10);


if counter=10 then counter=0;

if printswitch=1 and LastBarOnChart_s and barstatus=2 then print(file("c:\testing.txt"), ". Time ",time_s, " c of data1 = ", c of data1, " array_average 1 = ", array_average_1, ". function_average 1 =", function_average_1,
" c of data4 = ", c of data4, " array_average 2 = ", array_average_2, ". function_average 2 =", function_average_2," c of data5 = ", c of data5, " array_average 3 = ", array_average_3, ". function_average 3 =", function_average_3, " counter = ", counter);

You will see from the attched report that the averages calculated on data1 using array function and average function are the same whereas the averages calculated on data4 and data5 are different. It happens under realtime and backtest enviornments. Since my system uses array function extentively, I wonder what causes such discrepancy?

I have been checking on the mechanics of the array function and found some possible issues with the function when using multiple data series. I compare the results of simple average calculation using array function and the build-in average function. This is the code I used for testing on a 5-second chart with multiple data series:
input:printswitch(0);
var:array_average_1(0), function_average_1(0), array_average_2(0), function_average_2(0),array_average_3(0), function_average_3(0),
counter(0);
array:ma_1[10](0),ma_2[10](0),ma_3[10](0);
counter=counter+1;
ma_1[counter]=c of data1;
array_average_1=AverageArray(ma_1,10);
function_average_1=average(c,10);
ma_2[counter]=c of data4;
array_average_2=AverageArray(ma_2,10);
function_average_2=average(c of data4,10);
ma_3[counter]=c of data5;
array_average_3=AverageArray(ma_3,10);
function_average_3=average(c of data5,10);
if counter=10 then counter=0;
if printswitch=1 and LastBarOnChart_s and barstatus=2 then print(file("c:\testing.txt"), ". Time ",time_s, " c of data1 = ", c of data1, " array_average 1 = ", array_average_1, ". function_average 1 =", function_average_1,
" c of data4 = ", c of data4, " array_average 2 = ", array_average_2, ". function_average 2 =", function_average_2," c of data5 = ", c of data5, " array_average 3 = ", array_average_3, ". function_average 3 =", function_average_3, " counter = ", counter);
You will see from the attched report that the averages calculated on data1 using array function and average function are the same whereas the averages calculated on data4 and data5 are different. It happens under realtime and backtest enviornments. Since my system uses array function extentively, I wonder what causes such discrepancy?

see post #5
viewtopic.php?f=16&t=6929

ps.
[FAQ] How to Post Codes (that people can read)
viewtopic.php?f=16&t=11713

Thanks for the advice. Revised the code but same issue happens.

Where are the errors? Can you pick a date and time as example?
What should the proper values be?
Have you done a manual calculation to determine the differences?
hint: check for empty periods.

Posting a massive txt output does not point to any errors.



You can use the print command to do an audit.

For each steps, print out the values in the array. Not the average, but the value in each cell of the array, so that you can use this to compare it to the value in the chart.
Then print out the values that makes up your average, and compare it to the array.

To make your test easier,

1. start your test with 2 data series only.
2. keep the average short. eg. 3 periods, so that you can use a calculator to manually check the results.


ps. on your print out, you don't have to make such long descriptive labels. keep the print out small, so that it is easy to read and trace the error. eg. instead of "function_average 2 =", simply use "ma2=".

I thought the error was quite obvious and abundant so I didn't go further. Anyway, from 9:42:50 to 9:45:05, you can see that the averages of data4 are different, same for data5. Whereas the averages of data1 during the same period are the same. I have already manually checked the calculation before posting and the averages based on the array function is correct based on the closing prices of the corresponding data series in the report.

. Time 94250.00 c of data1 = 16068.00 array_average 1 = 16069.60. function_average 1 =16069.60 c of data4 = 1804.00 array_average 2 = 1804.35. function_average 2 =1804.25 c of data5 = 3431.00 array_average 3 = 3431.47. function_average 3 =3431.32 counter = 6.00
. Time 94300.00 c of data1 = 16068.00 array_average 1 = 16069.40. function_average 1 =16069.40 c of data4 = 1804.00 array_average 2 = 1804.30. function_average 2 =1804.15 c of data5 = 3430.75 array_average 3 = 3431.38. function_average 3 =3431.15 counter = 7.00
. Time 94315.00 c of data1 = 16068.00 array_average 1 = 16069.10. function_average 1 =16069.10 c of data4 = 1804.00 array_average 2 = 1804.25. function_average 2 =1804.03 c of data5 = 3430.50 array_average 3 = 3431.25. function_average 3 =3430.85 counter = 8.00
. Time 94320.00 c of data1 = 16068.00 array_average 1 = 16068.80. function_average 1 =16068.80 c of data4 = 1804.00 array_average 2 = 1804.20. function_average 2 =1804.00 c of data5 = 3430.50 array_average 3 = 3431.13. function_average 3 =3430.78 counter = 9.00
. Time 94325.00 c of data1 = 16068.00 array_average 1 = 16068.60. function_average 1 =16068.60 c of data4 = 1804.00 array_average 2 = 1804.15. function_average 2 =1804.00 c of data5 = 3430.50 array_average 3 = 3431.00. function_average 3 =3430.75 counter = 0.00
. Time 94350.00 c of data1 = 16069.00 array_average 1 = 16068.50. function_average 1 =16068.50 c of data4 = 1804.00 array_average 2 = 1804.10. function_average 2 =1804.00 c of data5 = 3431.25 array_average 3 = 3430.95. function_average 3 =3430.60 counter = 1.00
. Time 94355.00 c of data1 = 16069.00 array_average 1 = 16068.40. function_average 1 =16068.40 c of data4 = 1804.00 array_average 2 = 1804.05. function_average 2 =1804.00 c of data5 = 3431.00 array_average 3 = 3430.88. function_average 3 =3430.63 counter = 2.00
. Time 94405.00 c of data1 = 16069.00 array_average 1 = 16068.40. function_average 1 =16068.40 c of data4 = 1804.00 array_average 2 = 1804.03. function_average 2 =1804.00 c of data5 = 3431.00 array_average 3 = 3430.85. function_average 3 =3430.72 counter = 3.00
. Time 94505.00 c of data1 = 16069.00 array_average 1 = 16068.40. function_average 1 =16068.40 c of data4 = 1804.00 array_average 2 = 1804.00. function_average 2 =1804.00 c of data5 = 3430.75 array_average 3 = 3430.80. function_average 3 =3430.63 counter = 4.00

I thought the error was quite obvious and abundant so I didn't go further. Anyway, from 9:42:50 to 9:45:05, you can see that the averages of data4 are different, same for data5. Whereas the averages of data1 during the same period are the same. I have already manually checked the calculation before posting and the averages based on the array function is correct based on the closing prices of the corresponding data series in the report.
::

Let's pick one data point for examination.
For your record of Time= 94250.00, have you determined the source of the error?
e.g.
Ok you have added the values in the array.
Have you manually added up the closing values, with data from the chart, for the previous 10 periods,
then divide it by 10 to come up with your "Reference" moving average value?
Is this value the same as the indicator calculated moving average value? (ie the function_average_2)

Thank you for showing us how to calculate averages. I believe the issue we are dealing here is the asynchronous secondary market data series. May be i should reiterate my findings based on the above code and the attached report :

1. Averages of data1 calculated using array function and average function are consistently identical, therefore the calculation of both functions are based on identical data value over the entire period.

2. Averages of secondary data series using array function and average function are occasionally different.

3. The functions in 2. above therefore used a slightly differently data values to arrive at the answer. It is safe to assume that the data values are different is due to the fact that they are sampled by the two functions at slightly different time.

4. Because of the 1. and 3. above, indicators or systems that use multiple data series and array/averages functions may be subject to asynchronous market data issue. Bear in mid that the reason to use array function in the first place is to handle a larger amount of data, the frequency of this issue may actually happen more frequent.

5. Looking under the hood, i believe the issue stems from the summation function and summationarray function, but why they affect only secondary data series is beyond me and is best answered by the MC development team.

Hello atsui,

We will check that.

The code logic was wrong. Here is the proper code:

Hi Henry,

Thanks for the revised code. The average function and array function now return the same average; however, it should be noted that the revised code also highlights the asynchronous secondary data issue when using average function on multiple data series. Below is the output based on the revised code.

5279.00 10.0016078.00 4(1804.25) 5(3470.25). 1 =16078.30 ( 16078.30). 4 =1804.43 ( 1804.43). 5 =3470.18 ( 3470.18)
5280.00 10.0016077.00 4(1804.00) 5(3469.75). 1 =16078.20 ( 16078.20). 4 =1804.30 ( 1804.30). 5 =3470.20 ( 3470.20)
5281.00 10.0016076.00 4(1804.00) 5(3469.75). 1 =16078.00 ( 16078.00). 4 =1804.25 ( 1804.25). 5 =3470.15 ( 3470.15)
5282.00 10.0016076.00 4(1804.00) 5(3469.50). 1 =16077.80 ( 16077.80). 4 =1804.20 ( 1804.20). 5 =3470.07 ( 3470.07)
5283.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16077.70 ( 16077.70). 4 =1804.18 ( 1804.18). 5 =3470.03 ( 3470.03)
5284.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16077.50 ( 16077.50). 4 =1804.18 ( 1804.18). 5 =3470.03 ( 3470.03)
5285.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16077.20 ( 16077.20). 4 =1804.18 ( 1804.18). 5 =3470.03 ( 3470.03)
5286.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16077.00 ( 16077.00). 4 =1804.25 ( 1804.25). 5 =3469.93 ( 3469.93)
5287.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16077.00 ( 16077.00). 4 =1804.25 ( 1804.25). 5 =3469.88 ( 3469.88)
5288.00 10.0016077.00 4(1804.25) 5(3469.75). 1 =16076.90 ( 16076.90). 4 =1804.25 ( 1804.25). 5 =3469.88 ( 3469.88)


For example, in bar#5288, the 10 period average of data4 based on the value of data4 as at barstatus(1)=2 is 1804.175 instead of 1804.25. Therefore. if we use average function to calculate averages across different data series, the time period covered under each average function is different and the comparison among these averages may not yield meaningful result. Alternatively, using array function together with barstatus(1) to calculate averages on secondary data series may be more appropriate in the circumstance when one needs to compare data over the same time period.

We have changed the conditions for the output. Now it is more clear how the average is calculated.
Please also check the attached screenshot.