Data Range >> Ticks Back

Questions about MultiCharts and user contributed studies.
User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Data Range >> Ticks Back

Postby bensat » 17 Feb 2016

Version : MultiCharts64 Version 9.0 Release (Build 11210)

I'm plotting the L2 of several contracts and always wondering why the charts are crashing or getting stuck at some point in the afternoon. Today I was surprised to see the tick chart extending it's <DATA RANGE> I set. As you can see in the screenshot I set the data range for the contract as 2000 ticks back. Further you may notice, that ticks/data are shown way earlier then 12.30pm ...

Image

In the second chart you can see that i started this study around 10am in the morning.

Image

Knowing that the EuroStoxx50 Future is a busy contract, I believe the data of contract should show more than 2000 ticks from 10am till 6pm right where I am writing this post. In the 3rd chart you can see that more then 2000 ticks occured since 10am in the morning.

Image

My question is why the data range in the 1 tick chart exceeds the settings set and are not erased from tick 2001 on ?

Thank you.

Regards.

Ben

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 18 Feb 2016

Same today ... Chart exceeds settings in data range ... all bars are still shown while most of it should be already deleted from the chart ... In both, @FESX & @FGBL ... So is it teh setup or is it a 'bug' ???

Image

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 20 Feb 2016

So as nobody answers II would like to raise up the question ....

1. I set it up the wrong way and I don't get it ?

2. Nobody understands my problem because I made no clear explanation about the issue ?

3. It is not an issue and an expected behavior ?

Just give me a number and I am fine, but this question sitting here for 4 days without any answer makes me nervous. Have a nice weekend.

Regards

Ben

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Data Range >> Ticks Back

Postby TJ » 20 Feb 2016

Since when did you start observing this in T radeStation?


If MultiCharts were to delete the oldest bar on every increment,
then the Barnumber will be static.
Many indicators use barnumbers for calculations. They too will stand still.

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 20 Feb 2016

Hi TJ and thank you for your answer.

I observed this behavior the first time I wrote/worked with this study and my workspace (MC64) with one chart (ES 1tick) almost died after 30min. First I thought it is due to the high amount of plots or the high amount of data the study had/has to analyse. So I tested it within different and less order/tick active symbols without success. I tried further investigation and changed the data range settings while expecting less data to be plotted and therefore more easy work on cpu and memory ... puuuuh wrong expectations. I checked the charts and saw that the amount of data shown and analysed in the charts was changing/increasing from tick to tick ....

In your first and unedited comment you mentioned the described issue is an expected behavior and <Data Range> is just for the amount of data to be select/show when the chart opens. This argument surprised me. Because I expect to have the amount of data in the chart at a fix amount when I set the <data range> to a certain level.

I can not follow your <barnumber> argument, as a barnumber depended user won't choose <data ranges>. Further I can't follow the argumentation that it would be just to load less data at the opening of the chart, because the data always increases. Obviously there is a huge disadvantage of this setup, as a user can not limit the amount of data he needs for his calculations. As "max bars back" implementation is false/wrong (yeah I no this is another discussion), I saw this as the only chance to control the amount of data I really want to be analysed. In this case I'm stuck in my analysis because I can not the limit the amount of data. Result: workspace crashes or is unusable.

The current implementation of "Ticks Back" is dependent on heavy cpu and ram usage.

Further question is why I should choose an amount of data/data range to be respected just at the opening of the chart ? So it is the same with "Bars Back" in Daily-/Min-/Tick-/Point-/Contract-charts ? If not, why there is any difference between "Bars Back" and "Ticks Back". I will test that on Monday.

Thank you.

Regards.

Ben

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 22 Feb 2016

Is this forum still watched and maintained by the TSSupport guys ? Have not seen any answer/comment since a week or so ... I am worried, I am afraid.

Regards.

Ben

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Data Range >> Ticks Back

Postby arnie » 22 Feb 2016

Is this forum still watched and maintained by the TSSupport guys ? Have not seen any answer/comment since a week or so ... I am worried, I am afraid.

Regards.

Ben
Everybody knows that when a new MC version is in the works (ie. when it's in its final testing period) the forum support goes quiet for a while. All hands on deck...

But they are always available via live chat for all important issues that need immediate attention.
Last edited by arnie on 26 Feb 2016, edited 2 times in total.

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 24 Feb 2016

Arnie, thank you. I am aware of that. Unfortunately someone had a look at the forum posts and answered elsewhere, but wasn't able to answer on this issue. I feel ignored. Very much.

Just an answer like 'this behavior is expected' or 'we are investigating' is much appreciated. But hell no, .... nothing.

Regards.

Ben

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 26 Feb 2016

I just tested this issue with min-bars and set a 50 bars <DATA RANGE> noticing it is the same behavior in the min-charts like in the tick-charts with <Ticks Back> option.

min-chart set to 50 <Bars Back> data range

Image

min-chart set to 50 <Bars Back> data range after some time ... barnumber increases

Image

So my conclusion have to be that this issue is not an issue but an expected behavior ??!! Now I am even more confused as I don't see any advantage setting a <DATA RANGE> as <Bars Back> or <Ticks Back> and to expect this number to increase during the session ...

It feels awkward not to get into this idea ?

So is it really an 'expected behavior' or is it a bug ? TSSupport guys ignoring me day by day ... Is that intentional or just incidental ... Still confused.

Regards.

Ben

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Data Range >> Ticks Back

Postby arnie » 26 Feb 2016

When you open a chart MC needs to know how much data it has to load from the historical database (on your computer or on your datafeed servers). Otherwise how would he know what to load?

If you set it to 50 bars then 50 bars will be loaded from the database.
From that point forward it's realtime data (new data) being added to the chart. It has nothing to do with the data loaded.

It makes no sense for the chart to continually remove the first bar on the left side everytime a new bar is plotted on the right side.

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 26 Feb 2016

When you open a chart MC needs to know how much data it has to load from the historical database (on your computer or on your datafeed servers). Otherwise how would he know what to load?

If you set it to 50 bars then 50 bars will be loaded from the database.
From that point forward it's realtime data (new data) being added to the chart. It has nothing to do with the data loaded.
Yes for sure and it is not the issue.

It makes no sense for the chart to continually remove the first bar on the left side every time a new bar is plotted on the right side.
Just think about it again as this is the reason why my tick charts, with this type of analysis & plots is getting stuck so quickly, because there is nothing to be freed what is not needed anymore !!! That's the whole point. If the chart would stay with a data range of 2000 ticks only to be analysed and plotted, everything would be working fluid and smooth. But the more ticks are added, the more it is becoming unusable.

Regards.

Ben

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Data Range >> Ticks Back

Postby arnie » 26 Feb 2016

I'm sorry but makes no sense what you're saying.

Why don't you change the indicator so it can only read the last 2000 ticks and not the entire data series? This way you would release computer resources.
Also, by rearranging the chart window size, you can set it to only show the last 2000 ticks since prior data series becomes hidden on the left side.

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 27 Feb 2016

I'm sorry but makes no sense what you're saying.

Why don't you change the indicator so it can only read the last 2000 ticks and not the entire data series? This way you would release computer resources.
I considered it, but then the user would have to take care of every single chart, the input set for every single study & chart is correct. Further what you are talking about would do a correct implemented <MAXBARSBACK> function as it was supposed to be at the beginning of TS4. That's what I am talking about for quiet some time and getting my comments deleted now.
Also, by rearranging the chart window size, you can set it to only show the last 2000 ticks since prior data series becomes hidden on the left side.
First this does not solve the problem at, second I take it as sarcasm, a joke and therefore not serious.

Regards.

Ben

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Data Range >> Ticks Back

Postby arnie » 28 Feb 2016

First this does not solve the problem at, second I take it as sarcasm, a joke and therefore not serious.
Of course I was joking.
I have nothing better to do with my life than come here and try to help people searching for help with sarcastic comments.

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Data Range >> Ticks Back

Postby JoshM » 28 Feb 2016

So my conclusion have to be that this issue is not an issue but an expected behavior ??!! Now I am even more confused as I don't see any advantage setting a <DATA RANGE> as <Bars Back> or <Ticks Back> and to expect this number to increase during the session ...

It feels awkward not to get into this idea ?
Depending on how your scripts behave, you can try to implement CommandLine() with a data reload (see the list of parameters here). That way you might be able to 'reset' the data series during the trading day, automatically at a fixed interval.

Other than this I don't know of a way to keep the number of bars loaded around the same 2,000 figure. This would require a new feature being added to the platform, and judging from your comments there's not much interest to implement this it seems.

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Data Range >> Ticks Back

Postby JoshM » 28 Feb 2016

Just think about it again as this is the reason why my tick charts, with this type of analysis & plots is getting stuck so quickly, because there is nothing to be freed what is not needed anymore !!! That's the whole point. If the chart would stay with a data range of 2000 ticks only to be analysed and plotted, everything would be working fluid and smooth. But the more ticks are added, the more it is becoming unusable.
Perhaps off-topic, but I see in your forum profile that your computer has 2 x Xeon X5680 @4.2Ghz plus 96GB RAM ECC. Perhaps we should focus in this thread on what's causing your CPU and memory issues, and let the 'data range is not a fixed number of bars' issue rest for a while? Since you have such a good/fast computer, it's hard for me to imagine that tick charts give you problems. They clearly should not.

(I cannot see your imgur images by the way, so I might be missing something).

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 01 Mar 2016

JoshM, good morning and thank you for your comments.

As you are unable to see the Imgur-images I attach them to this comment.

The hardware setup is not an issue nor is it the design itself.

Implementing a limitation for reading ticks back makes it even more worse, because with all these fast updates in the tick charts, it is calculating, releasing plots, plotting and recalculating not fast enough, as a new update already came in. The looping process kills itself at some point and causes the study to be stuck much quicker. Resetting the study would lead into releasing all plots from the past bars. I am already working to save the data tick by tick and to be able to load them again locally if the chart is reset or loaded. But ELC makes me headache as it is slow. Further working on caching the whole system into RAM and so on and so on.

To the other comments, no limiting the plots and calculations doesn't release computer sources. Again, a correct implementation of <MAXBARSBACK>, as it was supposed to be implemented when RAM and CPU power was limited in the late 90's, would solve the problem ... <MAXBARSBACK> = 2000 ... just 2000 bars analysed by design ;)

Regards.

Ben
Attachments
c3.png
(188.35 KiB) Downloaded 1708 times
c2.png
(309.43 KiB) Downloaded 1706 times
c1.png
(289.43 KiB) Downloaded 1709 times

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re: Data Range >> Ticks Back

Postby JoshM » 01 Mar 2016

Implementing a limitation for reading ticks back makes it even more worse, because with all these fast updates in the tick charts, it is calculating, releasing plots, plotting and recalculating not fast enough, as a new update already came in. The looping process kills itself at some point and causes the study to be stuck much quicker. Resetting the study would lead into releasing all plots from the past bars. I am already working to save the data tick by tick and to be able to load them again locally if the chart is reset or loaded. But ELC makes me headache as it is slow. Further working on caching the whole system into RAM and so on and so on.
I'm not sure if this will help since you have a complex case here, but how about having a strategy calculate the values and then pass these to an indicator with the I_SetPlotValue() keyword?

I'm suggesting that because from my understanding your indicators perform a tick-by-tick calculation and plot that result every bar. However, a strategy with IOG enabled and the Bar Magnifier set to 1 tick can also perform a tick-by-tick calculation and that calculation can also be performed on historical data (something that indicators cannot do).

If the signal script then calculates the values and passes these to the indicator, then you don't need to work with ELC to find a way to save and restore the indicator's values. (Because the strategy will do that then).

It might also be a little bit more efficient since the strategy will then only need to process each tick once (and pass the computed value to the indicators), as opposed to each indicator processing that tick individually.

User avatar
bensat
Posts: 331
Joined: 04 Oct 2014
Has thanked: 46 times
Been thanked: 104 times

Re: Data Range >> Ticks Back

Postby bensat » 01 Mar 2016

JoshM, thank you.

Puuuh, this is something I did not consider so far. I will have to stick my nose into your suggestion and have to test it.

My only concern is that the data being analysed are not available as historical data. These are order book data for the 10 bids and ask levels and base of further analysis and calculations. So I have to store them at some point to have them available for further analysis after a reset of the studies, chart or something else. But this has nothing to do with the issue itself.

Something I'm interested in to know is, why the performance of the workspace is becoming worse over the time and added ticks in the chart, when every single added tick just adds 20 more plots per tick. What exactly happens with the plots from the previous ticks related to rest, re-plot and calculations for every added tick ?

An implementation of RecalcLastBarAfter doesn't help performance wise.

Regards.

Ben


Return to “MultiCharts”