TRUE global tick-by-tick playback - possible?

Questions about MultiCharts and user contributed studies.
vking
Posts: 235
Joined: 21 May 2009
Has thanked: 51 times
Been thanked: 41 times

Re: MultiCharts.NET advantages

Postby vking » 25 May 2012

Rather than pointing the disadvantages of NT - Let me point the strengths of NT compared to current version of MC and really hoping for these features to be available in MC.NET at some point.

- Total Market replay - all charts/instruments at the same time - tick by tick
- Order simulator during replay ( live like trading but during replay) - one can test any type of orders that one would use during live market - strategies/keys any thing
- stored level 2 data during replay ( though this feature is nice - it's not as important as the above features).
- Unmanaged order handling - one can build any type of advanced order handling system

thanks
Last edited by vking on 25 May 2012, edited 2 times in total.
These users thanked the author vking for the post (total 3):
LaurentStan BokovJoshM

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: MultiCharts.NET advantages

Postby Stan Bokov » 28 May 2012

vking wrote:Rather than pointing the disadvantages of NT - Let me point the strengths of NT compared to current version of MC


Precisely, thank you. This is more important, rather than things we are already better at.

- Total Market replay - all charts/instruments at the same time - tick by tick

I just have one question - how? How is this logically possible? I just can't get my head around it.

It would be very simple to play +1 tick on every chart, but not useful or accurate in any fashion. If some charts don't tick, do they stay still while others tick? Are they synched on a second basis, or millisecond? What if the data provider doesn't provide ms? What is they come more often than each ms - how do we know which to play first?

Pls share an algorithm of how you see it working.

- Order simulator during replay ( live like trading but during replay) - one can test any type of orders that one would use during live market - strategies/keys any thing

This I totally agree with, this is a good suggestion, for both MC versions.

- stored level 2 data during replay ( though this feature is nice - it's not as important as the above features).

OK, we'll keep it in mind.

- Unmanaged order handling - one can build any type of advanced order handling system

This is planned already. It was posted in the blog (one of the comments in response to a similar question).

thanks[/quote]
These users thanked the author Stan Bokov for the post (total 3):
LaurentarnieTJ

vking
Posts: 235
Joined: 21 May 2009
Has thanked: 51 times
Been thanked: 41 times

Re: MultiCharts.NET advantages

Postby vking » 28 May 2012

Stan - Based on my understanding - all instruments are for which the data is downloaded ( including level 2) I can replay as shown below:



Stan Bokov wrote:- Total Market replay - all charts/instruments at the same time - tick by tick

I just have one question - how? How is this logically possible? I just can't get my head around it.

It would be very simple to play +1 tick on every chart, but not useful or accurate in any fashion. If some charts don't tick, do they stay still while others tick? Are they synched on a second basis, or millisecond? What if the data provider doesn't provide ms? What is they come more often than each ms - how do we know which to play first?

Pls share an algorithm of how you see it working.

[quote]
Attachments
NT_Replay_4.png
even can place orders during replay ( both manual and strategy based ) any thing that we can do in live market - can be done during replay - from order management point of view.
NT_Replay_4.png (245.92 KiB) Viewed 484 times
NT_Replay_3.png
connect to the replay data and start playing..
NT_Replay_3.png (213.59 KiB) Viewed 477 times
NT_Replay_2.png
download any other instruments of instruments.
NT_Replay_2.png (16.7 KiB) Viewed 479 times
NT_Replay_1.png
Initially I decide which instruments I would be interested in replaying and start downloading the replay data ( both level 1 & level 2).
NT_Replay_1.png (15.16 KiB) Viewed 481 times

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: MultiCharts.NET advantages

Postby Stan Bokov » 28 May 2012

vking,

I don't think you quite understand. Correct me if I'm wrong, but I believe that "global tick-by-tick" in NT doesn't do what you think it does. There are no MICROseconds in NT stored (they don't stamp, and data sources don't send that kind of info). Therefore, it's not possible to truly arrange the ticks in the order they came, and replay them with the same delay as happened in real-life.

For individual symbols, we stamp with a unqiueID that tracks which tick came first within a second. This makes it even more accurate than microseconds, but this approach can't be applied to the entire universe of symbols, only "within" a symbol.

You showed me some screenshots - I've seen the interface. But under the hood, is it any different from our second-by-second playback? We could call it tick-by-tick, but that wouldn't make it any more accurate.

So, I'm still confused and my logical questions remain. Do you have an algorithm of how to address the questions I posed above?
These users thanked the author Stan Bokov for the post:
TJ

vking
Posts: 235
Joined: 21 May 2009
Has thanked: 51 times
Been thanked: 41 times

Re: MultiCharts.NET advantages

Postby vking » 28 May 2012

Stan - At present - NT does allow "Global Replay" below 1 minute time interval(1x replay - not sure what resolution is this but definitely less than 1 minute resolution- In MC - you can't go below 1 minute resolution ) . Entire market ( with instrument of choice) - I can replay which is less than 1 minute resolution - without missing any ticks in any instruments selected. I am not sure - 1x replay - at what speed it replays - but this is fine with me.

At present - MC - doesn't allow - replay tick by even for single instrument - when multiple tick based resolutions are available in a single chart ( you can only replay at 1 minute interval ). I am not sure about the technical reason behind not allowing less than 1 minute resolution for multi tick based chart - even for a single instrument( from single datasource). If you can address this issue - This itself would be a big thing for me :)

Thanks

Stan Bokov wrote:vking,

I don't think you quite understand. Correct me if I'm wrong, but I believe that "global tick-by-tick" in NT doesn't do what you think it does. There are no MICROseconds in NT stored (they don't stamp, and data sources don't send that kind of info). Therefore, it's not possible to truly arrange the ticks in the order they came, and replay them with the same delay as happened in real-life.

For individual symbols, we stamp with a unqiueID that tracks which tick came first within a second. This makes it even more accurate than microseconds, but this approach can't be applied to the entire universe of symbols, only "within" a symbol.

You showed me some screenshots - I've seen the interface. But under the hood, is it any different from our second-by-second playback? We could call it tick-by-tick, but that wouldn't make it any more accurate.

So, I'm still confused and my logical questions remain. Do you have an algorithm of how to address the questions I posed above?

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: MultiCharts.NET advantages

Postby Stan Bokov » 28 May 2012

vking,

So, are we talking about true global tick-by-tick playback, or global second-by-second playback (less than 1 minute as you mention)?

These are fundamentally different things. Global second-by-second is not technically different and present no logical conundrums, since our timestamps are more accurate than 1 second.
We will add it in one of the near MC8 versions. That will be a plus in its own right.

True global tick-by-tick, however, the questions are still open.
These users thanked the author Stan Bokov for the post:
vking

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 28 May 2012

A question was raised in the .NET thread, and I thought it deserved its own discussion. Some users mention that supposedly NT offers global tick-by-tick playback, while we do not see how that's technically possible. So, I'm asking for input from users to help me understand the algorithm, and solve this logical riddle.

Would like to hear opinions on how to accomplish this.

User avatar
arnie
Posts: 1567
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 470 times
Been thanked: 503 times

Re: TRUE global tick-by-tick playback - possible?

Postby arnie » 28 May 2012

Maybe we could achieve global tick/second by tick/second playback with multiple symbols using something that IRT presented recently. They call it Align to Master Periodicity. We could align several symbols/chart windows to a master symbol/chart window which would command all symbols playback.

You can see a video explanation here:

http://www.screencast.com/t/jof73fX2
These users thanked the author arnie for the post:
Laurent

vking
Posts: 235
Joined: 21 May 2009
Has thanked: 51 times
Been thanked: 41 times

Re: TRUE global tick-by-tick playback - possible?

Postby vking » 30 May 2012

Stan - Do you see any technical issues with tick-by-tick replay with One Single Chart - One single instrument ( but multi-series with different tick based time frames ) - Single data source.

If there are no technical issues with this approach - Would you please see if you can have tick by tick replay enable instead of proposed "seconds" based - for a single multiseries chart. At present this is not supported in MC and would be a very nice feature to have.

thanks

Stan Bokov wrote:A question was raised in the .NET thread, and I thought it deserved its own discussion. Some users mention that supposedly NT offers global tick-by-tick playback, while we do not see how that's technically possible. So, I'm asking for input from users to help me understand the algorithm, and solve this logical riddle.

Would like to hear opinions on how to accomplish this.
These users thanked the author vking for the post:
Stan Bokov

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 31 May 2012

vking wrote:Stan - Do you see any technical issues with tick-by-tick replay with One Single Chart - One single instrument ( but multi-series with different tick based time frames ) - Single data source.

If there are no technical issues with this approach - Would you please see if you can have tick by tick replay enable instead of proposed "seconds" based - for a single multiseries chart. At present this is not supported in MC and would be a very nice feature to have.


vking,

Thanks for your comments. I understand what you mean about global second-by-second playback. We are currently investigating how this can be implemented. Also looking at improving tick-by-tick playback on a single multi-series chart. It won't be in the release, but we are looking at it.

Jesh
Posts: 41
Joined: 04 Jan 2011
Has thanked: 12 times
Been thanked: 4 times

Re: TRUE global tick-by-tick playback - possible?

Postby Jesh » 05 Jun 2012

The NT Market Replay is downloaded from Kinetic which is basically there version of rebranded DTN IQ feed and this includes the tick timestamps from what I understand.

The market replay replay data for a select set of instruments is available at the end of the day and can be downloaded on a day-to-day basis directly from the NT servers. The market replay feature is something that keeps me still using NT as well as MC.

User avatar
JoshM
Posts: 2074
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1523 times
Been thanked: 1479 times
Contact:

Re: MultiCharts.NET advantages

Postby JoshM » 06 Jun 2012

Stan Bokov wrote:I just have one question - how? How is this logically possible? I just can't get my head around it.

It would be very simple to play +1 tick on every chart, but not useful or accurate in any fashion. If some charts don't tick, do they stay still while others tick? Are they synched on a second basis, or millisecond? What if the data provider doesn't provide ms? What is they come more often than each ms - how do we know which to play first?

Pls share an algorithm of how you see it working.

True global tick by tick replay cannot be done in MultiCharts in my opinion (that is, in the current state). This is because the milliseconds aren't saved in the QuoteManager database. The question is of such a thing is a problem. Would a user notice the difference if the replay has a fixed millisecond lag of say 100 between ticks while in real-time this was this fluctuated between 50 and 200 ms? (For strategies, that would be another matter).

Unless MC chooses to incorporate milliseconds (or lower) into the QuoteManager database, "true" tick by tick replay may not be possible (again, in my non-expert opinion).

However, the TickID can be used for a "true sequential tick-by-tick replay". For example:
globalTickByTickreplay.png
globalTickByTickreplay.png (92.79 KiB) Viewed 498 times


Don't know if this will work in practice though. :)

------
Edit:
Actually, a "true global sequential tick-by-tick replay" would be far superior than a "true global time based tick-by-tick replay" since the latter still doesn't ensure that the ticks are replayed exactly as they were received in real-time.

For example, with a time based replay they're still some assumptions to be made (such as, which tick was received first when the time stamps are the same). With a TickID based global replay, these assumptions aren't necessary and the ticks are truly displayed sequential. :)
These users thanked the author JoshM for the post (total 5):
vkingarniekhalaadStan BokovLaurent

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 06 Jun 2012

JoshM,

Thank you for your comments. Sequential TickID is currently implemented within a second and within each symbol separately. Another issue is that ticks are only stamped when you receive them in real-time. If you download historical from a data feed, we can't stamp them (we don't know which one came first), and they don't come with millisecond stamps.

What if a user didn't turn the platform on in time? Or had a power outage? Data will have to be downloaded from a data feed, and you'll have holes in your TickID's.

Everything would be much simpler if, say, IQFeed sent millisecond timestamps for all historical and real-time ticks. Then we don't even have to change a thing, simply play them in the right order. Think you can convince them to do that? ;)

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 06 Jun 2012

Jesh wrote:The NT Market Replay is downloaded from Kinetic which is basically there version of rebranded DTN IQ feed and this includes the tick timestamps from what I understand.


Can you show me?

User avatar
JoshM
Posts: 2074
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1523 times
Been thanked: 1479 times
Contact:

Re: TRUE global tick-by-tick playback - possible?

Postby JoshM » 06 Jun 2012

Stan Bokov wrote:Sequential TickID is currently implemented within a second and within each symbol separately. Another issue is that ticks are only stamped when you receive them in real-time. If you download historical from a data feed, we can't stamp them (we don't know which one came first), and they don't come with millisecond stamps.

What if a user didn't turn the platform on in time? Or had a power outage? Data will have to be downloaded from a data feed, and you'll have holes in your TickID's.

I know -- have you looked at the flowchart? It's covered in the 'left path'.

In such a case MC would need to make more assumptions. Indeed, that would not give true global tick-by-tick replay, but might be a viable alternative when the user doesn't has TickID's in his database. I don't think global tick-by-tick replay can be ever made so perfect as you'd like it (unless you succeed in convincing the majority of MC's data feed providers :) ) so a compromise has to made somewhere in my opinion. Otherwise, in two years from now it still won't be implemented and your competitors will still have a (smallish) edge over MC in this area.

I think if you implement both replay options (global based on time stamp of ticks and global based on TickID) you'd have the best of two world: true sequential global tick-by-tick (based on TickID) and global tick-by-tick for users with data without TickID's. I also suppose that most users will find those two options quite understandable given the technical limitations. This would be something similar as the Bar Magnifier option: this is available with tick resolution if the user has that in his database, or otherwise with minute resolution if that's available.

I'd suggest to consider implementing both options and provide in depth articles on the Wiki regarding the assumptions that are used in the global replay. That way, the user can judge the features of global replay to see if it's viable given his or her goals.
These users thanked the author JoshM for the post (total 4):
vkingTJLaurentStan Bokov

User avatar
JoshM
Posts: 2074
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1523 times
Been thanked: 1479 times
Contact:

Re: TRUE global tick-by-tick playback - possible?

Postby JoshM » 06 Jun 2012

Stan Bokov wrote:
Jesh wrote:The NT Market Replay is downloaded from Kinetic which is basically there version of rebranded DTN IQ feed and this includes the tick timestamps from what I understand.

Can you show me?

I find it somewhat odd that MC apparently doesn't use/test/review the product of their biggest competitor. Off course, it isn't needed to be obsessive about NT (or any other competitor), but I assumed that at least on one PC/VM you'd had a test environment with NT.

Anyway, as far as I can tell (with my dummy NT skills :) ), NT doesn't have a Market Replay with micro second time stamps:
scr.06-06-2012 16.21.05.png
scr.06-06-2012 16.21.05.png (185.52 KiB) Viewed 483 times
These users thanked the author JoshM for the post (total 2):
TJLaurent

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 06 Jun 2012

JoshM wrote:I find it somewhat odd that MC apparently doesn't use/test/review the product of their biggest competitor. Off course, it isn't needed to be obsessive about NT (or any other competitor), but I assumed that at least on one PC/VM you'd had a test environment with NT.

Anyway, as far as I can tell (with my dummy NT skills :) ), NT doesn't have a Market Replay with micro second time stamps:
scr.06-06-2012 16.21.05.png


We do look at it, we own a license in fact. As I said earlier in this thread, it appears that what they call tick-by-tick, is actually global second-by-second. We don't have that, but we'll add second-by-second soon.
These users thanked the author Stan Bokov for the post (total 4):
TJvkingLaurentJoshM

vking
Posts: 235
Joined: 21 May 2009
Has thanked: 51 times
Been thanked: 41 times

Re: TRUE global tick-by-tick playback - possible?

Postby vking » 06 Jun 2012

Stan - How about the mode for single instrument/single data source/multi-series(post #10) - ticky-by-tick(not second by second but tick-by-tick) - Would it be possible to add that as well along with the planned second-by-second global replay. There shoudn't be any technical limitations for this that I can think of.

Thanks

Stan Bokov wrote:
JoshM wrote:I find it somewhat odd that MC apparently doesn't use/test/review the product of their biggest competitor. Off course, it isn't needed to be obsessive about NT (or any other competitor), but I assumed that at least on one PC/VM you'd had a test environment with NT.

Anyway, as far as I can tell (with my dummy NT skills :) ), NT doesn't have a Market Replay with micro second time stamps:
scr.06-06-2012 16.21.05.png


We do look at it, we own a license in fact. As I said earlier in this thread, it appears that what they call tick-by-tick, is actually global second-by-second. We don't have that, but we'll add second-by-second soon.

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 06 Jun 2012

vking,

we are looking at that as well. I don't see a reason why we shouldn't be able to do that for a single instrument.
These users thanked the author Stan Bokov for the post (total 3):
vkingTJarnie

User avatar
arnie
Posts: 1567
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 470 times
Been thanked: 503 times

Re: TRUE global tick-by-tick playback - possible?

Postby arnie » 07 Jun 2012

Stan Bokov wrote:We do look at it, we own a license in fact.


I hope that when you guys decide to implement market/volume profile and footprint charts you subscribe to MD so you can see what and how they do what they do best.

I wonder if NT support have a MC license.....

I'd would prefer to see MC leading the pack, not the other way....
These users thanked the author arnie for the post:
Laurent

User avatar
Stan Bokov
Posts: 963
Joined: 18 Dec 2009
Has thanked: 367 times
Been thanked: 300 times

Re: TRUE global tick-by-tick playback - possible?

Postby Stan Bokov » 07 Jun 2012

JoshM,

That's a very good flowchart. Thank you for putting in the time and the effort to map it out. We are researching it and comparing to possible workloads and possibilities of MC. We'll update you all should we have more questions or need clarification.
These users thanked the author Stan Bokov for the post:
JoshM


Return to “MultiCharts”