A SOLUTION TO BROKEN CANDLES IN TIME BASED CHARTS

Questions about MCFX and MCFX Data Feed.
User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

A SOLUTION TO BROKEN CANDLES IN TIME BASED CHARTS

Postby terminal7 » 13 Apr 2007

A SOLUTION TO BROKEN CANDLES IN TIME BASED CHARTS

Copyright April 13, 2007/Patent Pending

A practical solution to the problem of broken candles/bars (candles that are closed by a session break before the chart criteria is meet) in data charting software for stock, Forex, and other markets. The solution applies only to TIME BASED charts as charts of other compressions, such as Point or Change, do not require a synchronization threshold since the criteria of such charts does not reference the horizontal axis.

to be revealed Monday...

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 16 Apr 2007

The solution involves moving the synchronization threshold UP THE SCALE from the daily or 24 hour level in Forex to a weekly or 168 hour (7 day) level. Whereas the current practice of breaking candles after 24 hours would result in a broken candle ratio of 1:3 (1 out of every 3 candles is broken) in a 10 hour chart, the 168 hour threshold would lower that ratio to 1:17. A daily or 24 hour chart would have a break ratio of 1:7. An even higher threshold, such as monthly or 672 hours (168 X 4), could be chosen and would reduce the broken candle ratio even further.

Yosnappyj correctly points out in his "bar anomalies..." thread that the current session break (synchronization threshold) of 24 hours is ENTIRELY ARBITRARY. Hence it could be moved by little more than a programming slight of hand to a level that would make broken candles in sub-daily charts so rare that they cause virtually no distortion or artificial inflation of the charts.

TSsupport correctly contends that a synchronization threshold IS REQUIRED in time-based charts but they are mistaken to imply that the broken candle delima cannot be easily solved. The solution is absurdly simple and could likely be implimented with little or no negative consequence. In fact, it would be transparent to traders. The only question is what to move the threshold up to.

The phenomena of broken candles occurs near the threshold (that is why there is no current broken candle issue in 5 min. charts - because the chart criteria is so far removed from the daily session break). Therefore, the higher you move the threshold, the higher you move the zone of chart resolutions that will be problematic with regard to broken candles. Since many traders use sub-daily charts to trade it makes sense to remove the broken candle problem from these charts. Moving the synchronization threshold upward nicely resolves this issue.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 16 Apr 2007

TSsupport:

I'm sure that you recognize that broken candles are a big issue to some of your clients. I hope that you are concerned about the internal integrity of your charts. Feel free to impliment the idea expressed above as it appears that I cannot patent it. It is yours to use and we hope that you'll use it to improve our beloved MCFX.

In a future posting I'll discuss the prudency of recognizing session breaks in non-time-based charts...

Rgds,
T7

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 17 Apr 2007

The solution involves moving the synchronization threshold UP THE SCALE from the daily or 24 hour level in Forex to a weekly or 168 hour (7 day) level.
I think that's a good, practical suggestion. There are more refined methods, but as the data feed is broken over the weekend in any case, synchronising at the weekly level makes practical sense as a first step.

Before that can happen though, TSSupport need to acknowledge that broken bars are "a bug and not a feature", even if produced intentionally.

Browsing back through the forum archive, I found this exchange from last summer:

http://forum.tssupport.com/viewtopic.ph ... ed7508351d

It seems as though they're unable to see the consequences of this regrettable choice.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

yosnappyj,

I was wondering what you thought of the idea so I was glad to see your reply (hoping also for a reply from TSsupport but that's looking more and more like hoping for manna from heaven...). If only more traders would look into this and speak up about this issue - we may stand a fighting chance of getting something done about it.

Thanks for your research. This is the first testimony we've had concerning the effects of broken candles on indicators. Here are a few excerpts for everyone's convienance:
from a July 23, 2006 thread...

Trader:
Referring back to a previous concern with the "empty" bars at session breaks. Is there anyway to hide these? It really makes my indicators look choppy.

For instance, if I set a 4000 tick bar chart - at 5pm EST (Session Break), it will break up my bar even if it is at less than 400 ticks. I need to exclude session breaks and have all my bars respect the number of ticks specified.

Stanley Miller:
We have to use this logic and start bar calculation from beginning of the session to make the charts always identical. If we won’t do that, then a chart built from 500 4000 tick bars will be radically different from a chart built from 501 4000 tick bars, as everything will depend on the starting point. In this case calculation always begins from the session start.

Trader:
Ok, I understand that. But is there any way to hide the session breaks on my chart - they look really choppy.

Stanley Miller:
You can hide the session break line if necessary.
What is most interesting to me about this exchange is the last question the trader asks about hiding session breaks. From his first line we clearly see that he is talking about eliminating broken "bars". However Mr. Miller sidesteps this question by referring instead to the ability of MCFX to hide session break "lines". This skirting of the issue is very telling...

TSsupport knows that the problem of broken candles can be easily solved by redefining the "session". Their plan seems to be to divert attention from the issue and/or to claim that the problem is too difficult to solve. Their silence over the past few days is really tacit aknowledgement of the correctness of our claims as to the solution. They either lack the backbone to make such a brave new change or they are embarassed to admit that traders discovered a solution that their development team has overlooked for years.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 17 Apr 2007

This is the first testimony we've had concerning the effects of broken candles on indicators.

...

What is most interesting to me about this exchange is the last question the trader asks about hiding session breaks. From his first line we clearly see that he is talking about eliminating broken "bars".
I think there are two separate issues here. It seems that a previous version of MCFX would display a gap between bars at the session breaks. This gap would disrupt the visual flow of the chart, but mathematically has no effect on indicators. The other, separate issue is truncated candles.

TSsupport knows that the problem of broken candles can be easily solved by redefining the "session". Their plan seems to be to divert attention from the issue and/or to claim that the problem is too difficult to solve. Their silence over the past few days is really tacit aknowledgement of the correctness of our claims as to the solution. They either lack the backbone to make such a brave new change or they are embarassed to admit that traders discovered a solution that their development team has overlooked for years.
Let me just say that I don't think it pays to be too belligerent over this. It's clear that only a minority of traders are using MCFX in ways that cause session break errors to become obvious. MCFX will have limited resources of time and manpower to devote to resolving the problem, and it doesn't seem to be among their priorities. I'd like to think they'll be able to implement a solution (perhaps the one you suggested), but it may be that for the cost in time and manpower of doing so, they'd prefer to lose those few customers for whom this is an issue. That's business.

What I do find hard to believe is that this session break protocol was adopted in the first place. It hog-ties what could be a very versatile program and, unless changed, will prevent the implementation of useful features in the future.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

It's true that a previous version of MCFX inserted gaps between candles at the locations of session breaks. (The trader in question however was definitely talking about the truncated candle issue. You can tell by the way he describes the problem of his 4,000 Tick candle being broken below 400 ticks.) That wasn't too long ago and I remember that problem. To their credit, TSsupport fixed that visual distortion in their charts... it's ironic though that they are unwilling to fix (at least so far) the visual distortion that results from having an excessive amount of candles broken by a very low synchronization threshold.

The difference between that case and this case might well be the number of traders who were perturbed over the problem. In the former case the distortion was obvious because you had all these whitespace gaps all over the place. In the current problem we're having with SBC's the distortion is much less obvious but still present. It is difficult to provide an example of the distortion because it's hard to visualize what an un-infected chart would look like. But just by thinking about the problem one can see that the sheer corruption in the logic and internal integrity of the charts is undeniable.

The fact that only a minority of traders seem to be concerned about this issue is, in my opinion, no justification for dismissing it. TSsupport should be interested in improving their product and providing their clients with a more accurate picture of the market regardless of how many people complain about the present state of affairs. That's good business because it's forward thinking, it's progressive, and it results in a truly better product. If TSsupport is in the habit of using a "complaint meter" to initiate innovative changes, then clearly there are many smart changes that will never be made. This is because only a minority of traders ever subject the market to the studious rigor that it deserves. The majority follow the bandwagon and innovation rarely blooms from this group.
Let me just say that I don't think it pays to be too belligerent over this. It's clear that only a minority of traders are using MCFX in ways that cause session break errors to become obvious. MCFX will have limited resources of time and manpower to devote to resolving the problem, and it doesn't seem to be among their priorities.
Yosnappyj, let me just say that I have appreciated your calmness, your reasonableness, and your willingness to sympathize with support staff in discussing this issue. Your manner is more apealling, and in many cases I'm sure, would produce better results than mine - which falls more into the category of 'break the door down and storm the room'. However, there is a reason why I am so passionate about this particular issue and, in view of TSsupport's behavior so far, I don't think the 'Mr. Nice Guy' approach will get us very far.

For the past 5 years of my life I've studied (mostly) the stock market in hopes of unravelling the mysteries of price movement. I knew that anything that produced trending behavior had to have a structured process lying behind it. And this was my impetous for continued study. It isn't too much of a stretch to say that I paid for my knowledge with blood, sweat, and tears - perhaps not the first, but definitely the latter two. At this point in time I really believe that I've cracked it. Part of the model I came up with is explained in the informational site below...

http://www.reactionaryeventmodel.info

I never expected that I would end my quest, my journey of discovery, by wrangling with a bunch of stick in the muds over an issue like broken candles. But I am left with no other choice since truncated candles are playing fast and loose with my counts. They're messing with my baby...

In July I will marry and take on many new financial responsibilities. My financial future depends on my ability to impliment my market model. This issue threatens to undo much of what I've accomplished over the past 5 years. Can you imagine how frustrating this is for me? after working hard to become the rightful owner of the knowledge I need, to now have to beat my head against this beaurocratic blockade and plead simply for a true picture of the market (something that should be a given)?

It isn't just that... I view the market in an idealistic way. Since Dow, Gann, and Elliott men have put out great efforts to understand market movement. Elliott never really got it. But in some respects he came close. The point is that the more friction people like that get from people like TSsupport, the less likely it becomes that men will ever understand price movement in the full sense. As they really should. If I had the money I would buy TSsupport and MAKE them do the right thing. But as it is I can only appeal to them to do the right thing - not because it's good business - but because it is THE RIGHT THING.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

What I do find hard to believe is that this session break protocol was adopted in the first place.
It IS necessary to have a break somewhere in a TIME-based chart. Since it is a TIME chart it has to be synchronized by time. What IS hard to believe is that TSsupport never realized that the break did not have to correspond to the daily session close and that using a 'session span' of greater duration would virtually solve the broken candle issue in sub-daily charts...

What is even harder to believe is that they currently break PRICE and CHANGE charts by session as well. This practice is absolutely unecessary as these types of charts need no time synchronization at all. TSsupport, I challenge you to tell me that I'm wrong about this.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 17 Apr 2007

It's true that a previous version of MCFX inserted gaps between candles at the locations of session breaks. (The trader in question however was definitely talking about the truncated candle issue. You can tell by the way he describes the problem of his 4,000 Tick candle being broken below 400 ticks.)
Right, but when Stanley Miller says that the break line can be hidden if necessary, it's in reply to the trader's repeated question about "empty bars at session breaks".
Yosnappyj, let me just say that I have appreciated your calmness, your reasonableness, and your willingness to sympathize with support staff in discussing this issue. Your manner is more apealling, and in many cases I'm sure, would produce better results than mine - which falls more into the category of 'break the door down and storm the room'. However, there is a reason why I am so passionate about this particular issue and, in view of TSsupport's behavior so far, I don't think the 'Mr. Nice Guy' approach will get us very far.
In that case, maybe our good-cop-bad-cop act will wear them down.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 17 Apr 2007

It IS necessary to have a break somewhere in a TIME-based chart. Since it is a TIME chart it has to be synchronized by time.
In fact there are synchronisation protocols that need never break a bar, though they're more complicated to implement than the current brutal system, and the weekly break you suggest above.
What IS hard to believe is that TSsupport never realized that the break did not have to correspond to the daily session close and that using a 'session span' of greater duration would virtually solve the broken candle issue in sub-daily charts...
I think the explanation is clear. MCFX is an adaptation of Multicharts to the FX market. I presume Multicharts was first developed for markets with a clearly defined daily session, and the daily session break was incorporated at that stage. That would explain the presence of a daily session break protocol in a software package targeted at a continuous market, and the reluctance of TSSupport to adopt a more reasonable protocol.
What is even harder to believe is that they currently break PRICE and CHANGE charts by session as well. This practice is absolutely unecessary as these types of charts need no time synchronization at all.
Absolutely.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

Right, but when Stanley Miller says that the break line can be hidden if necessary, it's in reply to the trader's repeated question about "empty bars at session breaks".
Okay, I see what you're saying. My apologies to Mr. Miller. Aparently Stepwise refers to the gap problem in the first post and doesn't touch on the SBC problem until the second posting under his #1 entry. His last question is a reference to the gap problem that existed at the time not broken candles. Still, it is apparent that he was concerned with the truncated candle issue as well.
In that case, maybe our good-cop-bad-cop act will wear them down.
LOL! Yeah, maybe. But I ain't holdin' my breath.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

In fact there are synchronisation protocols that need never break a bar, though they're more complicated to implement than the current brutal system, and the weekly break you suggest above.
Hmmm... interesting. I really love your choice of words - "brutal system" is a perfect description.
I think the explanation is clear. MCFX is an adaptation of Multicharts to the FX market. I presume Multicharts was first developed for markets with a clearly defined daily session, and the daily session break was incorporated at that stage. That would explain the presence of a daily session break protocol in a software package targeted at a continuous market, and the reluctance of TSSupport to adopt a more reasonable protocol.
A very insightful and eye opening observation. I think TSsupport needs to rethink the Forex market and ask themselves this soul searching question:

Does the "presence of a daily session break protocol in a software package targeted at a continuous market" make sense?

God help me but I am fully willing to become a thorn in the side of TSsupport for as long as it takes to at least get an intelligent response to this issue if not a resolution. If it is a good idea but they are just unwilling to impliment it - for whatever reasons - why don't they just say so. It is much more becoming and appropriate than SILENCE. We have a legitimate problem and we've offered a legitimate solution, and in return we get - DEAD SILENCE... How can anyone respect that?

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 17 Apr 2007

Does the "presence of a daily session break protocol in a software package targeted at a continuous market" make sense?

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 19 Apr 2007

TSSupport:

What is your reason for breaking candles by session in POINT and CHANGE charts? It appears to me that these types of charts do not require synchronization as do TIME-based charts. If this is true simply do not reply to this. Traders will assume your silence to be tacit acknowledgement of the fact that you have implemented an entirely un-needed procedure in your POINT and CHANGE charts.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 20 Apr 2007

I think that's a good, practical suggestion. There are more refined methods, but as the data feed is broken over the weekend in any case, synchronising at the weekly level makes practical sense as a first step.
I was going to add that, of course, this shouldn't apply to multi-day candles, when it occurred to me to look at the current synchronising of multi-day charts in MCFX.

From a cursory inspection, it looks as though multi-day charts use the end of the current week as their one synchronisation point, and load up all n-day bars backwards from that point without any arbitrary breaks.

I created 23 and 17 day charts, which should coincide every 391 days if there are no broken bars. They do. I added a 3-day chart, which should align with the 23 and 17 once every 1173 days. It does.

So MCFX already contains a second synchronisation protocol which can synchronise long data sequences without breaking any bars.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 20 Apr 2007

MCFX already contains a second synchronisation protocol which can synchronise long data sequences without breaking any bars.
Whoa! That's a pretty piece of detective work there. Very nice. I had been thinking about whether breakage was a problem in the supra-daily area of resolution. So that means that the broken bar problem is confined to the sub-daily area, which ironically, is where most traders operate.

According to my understanding, the reason the current system is able to sync so well is that all the higher scale session closes (weekly, monthly, etc...) always correspond to the session break at the lowest scale (daily). No matter where a weekly or monthly candle ends, it's always a daily close as well - thus the foundation for synchronization.

In other words, where your session breaks or sync point(s) are doesn't matter. What matters is that they all agree with your lowest scale breakpoint. If that is the case then you have synchronization across scale.

If you were to impliment an hourly session then you would begin to have broken candle issues in the minute charts. The reason you don't is that the lowest scale session break (daily) is so far away that it gives minute candles 'breathing room' to be generated according to the chart criteria, or as Stepwise said in the thread you referenced from last summer, it allows them to 'RESPECT the number of ticks...'
I need to exclude session breaks and have all my bars respect the number of ticks specified.
So it follows that making the weekly session the lowest scale session or sync point would solve the broken candle problem in sub-daily charts as the sync threshold would be further removed from these charts. This, according to your statement above, would not create any disharmony in the supra-daily charts (daily or above) since the protocol that prevents broken bars in that area at present would remain. All you'd be doing is syncing the sub-daily charts to a higher protocol (that almost sounds religious <lol>). In fact, it seems that this change would be transparent, not affecting MCFX's ability to sync scales at all.

To bring up a daily chart under the new weekly synced system MCFX could be programmed to fetch a 24 hour chart when "1" + "day" is entered by a trader. The same for 2 day / 48 hour, etc...

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 20 Apr 2007

Here's another graphic example of the distortion and inflation that occurs in charts containing an excessive number of broken candles. The solution we recommend would lower the number of broken candles to such an extent that artificial inflation and event distortion would be kept to a minimum thereby increasing the fidelity of the charts.

Yosnappyj, could you help me with the "distortion rating"? The broken candle ratio (which in POINT charts is obtained by counting the number of candles in the chart VS. the number of broken candles in the chart) for FIG. B1 is 16:37. For B2 it is 5:26. What I'd like to know is, percentage wise, what the percentage of broken candles would be in both cases. For B1 it should be around 48%. For B2 it would probably be around 20%. But of course B2 does not have ALL the broken candles removed as they would be if the chart was generated by a non-corrupted version of MCFX.
Attachments
SBC2.gif
(22.13 KiB) Downloaded 2728 times

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 23 Apr 2007

So it follows that making the weekly session the lowest scale session or sync point would solve the broken candle problem in sub-daily charts as the sync threshold would be further removed from these charts. This, according to your statement above, would not create any disharmony in the supra-daily charts (daily or above) since the protocol that prevents broken bars in that area at present would remain. All you'd be doing is syncing the sub-daily charts to a higher protocol (that almost sounds religious <lol>). In fact, it seems that this change would be transparent, not affecting MCFX's ability to sync scales at all.
Unless there's some byzantine internal complication that only the developers are aware of, I'd say that's true. But better still would be to adopt the protocol currently used by the multi-day charts for all time scales, and synchronise to a single point. That point could be the end of the current trading week, or (for greater flexibility) it could be user-defined. The point is that it's arbitrary. There should be no need to break any bars in any time scale, nor to synchronise more than once.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 23 Apr 2007

What I'd like to know is, percentage wise, what the percentage of broken candles would be in both cases. For B1 it should be around 48%. For B2 it would probably be around 20%. But of course B2 does not have ALL the broken candles removed as they would be if the chart was generated by a non-corrupted version of MCFX.
I'm not sure I understand the question. Are you asking me how to calculate a percentage?

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 23 Apr 2007

I'm not sure I understand the question. Are you asking me how to calculate a percentage?
:oops: Yeah, that's what I need to know. I'm getting ready to move in a few months and my Cliff Notes on mathematics are packed away :)

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 23 Apr 2007

Yeah, that's what I need to know. I'm getting ready to move in a few months and my Cliff Notes on mathematics are packed away
The percentage of broken bars is simply the number of broken bars multipled by one hundred, divided by the total number of bars.

So, for your examples:

16:37

16 x 100 / 37 = 43%

5:26

5 x 100 / 26 = 19%

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 23 Apr 2007

Yosnappyj, thanks for the math work above. In a time-based chart you can find the broken candle ratio by dividing the length of the session break, namely 24, by the chart resolution. Therefore the ratio of a 10 hour chart is 2.4 which means that every third candle gets broken by the session. However, in a POINT chart the chart criteria is based on the vertical axis which cannot be used with session break duration in a time calculation. This fact also serves to highlight the TOTAL SENSELESSNESS of breaking POINT charts by a time element in the first place, or really of breaking the candles at all.

Yosnappyj, it seems that I've somehow overlooked your post, as quoted below and missed reading it. I'm so glad I stumbled over it though... In response to what I said about synchronizing high-scale session breaks with the lowest scale session break you said:
...better still would be to adopt the protocol currently used by the multi-day charts for all time scales, and synchronise to a single point. That point could be the end of the current trading week, or (for greater flexibility) it could be user-defined. The point is that it's arbitrary. There should be no need to break any bars in any time scale, nor to synchronise more than once.
I think that your observation here is inspired! I love it! Both the idea of a single syncing point and the idea of letting the sync threshold be controlled by the user are ingenious. IF TSSUPPORT IS NOT READING AND GIVING SOME SERIOUS THOUGHT TO THIS QUOTE, THEY ARE REALLY MISSING OUT ON A FLAWLESS AND INNOVATIVE THOUGHT... We can only hope that the moderator of this forum or one of the tech support people will have the good sense to pass this idea along to the development team.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 23 Apr 2007

I think that your observation here is inspired! I love it! Both the idea of a single syncing point and the idea of letting the sync threshold be controlled by the user are ingenious. IF TSSUPPORT IS NOT READING AND GIVING SOME SERIOUS THOUGHT TO THIS QUOTE, THEY ARE REALLY MISSING OUT ON A FLAWLESS AND INNOVATIVE THOUGHT...
Let's not get carried away. I'm only suggesting a broader application of a protocol that TSSupport has already implemented. There may be reasons that this protocol wasn't adopted for the sub-daily timescales. I can't see any disadvantages myself, but I'd be very interested to hear the developers' point of view. I know that Kate has declared the discussion closed, but I'm convinced that this protocol must have been considered already and rejected, and I'd like to understand why.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 23 Apr 2007

Let's not get carried away. I'm only suggesting a broader application of a protocol that TSSupport has already implemented.
Oh, right. You did mention that a single sync point is already in use at resolutions above 1 day. I think we're starting to confuse ourselves though... we should consider the system as a whole... if MCFX uses one supra-daily sync point AND one daily session sync point then we have a system that uses two synchronization thresholds to synchronize charts - not a single sync point system as you proposed.

By the way, what is that supra-daily sync point and are you sure it exists? For instance, how can you be sure that for chart resolutions above 1 day MCFX doesn't just say something like "skip X number of daily sessions before closing the candle"? Plugging in 7 for X would produce a weekly chart.

So let me see if I understand this correctly...

1. there are no broken candle issues in resolutions above 1 day

2. MCFX currently uses multiple sync points system-wide

True?

My current thinking is that, if you have a threshold anywhere, you're going to have breakage problems just below it, but not above it. If this is true then it makes sense to move it up-scale as much as is possible and reasonable.
There may be reasons that this protocol wasn't adopted for the sub-daily timescales. I can't see any disadvantages myself, but I'd be very interested to hear the developers' point of view. I know that Kate has declared the discussion closed, but I'm convinced that this protocol must have been considered already and rejected, and I'd like to understand why.
Good luck with that. TSSupport will tell us only what we need to know to operate their software. I really don't think they're interested in helping traders to learn how things work. Everything we know about synchronization right now we've had to find out on our own... aside from some very general and vague comments from support.

The thing about Kate closing the discussion is that she did that prior to a solution being proposed. So her continued and stubborn silence is really inappropriate. Who knows though, perhaps she's working under some higher order not to breach the subject of a solution...

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 24 Apr 2007

You did mention that a single sync point is already in use at resolutions above 1 day. I think we're starting to confuse ourselves though... we should consider the system as a whole... if MCFX uses one supra-daily sync point AND one daily session sync point then we have a system that uses two synchronization thresholds to synchronize charts - not a single sync point system as you proposed.
Let's try to drop the jargon, because I think it's leading to misunderstanding.

MCFX currently appears to use two completely separate methods of ensuring its time-based charts are synchronised with one another - one method for n-day charts, and one for n-hour, n-minute, n-second, n-tick. I exclude discussion of n-week, n-month etc. charts: they don't interest me for the moment.

In general, all that's necessary for time-based charts to be correctly synchronised - by which I mean that all bars of a given time scale open and close together, and all multiples and fractions of that time scale correspond as they should, is to choose one arbitrary moment in time, and ensure that all charts in all time scales break their bars together at that time. That moment could be 12 noon on Christmas day 1987, or it could be (more practically) NY close this Friday. Once that unique point of synchronisation is defined, simple arithmetic will take care of the rest.

For example, if I define my one fixed point as "now", then I can divide up the future and the past arbitrarily into equal parcels of time, backwards and forwards to infinity, and any data tick that will occur or has occurred inside any of these parcels contributes to the candle that corresponds to that parcel. As long as the criteria of the candles are respected, and assuming no cumulative errors in the data stream, the simple correspondence of arithmetic will ensure that synchronisation occurs between time scales that are co-factors, and no candlesticks are ever broken before their time. Two 3-hour candlesticks will make a 6-hour candlestick, and every time a 12-hour candlestick opens, a 6-hour, a 3-hour, a 90-minute, a 45-minute, a 15-minute, and a 5-minute candlestick will open. A 10-hour candlestick break and a 12-hour candlestick break will coincide every 60 hours, but when a 10-hour candlestick opens, a 5-hour candlestick will open at the same moment.

That's the method the n-day charts use. The arbitrary fixed point is the end of the current trading week. Creating an n-day chart causes time to be divided up into n-day parcels streaming back from that point. An m-day chart likewise, m-day parcels. n-day and m-day charts will correspond at any point at which n and m are both factors, i.e. multiples of their lowest common multiple. To use my example from the other day, 17 and 23-day bars will correspond naturally every 391 days - no more, no less.

The second method is used by all the sub-daily time scales. That method is to divide the trading day up according to the bar critieria, but any bar open at NY Close is broken at NY Close, and a new day (and hence a new bar) begins.

If n-day charts used this method, there could be no value of n greater than 1.

All I'm suggesting is that there need be no arbitrary candlestick breakage. If all time scales are managed as the n-day charts are currently managed, using one unique, arbitrary synchronisation point (the end of the current week, for example) those charts which correspond naturally (e.g. 1,2,4,8,12,24h charts) will continue to do so. Those which are currently useless due to broken candles (e.g. 10h, 16h) will correspond naturally to those other time scales with which they have common factors mathematically, with no broken candles.

There may be a need for error checking, depending on how the data stream is supplied, but that can be done invisibly, and the good accuracy of long n-day charts shows that MCFX is already capable of doing what's necessary.

Apologies if that all seems simplistic, but I just want to be sure we're talking about the same thing.

By the way, what is that supra-daily sync point and are you sure it exists? For instance, how can you be sure that for chart resolutions above 1 day MCFX doesn't just say something like "skip X number of daily sessions before closing the candle"? Plugging in 7 for X would produce a weekly chart.
Essentially that's exactly what does happen, and what should happen. What you call the "supra-daily sync point" is just the one arbitrary moment (the end of the current trading week) at which MCFX decides all candles should line up.

It's clear that such a point exists, because creating a chart that superimposes several different multiday charts (e.g. 13, 17, 23-day) which should only have rare correspondences, the bars are all aligned at the end of the current trading week.

So let me see if I understand this correctly...

1. there are no broken candle issues in resolutions above 1 day
Right, at least not in the n-day charts.

2. MCFX currently uses multiple sync points system-wide

True?
I don't know what you mean by that.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 24 Apr 2007

You've obviously done some deep thinking on MCFX synchronization strategy. I really appreciate your efforts to explain these things in detail (I thought that your explanations were excellent) and I'm happy to report that our efforts until now may actually pay off. Although interest in this issue by other traders in this forum has been about as inspiring as a death in the family, I have heard back from TSS privately to the effect that they will "reconsider this issue with session breaks" once their current development project is finished.

:o :) :D :D :D

I am sure they will use the threads you and I have worked on over the past two weeks or so (esp. this one) as a springboard for this reconsideration.
Let's try to drop the jargon, because I think it's leading to misunderstanding.
My apology for any misunderstanding, but in the absence of correct (or for that matter any) terminology for the things we're talking about (which could only be provided by MCFX developers) it is quite necessary that I develop some descriptive terms. Not only to aid my own thought processes but so that I have a means of refering to the concepts that underpin this subject and to promote efficient use of text (for instance, saying "supra-daily" is alot easier than saying "time scales of 1 day or above").

For future reference:

synchronization threshold = sync point = synchronization point = session break = session = (any point in time that MCFX uses to synchronize chart resolutions)

supra daily = (daily and above time scales) = n-day = multiday

sub daily = (time scales below (and not including) the daily level)

Allow me to attempt a rendition of the quote you lead off with in the previous post:

You proposed a system that synchronizes using a single point in time. A great idea I think. You claim that it isn't novel, but your description of current MCFX sync strategy is of a system that uses two points in time to achieve synchronization, a daily session and a weekly session. The daily break is used to manage resolutions below the daily level via division and the weekly break is used to manage resolutions of daily and above via division below the threshold and multiplication above it. Thinking of the system as a whole, that is a dual-sync-point system because it uses two thresholds for system-wide synchronization.

From what I can see, the only real difference between what you call "two completely separate methods" is that one uses division only, and of course, they use two seperate points in time. It's interesting that you've uncovered this dichotomy but the processes themselves seem to me to be of essentially the same nature.

Assuming my understanding of your explanation is correct, the crux of the matter is this: If charts of sub-daily resolutions can be managed by division based on a daily break, then they can also be managed by division based on a weekly break.
If all time scales are managed as the n-day charts are currently managed, using one unique, arbitrary synchronisation point (the end of the current week, for example) those charts which correspond naturally (e.g. 1,2,4,8,12,24h charts) will continue to do so. Those which are currently useless due to broken candles (e.g. 10h, 16h) will correspond naturally to those other time scales with which they have common factors mathematically, with no broken candles.
Are you sure there would be absolutely "no broken candles"? I was under the assumption that having a higher scale sync point would simply REDUCE the number of broken candles to a level that would present no practical obstacle to analysis.
In general, all that's necessary for time-based charts to be correctly synchronised... is to choose one arbitrary moment in time...
What is your opinion as to why MCFX uses dual-sync-points when only a single point in time is needed? Do they have a good reason for this or is it just an unnecessary appendage in an otherwise efficient system?

BTW, what do you mean by "m-day"? For reference here's the paragraph...
That's the method the n-day charts use. The arbitrary fixed point is the end of the current trading week. Creating an n-day chart causes time to be divided up into n-day parcels streaming back from that point. An m-day chart likewise, m-day parcels. n-day and m-day charts will correspond at any point at which n and m are both factors, i.e. multiples of their lowest common multiple. To use my example from the other day, 17 and 23-day bars will correspond naturally every 391 days - no more, no less.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 25 Apr 2007

Although interest in this issue by other traders in this forum has been about as inspiring as a death in the family, I have heard back from TSS privately to the effect that they will "reconsider this issue with session breaks" once their current development project is finished.
That is good news!

My apology for any misunderstanding, but in the absence of correct (or for that matter any) terminology for the things we're talking about (which could only be provided by MCFX developers) it is quite necessary that I develop some descriptive terms.
I agree, but in the absence of definitions, the risk is that we stray into using the same words to describe different things, as I think may have occurred here.

You proposed a system that synchronizes using a single point in time. A great idea I think. You claim that it isn't novel, but your description of current MCFX sync strategy is of a system that uses two points in time to achieve synchronization, a daily session and a weekly session. The daily break is used to manage resolutions below the daily level via division and the weekly break is used to manage resolutions of daily and above via division below the threshold and multiplication above it. Thinking of the system as a whole, that is a dual-sync-point system because it uses two thresholds for system-wide synchronization.

From what I can see, the only real difference between what you call "two completely separate methods" is that one uses division only, and of course, they use two seperate points in time. It's interesting that you've uncovered this dichotomy but the processes themselves seem to me to be of essentially the same nature.

Assuming my understanding of your explanation is correct, the crux of the matter is this: If charts of sub-daily resolutions can be managed by division based on a daily break, then they can also be managed by division based on a weekly break.
This makes me think we're still not quite on the same page.

The italicised part of your last paragraph is, of course, true, but I don't think that's the crux of the issue.

The fundamental difference between the two synchronisation methods is like the difference between a point and an interval. A point is unique, while an interval is a set of regularly spaced, recurring points. The sub-daily charts use a synchronisation interval, with the result that any bar that is not a simple fraction of that interval (1 day) is broken every time that interval occurs. The n-day charts use a unique point, with the result that no bars are ever broken.

Are you sure there would be absolutely "no broken candles"? I was under the assumption that having a higher scale sync point would simply REDUCE the number of broken candles to a level that would present no practical obstacle to analysis.
That would be the case for synchronising at some greater interval, but that's not what I'm suggesting, and it's not what happens in n-day charts. I'm talking about synchronising at a single, unique, arbitrary point in time. That point can be in the past or the future, and it occurs only once. Thus, by definition, there can be no broken candles.

What is your opinion as to why MCFX uses dual-sync-points when only a single point in time is needed? Do they have a good reason for this or is it just an unnecessary appendage in an otherwise efficient system?
I don't know, and I'd love to hear from the developers. My hunch, as I wrote before, is that this stems from MCFX being (I presume) an adaptation of MultiCharts, and MultiCharts having been developed for markets that have a clear session break every day. It might be useful for catching cumulative errors, but assuming each data point arrives with a time stamp (as I think it must) such errors should be impossible.

BTW, what do you mean by "m-day"?
By "m", I meant to imply an integer that isn't "n". So an n-day chart and an m-day chart will coincide regularly at periods defined by the common multiples of n and m.

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 25 Apr 2007

Here are a few screen grab examples:

1d3d13d17d23d.png shows (as the name implies), 1-day, 3-day, 13-day, 17-day, and 23-day charts on the same x-axis. You can see the one synchronisation point, which is (in this example) NY Close on the 24th. The bars only align at that one point, and where it makes mathematical sense for them to do so, i.e. where they have common multiples. As these are prime numbers, there are few common multiples, but there are no broken candles.

1d3d13d17d23d b.png is the same chart zoomed out. There are no broken candles.

1h2h4h8h16h.png shows 1-hour, 2-hour, 4-hour, 8-hour, 16-hour charts on the same x-axis. You can see how the daily session break chops up the 16-hour chart into alternating 16h and 8h bars.

1d2d4d8d16d.png is the n-day equivalent. Again, the one synchronisation point is last night's NY Close, all the bars are correctly aligned, and there are no broken candles.

So it seems I was mistaken about the choice of the arbitrary synchronisation point in n-day charts. When I was looking at this issue last week, the charts I examined were synchronised to the end of that week. However, it is arbitrary, and it's a single point, not an interval.
Attachments
MCFX Grabs 2.rar
(289.88 KiB) Downloaded 191 times

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 26 Apr 2007

What type of file is ".rar"? Can I open it with winzip? It might be easier if you just posted the graphics seperately as .gif or .jpg...

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 26 Apr 2007

What type of file is ".rar"? Can I open it with winzip?
.rar is a WinRAR archive, similar to a .zip file. I don't know whether or not WinZip will open .rar files, but WinRAR will open .zip files and .rar files.

http://www.win-rar.com/download.html
It might be easier if you just posted the graphics separately as .gif or .jpg...
Well, easier for you, perhaps!

Here are the first two...
Attachments
1d3d13d17d23d b.png
(82.77 KiB) Downloaded 2742 times
1d3d13d17d23d.png
(77.32 KiB) Downloaded 2728 times

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 26 Apr 2007

Here are the first two...
...and here are the last.
Attachments
1h2h4h8h16h.png
(79.11 KiB) Downloaded 2728 times
1d2d4d8d16d.png
(70.35 KiB) Downloaded 2726 times

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 26 Apr 2007

Well, I guess it wouldn't have mattered because my copy of Winzip has expired... Sorry, I thought you only had two files to post (didn't know you had four)...

Don't know of anyone who uses WinRAR. I'll check the site out though. If it's freeware and opens zip files, as you said, it would definitely be worth checking out. Anyway, thanks for the trouble!

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 27 Apr 2007

Question about your graphics:

Why do the gaps between candles increase with the resolution?

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 29 Apr 2007

Why do the gaps between candles increase with the resolution?
Because the charts are all drawn to the same horizontal scale, and MCFX aligns the subcharts by timestamp. As the candles are all drawn the same width, the longer time scales have wider gaps.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 03 May 2007

Because the charts are all drawn to the same horizontal scale, and MCFX aligns the subcharts by timestamp. As the candles are all drawn the same width, the longer time scales have wider gaps.
Okay, I see. Well, it would seem at present that, until TSS finds the time to deal with broken candles there are a few things that a trader who wants to avoid them could do...

1. If you use time-based charts then make sure you use hour charts that naturally align with the 24 hour session (3, 6, 12, etc...) as yosnappyj has previously pointed out.

2. If you use point-based charts as I do, then revert to trading at lower scales where broken candles are less common and therefore not a practical concern - I would recommend 25 point and lower. The same advice would apply to Change charts.

I recently got burned with an 85 pip loss because I based a trade on some faulty analysis in a 70 point chart that was litered with broken candles and spurious events. I hope TSS knows that the longer they take to act, the more we traders stand to lose...

User avatar
Kate
Posts: 758
Joined: 08 Dec 2006

Postby Kate » 04 May 2007

Terminal7 and Yosnappyj,

First of all, thank you for all the information you shared with us regarding this issue. Our developers have reconsidered the way we construct data series and as an alternative to the current synchronization mechanism we can consider implementing 168-hour (24/7) session breaks as you suggested but for all types of charts. We don't think that there is any logical ground to eliminate the synchronization protocol for count-based charts. Maybe, there will be an option to either plot 24-hour thresholds, or 24/7 ones.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 04 May 2007

Terminal7 and Yosnappyj,

First of all, thank you for all the information you shared with us regarding this issue. Our developers have reconsidered the way we construct data series and as an alternative to the current synchronization mechanism we can consider implementing 168-hour (24/7) session breaks as you suggested but for all types of charts. We don't think that there is any logical ground to eliminate the synchronization protocol for count-based charts. Maybe, there will be an option to either plot 24-hour thresholds, or 24/7 ones.
OMG! This is where I get overly emotional... Oh gosh! I thought you guys were gonna shoot us down like clay pigeons! I thought we had a strong argument but, judging by your initial reaction, I didn't expect anything to come of it... Whew! What a relief! Thank you soooo much for deciding to provide this alternative. You really don't know how much it means to my research and my trading. I really appreciate it!

Yosnappyj, what do you think! :shock: :D

Although I still can't see why Point (Count) charts need to be synchronized by TIME I am overjoyed that you will be implementing this across the board on all compressions - TIME, POINT, and CHANGE. The 168 hour session should definitely reduce the number of broken candles to a tolerable level, to a level that doesn't interfere with analysis. This will give traders who use Point compressed charts the freedom to roam into higher resolutions without worrying about the market picture being increasingly distorted. And traders who use Time compressed charts can now rest assured that they will be looking at charts that are more faithful to the stated resolution, more pure, and more accurate.

I really congratulate the development team for making an intelligent and reasonable decision on this issue and for really caring about the quality of MCFX and about giving their clients a more accurate picture of the market. Needless to say I'm very impressed.

Any idea, roughly, when a Beta with the new 168 hour threshold option might be available? Are we looking at a major overhaul or just a small modification of the current version?

Thanks again! Thank you so much, really...

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 05 May 2007

Terminal7 and Yosnappyj,

First of all, thank you for all the information you shared with us regarding this issue
Kate,

Kinda got carried away in the moment on the post above but just wanted to add a personal thanks to you for letting the developers know about the issue and being our mediatrix of sorts. You're welcome for the information and, you may think I'm exaggerating, but the entire FX trading community will be in your debt for this...

Erek Allon Daniels
http://www.reactionaryeventmodel.info

yosnappyj
Posts: 112
Joined: 14 Feb 2007

Postby yosnappyj » 08 May 2007

Our developers have reconsidered the way we construct data series and as an alternative to the current synchronization mechanism we can consider implementing 168-hour (24/7) session breaks as you suggested but for all types of charts.
That's excellent news, and a real improvement over the current situation. It's heartening to know that you're listening to your customers.

When you say "for all types of charts", I assume you're excluding n-day charts which currently employ a superior protocol.
We don't think that there is any logical ground to eliminate the synchronization protocol for count-based charts. Maybe, there will be an option to either plot 24-hour thresholds, or 24/7 ones.
I'm still not sure that I understand your reasons for breaking bars in any charts when an alternative exists (synchronising to a single point). In the medium term, I hope you'll consider applying the current n-day protocol to all charts, ideally with a user-defined synchronisation point.

Such a change would not only eliminate all broken bars from all charts, but would also have the advantage of giving the user control over the start time of bars - for example 24h bars that begin at London Open, or 60-minute bars that span half-hour to half-hour periods. This would be a very desirable feature.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 08 May 2007

When you say "for all types of charts", I assume you're excluding n-day charts which currently employ a superior protocol.
I think what Kate meant there was 'all compressions' - TIME, PRICE, and CHANGE charts. If you recall I had been encouraging them to totally eliminate the session break in PRICE compressed charts (or what she calls "count-compressed" charts in the quote below)...
We don't think that there is any logical ground to eliminate the synchronization protocol for count-based charts.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 12 May 2007

Kate. It's been a week and you haven't responded to my question as quoted below...
Any idea, roughly, when a Beta with the new 168 hour threshold option might be available? Are we looking at a major overhaul or just a small modification of the current version?
Is Dev considering alternatives to the 168 hr. threshold (such as single, non-reaccuring sync point (as mentioned by yosnappyj)) or is it that you just don't know yet when a new beta will be available?

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 18 May 2007

Kate. Please reply...
Any idea, roughly, when a Beta with the new 168 hour threshold option might be available?
Is Dev considering alternatives to the 168 hr. threshold?
Please keep us updated. It's been two weeks since we've heard anything from you on this subject.

T7

User avatar
Kate
Posts: 758
Joined: 08 Dec 2006

Postby Kate » 21 May 2007

Erek,

Our developers said that they will implement an option that will allow our customers to use either 168-hour or 24-hour threshold but they are not planning to eliminate session breaks at all, even for count-based charts. As for the new beta, as far as I know this improvement won't be included in the next Vista compatible version that is planned to be released in two weeks.

User avatar
terminal7
Posts: 143
Joined: 14 Dec 2006
Location: Ohio, U.S.
Contact:

Postby terminal7 » 21 May 2007

Erek,

Our developers said that they will implement an option that will allow our customers to use either 168-hour or 24-hour threshold... As for the new beta, as far as I know this improvement won't be included in the next Vista compatible version that is planned to be released in two weeks.
Okay. Thank you. I'll look forward to seeing that option in the release after the next release...


Return to “MCFX”