Is this correct behavior for "Build From Minutes" bars?  [SOLVED]

Questions about MultiCharts and user contributed studies.
StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 16 Jan 2014

The date is New Years Day 2014 (the same issue happened on Thanksgiving day 2013)

On the Left is a daily chart with "Build from Minute" set to OFF. On the Right is the same chart with "Build from Minute" set to ON.

The Right chart shows a bar on 1/1/2014, the left does not. Also, the close and Open on the previous bars to not match either.


Image
Attachments
Discrepancy.gif
(70.5 KiB) Downloaded 1292 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby TJ » 16 Jan 2014

What are your session times?

Please go to QuoteManager,
look under

Tools > Session Templates

as well as

Instrument > Edit Instruments > Sessions

also from your chart, look under

Format Instrument > Chart Settings > Sessions

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 16 Jan 2014

Below are my settings. The thing is, both ES charts are identical except for "Build From Minutes" is selected on one.

Tools > Session Templates:
Image

Instrument > Edit Instruments > Sessions:
Image

Format Instrument > Chart Settings > Sessions: Session is "Default"
Attachments
settings.gif
(53.91 KiB) Downloaded 1237 times
Instrument_session.gif
(118.47 KiB) Downloaded 1264 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby Henry MultiСharts » 17 Jan 2014

Hello StefanoSF,

Do you use "Local" or "Exchange" time zone on your chart (Format->Instrument->Settings tab->Time Zone)?

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 20 Jan 2014

I tend to use "Local", so I switched to "Exchange" which still produces the lonely bar. Can the issue be that New Years Day and Thanksgiving are half days?

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 23 Jan 2014

I just ran into this issue again. Attached is a workspace with two identical Daily charts. One uses "Build on Minutes" the other does not. There are huge bar discrepancies.

I have marked discrepancy bars and areas of interest with while lines.
Chart settings:, Time Zone= Local, Session=Default, Quote Field=Trade

Image
Attachments
Build on Minutes Workspace-- On-Off.zip
(31.97 KiB) Downloaded 157 times
Build_on_Minutes.gif
(67.73 KiB) Downloaded 1208 times

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby arnie » 24 Jan 2014

Please create your own session templates and do not the default ones that comes with MC.

I can't understand the session times in that template.

What is your goal?
RTH (pit) session only, ETH (globex) session only, full 24h session or do you just want from the globex start till the end of the pit session, dismissing this way the last 45 min of the day?

if you want 24h session than you should have something like:
Sunday 1700, Monday 1615
Monday 1700, Tuesday 1615
... and so forth (I'm using exhange time)

if you want to dismiss the last 45 min than:
Sunday 1700, Monday 1515
Monday 1700, Tuesday 1515
... and so forth

if you want globex only than:
Sunday 1700, Monday 830
Monday 1700, Tuesday 830
... and so forth

if you want pit only than:
Monday 830, Monday 1515
Tuesday 830, Tuesday 1515
...and so forth


For the SPY, I actually never had any issues with the default template but please create your own also.

Monday 930, Monday 1600
Tuesday 930, Tuesday 1600
... and so forth (again, exchange time)

Just to confirm, SPY trades in NY so it's eastern time, right?

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 24 Jan 2014

For simplicity, lets focus on SPY. Below are all my settings. I tend to use the broker defualts for 95% of all my symbols as most users do.

I have attached a Spy workspace. Someone please download it and see if they get data discrepancies. My data feed is IQ but I tested with IB as well and had similar results. Time Zone is Exchange. Date range is Jan 2014.

I need to resolve this ASAP.

Image

Image
Attachments
Screenshot.GIF
(63.76 KiB) Downloaded 1189 times
Spy Workspace zipf.zip
(32.73 KiB) Downloaded 158 times
SPY Settings.gif
(156.1 KiB) Downloaded 1174 times

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby arnie » 24 Jan 2014

Have you compared a 5 minute chart against the daily chart and the daily charts minute based?
You will see that the 5 minute chart is identical to the daily minute based chart.

The "problem" is in fact with the daily bars.
If I'm not mistaken, daily bars are based on the data exchanges release and not based on minute data themselves, hence the discrepancies.

Exchanges do filter the data and the final settlement price rarely is the same we see on the minute data.

I'm sure IQFeed guys will explain this much better than I.

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 28 Jan 2014

Exchanges do filter the data and the final settlement price rarely is the same we see on the minute data.
Yes, but a 30 cent difference on the low of the day on Jan 15, 2014? How can anyone backtest on daily data with this type of discrepancy. I'm finding these discrepancies on many daily bars using both IQ Feed and IB as data sources.

If no one else is seeing this than the issue has to be local on my machine.

Image
Attachments
Discrepancy Jan 15 2014.gif
(76.18 KiB) Downloaded 1162 times

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby arnie » 28 Jan 2014

Exchanges do filter the data and the final settlement price rarely is the same we see on the minute data.
Yes, but a 30 cent difference on the low of the day on Jan 15, 2014? How can anyone backtest on daily data with this type of discrepancy. I'm finding these discrepancies on many daily bars using both IQ Feed and IB as data sources.

If no one else is seeing this than the issue has to be local on my machine.
I'm seeing exactly what you're seeing.

On futures data sometimes the differences between high's and low's might have to do with which template we might be using. Exchanges use 24h only if I'm not mistaken. Also, on expiration week there might be also discrepancies regarding the contract change.
Holidays and half days can also generate differences between intra and daily.

By the way, can you find those wide discrepancies on stocks too?

ETF's were always wild.
Do you know if exchanges use any pre or post market quotes?

Do a test.
A regular daily chart and then a minute based chart BUT using the _US Stocks session templates. There's one with pre, another with post and another with both.
Compare them with the daily data. Although differences continues to exist, it seems the exchange is using the pre and post market data in some weird way.

Image

Have you talk with IQFeed support yet?
Attachments
spy.jpg
(272.15 KiB) Downloaded 1174 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Is this correct behavior for "Build From Minutes" bars?

Postby TJ » 28 Jan 2014

Yes, but a 30 cent difference on the low of the day on Jan 15, 2014? How can anyone backtest on daily data with this type of discrepancy. I'm finding these discrepancies on many daily bars using both IQ Feed and IB as data sources.
If no one else is seeing this than the issue has to be local on my machine.
From your observation, do you think this is a datafeed problem? or MultiCharts problem?

StefanoSF
Posts: 116
Joined: 19 Aug 2012
Location: San Francisco
Has thanked: 23 times
Been thanked: 30 times
Contact:

Re: Is this correct behavior for "Build From Minutes" bars?

Postby StefanoSF » 28 Jan 2014

Arnie, TJ, thanks for taking the time to address this. The daily bar data discrepancy issue doesn't seem to be a MC issue but rather a daily data issue. MC's new feature "Build From Minutes" on daily bars simply highlights the discrepancies in Daily vs Minute data.

The thing we need to be aware of is that if we build systems on historical daily bar data, that the data may not match with live minute data moving forward. As in the SPY example above, the Daily low for Jan 15, 2014 was 183.71 whereas the daily low on a daily "Build from Minute" chart for the same day is only 184.01. A difference of 30 points. That's quite large in this business and can easily be the reason a system fails moving forward in real-time.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Is this correct behavior for "Build From Minutes" bars?  [SOLVED]

Postby TJ » 28 Jan 2014

Arnie, TJ, thanks for taking the time to address this. The daily bar data discrepancy issue doesn't seem to be a MC issue but rather a daily data issue. MC's new feature "Build From Minutes" on daily bars simply highlights the discrepancies in Daily vs Minute data.
The thing we need to be aware of is that if we build systems on historical daily bar data, that the data may not match with live minute data moving forward. As in the SPY example above, the Daily low for Jan 15, 2014 was 183.71 whereas the daily low on a daily "Build from Minute" chart for the same day is only 184.01. A difference of 30 points. That's quite large in this business and can easily be the reason a system fails moving forward in real-time.
Bear in mind that SPY is notorious for bad ticks from the exchange. I would not use a fine resolution system to autotrade or backtest SPY.

For your "Build From Minutes" exploration, I would use a more stable symbol for comparison.


Return to “MultiCharts”