Hi everybody,
I feed MC12 64 with IB data (TWS API).
Most of the time they work together like a breeze and without any hassle.
Sometimes, with no apparent reason, MC "get" some strange data ticks that are well below (or above) current prices.... . The same "strange" ticks are not present in TWS own chart (Please see jpeg, for AMC today berween 13:20 - 13:40 NY time). It's annoying because - with automatic trading - one risks to activate STOP signasl triggered by prices that don't really exist and/or should be filtered out.
I tried to contact IB support, bit they had nothing useful to say about it.
Has anyone faced the same issue and find a solution?
Thanks for your time.
Fake ticks form IB TWS
Fake ticks form IB TWS
- Attachments
-
- AMC from MC12.jpg
- (128.12 KiB) Not downloaded yet
-
- AMC from Mosaic 2021 06 07.doc
- (120 KiB) Downloaded 163 times
- rrams
- Posts: 129
- Joined: 10 Feb 2011
- Location: USA
- Has thanked: 7 times
- Been thanked: 70 times
- Contact:
Re: Fake ticks form IB TWS
Hello Ima,
Yes, everybody who uses IB for realtime data will eventually encounter that same situation. Historical data from them is less terrible.
Yes, there are ways to mitigate the problem:
1. Use a better data provider (IQFeed).
2. Trade a higher timeframe (daily).
3. Trade a different instrument (with high liquidity and low volitility like ZB futures).
4. Use Bid and Ask to determine signal triggers (load in Data2 and Data3).
5. Attempt to filter bad realtime data (e.g. by using CommandLine(".rld int 2 mins"); whenever prices fall outside a standard deviation).
You have to choose which is right for you.
Yes, everybody who uses IB for realtime data will eventually encounter that same situation. Historical data from them is less terrible.
Yes, there are ways to mitigate the problem:
1. Use a better data provider (IQFeed).
2. Trade a higher timeframe (daily).
3. Trade a different instrument (with high liquidity and low volitility like ZB futures).
4. Use Bid and Ask to determine signal triggers (load in Data2 and Data3).
5. Attempt to filter bad realtime data (e.g. by using CommandLine(".rld int 2 mins"); whenever prices fall outside a standard deviation).
You have to choose which is right for you.
Re: Fake ticks form IB TWS
Hello rrams,5. Attempt to filter bad realtime data (e.g. by using CommandLine(".rld int 2 mins"); whenever prices fall outside a standard deviation).
thanks for your reply.
Could you (or someone else in the know) please elaborate a bit (or re-direct me if the issue has already been discussed) on the "How to" of point 5?
I think
- the filter should be based on volume, these "bad ticks" have always have heavy volume (see jpeg).
I've been told by IB support they are "block trades" supposedly concluded at earlier time (hence, at "older" prices) that hit the tape at the moment the counterparties tells the exchange about them ..... and that they (IB) cannot do nothing to exclude them (which I translate as "we don't want to exclude them...").
- the issue has nothing to do with liquidity, yesterday AMC was one of the the more liquid US stocks, and it happens the same with the "FTSEMIB" future (the big one, not the mini), which is the stock index future of the Italian Stock market which has no liquidity holes, aside from "flash crashes...." (well of course "liquidity" is a relative concept..... but you know what I mean...).
- Attachments
-
- AMC _Block trades.jpg
- (308.3 KiB) Not downloaded yet
- rrams
- Posts: 129
- Joined: 10 Feb 2011
- Location: USA
- Has thanked: 7 times
- Been thanked: 70 times
- Contact:
Re: Fake ticks form IB TWS
You're welcome Ima. Hopefully I can clarify what I meant by switching to a more liquid instrument. I am not referring to the total trading volume.
Imagine hypothetically, a world where a broker gets paid to provide good fills to dark pools which hide orders and also that broker provides realtime quote data that can be manipulated because only the adjusted historical data is recorded. If that broker wanted to run stops to generate market orders at the real price, it would still cost them an amount close to the number of orders on the book between the two prices. ZB has a high nominal value and a deep book of orders at all times, so it costs many millions of dollars to move the price of ZB 2%. You can probably move AMC 2% with just thousands of dollars at some periods during the day. I don't know about FTSEMIB, but I just know I see many more IB funny candles on stocks like AMC and futures like AW than on slow moving bohemoths.
If you are trading one minute bars of AMC, then you are a big boy professional and need a good data provider. Solution #5 would be too slow and not practical for you until MC incorporates native bar filtering. I use something like it on a four hour chart of natural gas, though.
Good trading and thank you for sharing your experiences.
Imagine hypothetically, a world where a broker gets paid to provide good fills to dark pools which hide orders and also that broker provides realtime quote data that can be manipulated because only the adjusted historical data is recorded. If that broker wanted to run stops to generate market orders at the real price, it would still cost them an amount close to the number of orders on the book between the two prices. ZB has a high nominal value and a deep book of orders at all times, so it costs many millions of dollars to move the price of ZB 2%. You can probably move AMC 2% with just thousands of dollars at some periods during the day. I don't know about FTSEMIB, but I just know I see many more IB funny candles on stocks like AMC and futures like AW than on slow moving bohemoths.
If you are trading one minute bars of AMC, then you are a big boy professional and need a good data provider. Solution #5 would be too slow and not practical for you until MC incorporates native bar filtering. I use something like it on a four hour chart of natural gas, though.
Good trading and thank you for sharing your experiences.
Re: Fake ticks form IB TWS
Just to put thing into the right perspective: I showed the problem using 1 minute chart, because the granularity helps in the understanding of the problem (tick chart is even more useful................), but one gets the same effect in all timeframes, eg houly... 4 hour even daily timeframe, sometimes.If you are trading one minute bars of AMC, then you are a big boy professional and need a good data provider. Solution #5 would be too slow and not practical for you until MC incorporates native bar filtering. I use something like it on a four hour chart of natural gas, though.
This problem aside, IB data are perfect for my need (I also think the need of a "good data provider" is higly overrated by retail traders, no matter how big or small we are..... as we are always awfully late compared to the really big fish .... but that's another matter )
The point is.... given that IB isn't willing to filter his data, if someone has a way to filter data INSIDE MC I'd be delighted to know.............. no matter how slow is....
Have a good day!
Re: Fake ticks form IB TWS
Just another example from some instants ago.
1 min chart vs 15 min chart
1 min chart is simply more evident, bit the issue is exactly the same on 15 min ..... 30 min .... hourly....
1 min chart vs 15 min chart
1 min chart is simply more evident, bit the issue is exactly the same on 15 min ..... 30 min .... hourly....
- Attachments
-
- 2021 06 08_AMC 15 min.jpg
- (196.18 KiB) Not downloaded yet
-
- 2021 06 08_AMC 1 min.jpg
- (175.94 KiB) Not downloaded yet
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
Hello Ima,
Please send a screenshot of your IB settings (QuoteManager - Tools - Data Sources - IB - Settings).
Please send a screenshot of your IB settings (QuoteManager - Tools - Data Sources - IB - Settings).
Re: Fake ticks form IB TWS
Hello Svetlana, here it is (should be the default options, I haven't changed anything).Hello Ima,
Please send a screenshot of your IB settings (QuoteManager - Tools - Data Sources - IB - Settings).
Thanks for your time!
- Attachments
-
- IB quote settings.jpg
- (79.39 KiB) Not downloaded yet
Re: Fake ticks form IB TWS
Hello Ima,
Does it help to do via the untick of "Include Unreportable Last Trades" in IB data feed config ?
Also,
did the off-price trade exist in quotemanger / the symbol row / editData ?
Then, if it exists, does the off-price trade disappear after reloading of the symbol via mc reload ?
Regards
Does it help to do via the untick of "Include Unreportable Last Trades" in IB data feed config ?
Also,
did the off-price trade exist in quotemanger / the symbol row / editData ?
Then, if it exists, does the off-price trade disappear after reloading of the symbol via mc reload ?
Regards
Re: Fake ticks form IB TWS
Don't know, quite frankly I was waiting whether Svetlana (or someone else in the MC team) could have a look at my settings, checking if there's something to change.Hello Ima,
Does it help to do via the untick of "Include Unreportable Last Trades" in IB data feed config ?
From here:
https://www.multicharts.com/trading-sof ... ew_desktop
I read
"Check this option for the small size transactions (less than 1 lot) to be included."
which - frankly - is not very meaningful to me.
yes, I posted some jpeg abovedid the off-price trade exist in quotemanger / the symbol row / editData ?
That's the weirdest thing at all... sometimes yes sometimes no.Then, if it exists, does the off-price trade disappear after reloading of the symbol via mc reload ?
I guess (but that's just my guess..) it dipends ......
1) if it's a BLOCK trade it does NOT disappear reloading the symbol.
2) But when it's an error..... afert reloading could disappear, supposing IB dartabese has found it and corrected their database.
Of couese, here I'm not asking MC to face problem 2..... as it is 100% IB responsability
I'm asking if there's a way to face problem 1 (that is, filter block trades from MC), and - if I can talk straight - I find a bit strange to be alone dealing with this matter (given IB is a broker very popular among traders ).
Of course, there are workarounds, ad another user suggested, but - IMO - none of them is good enough.
The more simple one's, using bid-ask data, is fine when markets are open, but gives "not what you are expecting" data during opening and closing auctions.... (you get an highest bid ad lowest ask, which most often are out of scale with the final auction price)
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
Please uncheck Include "Unreportable Last" Trades.
Restart all MultiCharts components to apply the changes.
If it does not help, uncheck Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update too.
Restart all MultiCharts components to apply the changes.
If it does not help, uncheck Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update too.
- lingwuchung
- Posts: 50
- Joined: 03 Feb 2014
- Has thanked: 6 times
Re: Fake ticks form IB TWS
I used to have the same problem. I received similar advice from previous post and I think I don't see the problem anymore. However, this raises two questions:Please uncheck Include "Unreportable Last" Trades.
Restart all MultiCharts components to apply the changes.
If it does not help, uncheck Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update too.
1. What does "unreportable" mean? How MC could tell from the data?
2. I suppose IB sometimes does provide zero volume data, which is faulty.
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
Include "Unreportable Last" Trades.
If this option is checked, the small size transactions (less than 1 lot) are included.
Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update.
If the discrepancy between DailyHigh (DailyLow) and IB real time data is detected, this option allows for the generation of a new tick with the required price and zero volume to eliminate the discrepancy. When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible.
If this option is checked, the small size transactions (less than 1 lot) are included.
Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update.
If the discrepancy between DailyHigh (DailyLow) and IB real time data is detected, this option allows for the generation of a new tick with the required price and zero volume to eliminate the discrepancy. When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible.
- lingwuchung
- Posts: 50
- Joined: 03 Feb 2014
- Has thanked: 6 times
Re: Fake ticks form IB TWS
Hi Svetlana,
"When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible." -> so the resolution is to make "the discrepancy possible" rather than "impossible"?
Why would IB need to send such tick to align the DailyHigh? To rectify their previous data issue (much less frequently in other datafeed providers), or just in case MC missed the DailyHigh (from individual bars?) collected so far?
"When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible." -> so the resolution is to make "the discrepancy possible" rather than "impossible"?
Why would IB need to send such tick to align the DailyHigh? To rectify their previous data issue (much less frequently in other datafeed providers), or just in case MC missed the DailyHigh (from individual bars?) collected so far?
Include "Unreportable Last" Trades.
If this option is checked, the small size transactions (less than 1 lot) are included.
Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update.
If the discrepancy between DailyHigh (DailyLow) and IB real time data is detected, this option allows for the generation of a new tick with the required price and zero volume to eliminate the discrepancy. When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible.
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
Hi lingwuchung,
IB may send such updates as shown in the screenshot. According to IB support this is the high of the day due to the ticket type (6, see here).
IB may send such updates as shown in the screenshot. According to IB support this is the high of the day due to the ticket type (6, see here).
- lingwuchung
- Posts: 50
- Joined: 03 Feb 2014
- Has thanked: 6 times
Re: Fake ticks form IB TWS
Thanks Svetlana. If this is the case, I think MC should uncheck this setting by default for IB data feed.
Hi lingwuchung,
IB may send such updates as shown in the screenshot.phantom tick.jpg
According to IB support this is the high of the day due to the ticket type (6, see here).
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
"Generate a new tick with zero volume upon DailyHigh (DailyLow) update" is turned OFF by default since MultiCharts 14 Release 2 (if MC has been installed on a new computer or IB settings have been reset).
Re: Fake ticks form IB TWS
Have unticked the check box "Generate a new tick with zero volume upon DailyHigh (DailyLow) update" ,
I still see the block trade which is at price-way-off the current bid ask level.
When look in the quote manager,
it is seen as one trade with volume of 200 lot at price 65.67 which is far way-off below the nearby trade price at 73.65.
When look into the corresponding bid and ask ,
there is no bid-ask tick at the 65.67 price level at all.
While on tws, we don't see trade price at 65.67 price level at all.
My issue is that this block trade volume is not having to do with what retail traders can trade with,
but it affects badly to the calculation of strategy signal which bases on trade price.
Is there is a way to exclude these block trade by mc pre-sending this to mc user ?
May be if price is off by > some standard deviation amount of current price and the price is no near to the current bid ask at all,
then we have the option in mc check box to not receive this kind of block trade ticks at all.
Or may be there is a way to segregate this kind of block trades within the api sent from TWS ?
Again so we have the option to switch this off and not seen this price-way-off block trade at all.
so our strategy signal calculation is not affected.
I still see the block trade which is at price-way-off the current bid ask level.
When look in the quote manager,
it is seen as one trade with volume of 200 lot at price 65.67 which is far way-off below the nearby trade price at 73.65.
When look into the corresponding bid and ask ,
there is no bid-ask tick at the 65.67 price level at all.
While on tws, we don't see trade price at 65.67 price level at all.
My issue is that this block trade volume is not having to do with what retail traders can trade with,
but it affects badly to the calculation of strategy signal which bases on trade price.
Is there is a way to exclude these block trade by mc pre-sending this to mc user ?
May be if price is off by > some standard deviation amount of current price and the price is no near to the current bid ask at all,
then we have the option in mc check box to not receive this kind of block trade ticks at all.
Or may be there is a way to segregate this kind of block trades within the api sent from TWS ?
Again so we have the option to switch this off and not seen this price-way-off block trade at all.
so our strategy signal calculation is not affected.
- Svetlana MultiCharts
- Posts: 645
- Joined: 19 Oct 2017
- Has thanked: 3 times
- Been thanked: 163 times
Re: Fake ticks form IB TWS
Hello Meecc,
Please turn OFF Include "Unreportable Last" Trades in IB data feed settings and restart all MultiCharts components. See screenshot. Here is a comment from IB API support:
Please note that historical data at IB is filtered for trade types which occur away from the NBBO such as combo legs, block trades, and derivative trades. You may read more about this on the below link.
Please turn OFF Include "Unreportable Last" Trades in IB data feed settings and restart all MultiCharts components. See screenshot. Here is a comment from IB API support:
Please note that historical data at IB is filtered for trade types which occur away from the NBBO such as combo legs, block trades, and derivative trades. You may read more about this on the below link.
Re: Fake ticks form IB TWS
Thank you Svetlana,
With
May be this should also be default setting ,
and with documentation that it also
solves the block trades inclusion issue, which would not be known by the name and explanation of the check box.
With best regards,
With
, it has worked well for the past 3 days.turn OFF Include "Unreportable Last" Trades in IB data feed settings
May be this should also be default setting ,
and with documentation that it also
solves the block trades inclusion issue, which would not be known by the name and explanation of the check box.
With best regards,
- lingwuchung
- Posts: 50
- Joined: 03 Feb 2014
- Has thanked: 6 times
Re: Fake ticks form IB TWS
He did say it is default since v14 release 2 in his 25Jun post.Thank you Svetlana,
With, it has worked well for the past 3 days.turn OFF Include "Unreportable Last" Trades in IB data feed settings
May be this should also be default setting ,
and with documentation that it also
solves the block trades inclusion issue, which would not be known by the name and explanation of the check box.
With best regards,
Re: Fake ticks form IB TWS
There are 2 check boxes .
1) Default-Off was said for "Generate a new tick with zero volume"
2) was not said as Default-Off for "Include Unreportable Last Trades"
A week has past, with all 2 boxes unchecked, still not seen block trades. it's ok.
Best regards,
1) Default-Off was said for "Generate a new tick with zero volume"
"Generate a new tick with zero volume upon DailyHigh (DailyLow) update" is turned OFF by default since MultiCharts 14 Release 2 (if MC has been installed on a new computer or IB settings have been reset).
2) was not said as Default-Off for "Include Unreportable Last Trades"
A week has past, with all 2 boxes unchecked, still not seen block trades. it's ok.
Best regards,