Fake ticks form IB TWS

Ima · Postby **Ima** » 07 Jun 2021

Hi everybody,

I feed MC12 64 with IB data (TWS API).

Most of the time they work together like a breeze and without any hassle.

Sometimes, with no apparent reason, MC "get" some strange data ticks that are well below (or above) current prices.... . The same "strange" ticks are not present in TWS own chart (Please see jpeg, for AMC today berween 13:20 - 13:40 NY time). It's annoying because - with automatic trading - one risks to activate STOP signasl triggered by prices that don't really exist and/or should be filtered out.

I tried to contact IB support, bit they had nothing useful to say about it.

Has anyone faced the same issue and find a solution?

Thanks for your time.

rrams · Postby **rrams** » 07 Jun 2021

Hello Ima,
Yes, everybody who uses IB for realtime data will eventually encounter that same situation. Historical data from them is less terrible.
Yes, there are ways to mitigate the problem:
1. Use a better data provider (IQFeed).
2. Trade a higher timeframe (daily).
3. Trade a different instrument (with high liquidity and low volitility like ZB futures).
4. Use Bid and Ask to determine signal triggers (load in Data2 and Data3).
5. Attempt to filter bad realtime data (e.g. by using CommandLine(".rld int 2 mins"); whenever prices fall outside a standard deviation).
You have to choose which is right for you.

Ima · Postby **Ima** » 08 Jun 2021

5. Attempt to filter bad realtime data (e.g. by using CommandLine(".rld int 2 mins"); whenever prices fall outside a standard deviation).

Hello rrams,

thanks for your reply.

Could you (or someone else in the know) please elaborate a bit (or re-direct me if the issue has already been discussed) on the "How to" of point 5?

I think

- the filter should be based on volume, these "bad ticks" have always have heavy volume (see jpeg).
I've been told by IB support they are "block trades" supposedly concluded at earlier time (hence, at "older" prices) that hit the tape at the moment the counterparties tells the exchange about them ..... and that they (IB) cannot do nothing to exclude them (which I translate as "we don't want to exclude them...").

- the issue has nothing to do with liquidity, yesterday AMC was one of the the more liquid US stocks, and it happens the same with the "FTSEMIB" future (the big one, not the mini), which is the stock index future of the Italian Stock market which has no liquidity holes, aside from "flash crashes...."

(well of course "liquidity" is a relative concept..... but you know what I mean...).

rrams · Postby **rrams** » 08 Jun 2021

You're welcome Ima. Hopefully I can clarify what I meant by switching to a more liquid instrument. I am not referring to the total trading volume.

Imagine hypothetically, a world where a broker gets paid to provide good fills to dark pools which hide orders and also that broker provides realtime quote data that can be manipulated because only the adjusted historical data is recorded. If that broker wanted to run stops to generate market orders at the real price, it would still cost them an amount close to the number of orders on the book between the two prices. ZB has a high nominal value and a deep book of orders at all times, so it costs many millions of dollars to move the price of ZB 2%. You can probably move AMC 2% with just thousands of dollars at some periods during the day. I don't know about FTSEMIB, but I just know I see many more IB funny candles on stocks like AMC and futures like AW than on slow moving bohemoths.

If you are trading one minute bars of AMC, then you are a big boy professional and need a good data provider. Solution #5 would be too slow and not practical for you until MC incorporates native bar filtering. I use something like it on a four hour chart of natural gas, though.

Good trading and thank you for sharing your experiences.

Ima · Postby **Ima** » 08 Jun 2021

If you are trading one minute bars of AMC, then you are a big boy professional and need a good data provider. Solution #5 would be too slow and not practical for you until MC incorporates native bar filtering. I use something like it on a four hour chart of natural gas, though.

Just to put thing into the right perspective: I showed the problem using 1 minute chart, because the granularity helps in the understanding of the problem (tick chart is even more useful................), but one gets the same effect in all timeframes, eg houly... 4 hour even daily timeframe, sometimes.

This problem aside, IB data are perfect for my need (I also think the need of a "good data provider" is higly overrated by retail traders, no matter how big or small we are..... as we are always awfully late compared to the really big fish .... but that's another matter )

The point is.... given that IB isn't willing to filter his data, if someone has a way to filter data INSIDE MC I'd be delighted to know.............. no matter how slow is....

Have a good day!

Ima · Postby **Ima** » 08 Jun 2021

Just another example from some instants ago.

1 min chart vs 15 min chart

1 min chart is simply more evident, bit the issue is exactly the same on 15 min ..... 30 min .... hourly....

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 11 Jun 2021

Hello Ima,

Please send a screenshot of your IB settings (QuoteManager - Tools - Data Sources - IB - Settings).

Ima · Postby **Ima** » 11 Jun 2021

Hello Ima,

Please send a screenshot of your IB settings (QuoteManager - Tools - Data Sources - IB - Settings).

Hello Svetlana, here it is (should be the default options, I haven't changed anything).
Thanks for your time!

Meecc · Postby **Meecc** » 20 Jun 2021

Hello Ima,

Does it help to do via the untick of "Include Unreportable Last Trades" in IB data feed config ?

Also,
did the off-price trade exist in quotemanger / the symbol row / editData ?
Then, if it exists, does the off-price trade disappear after reloading of the symbol via mc reload ?

Regards

Ima · Postby **Ima** » 20 Jun 2021

Hello Ima,
Does it help to do via the untick of "Include Unreportable Last Trades" in IB data feed config ?

Don't know, quite frankly I was waiting whether Svetlana (or someone else in the MC team) could have a look at my settings, checking if there's something to change.

From here:
https://www.multicharts.com/trading-sof ... ew_desktop
I read
"Check this option for the small size transactions (less than 1 lot) to be included."

which - frankly - is not very meaningful to me.

did the off-price trade exist in quotemanger / the symbol row / editData ?

yes, I posted some jpeg above

Then, if it exists, does the off-price trade disappear after reloading of the symbol via mc reload ?

That's the weirdest thing at all... sometimes yes sometimes no.

I guess (but that's just my guess..) it dipends ......

1) if it's a BLOCK trade it does NOT disappear reloading the symbol.
2) But when it's an error..... afert reloading could disappear, supposing IB dartabese has found it and corrected their database.

Of couese, here I'm not asking MC to face problem 2..... as it is 100% IB responsability

I'm asking if there's a way to face problem 1 (that is, filter block trades from MC), and - if I can talk straight - I find a bit strange to be alone dealing with this matter (given IB is a broker very popular among traders ).

Of course, there are workarounds, ad another user suggested, but - IMO - none of them is good enough.
The more simple one's, using bid-ask data, is fine when markets are open, but gives "not what you are expecting" data during opening and closing auctions.... (you get an highest bid ad lowest ask, which most often are out of scale with the final auction price)

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 21 Jun 2021

Please uncheck Include "Unreportable Last" Trades.
Restart all MultiCharts components to apply the changes.

If it does not help, uncheck Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update too.

lingwuchung · Postby **lingwuchung** » 23 Jun 2021

Please uncheck Include "Unreportable Last" Trades.
Restart all MultiCharts components to apply the changes.

If it does not help, uncheck Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update too.

I used to have the same problem. I received similar advice from previous post and I think I don't see the problem anymore. However, this raises two questions:

1. What does "unreportable" mean? How MC could tell from the data?
2. I suppose IB sometimes does provide zero volume data, which is faulty.

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 23 Jun 2021

Include "Unreportable Last" Trades.
If this option is checked, the small size transactions (less than 1 lot) are included.

Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update.
If the discrepancy between DailyHigh (DailyLow) and IB real time data is detected, this option allows for the generation of a new tick with the required price and zero volume to eliminate the discrepancy. When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible.

lingwuchung · Postby **lingwuchung** » 23 Jun 2021

Hi Svetlana,

"When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible." -> so the resolution is to make "the discrepancy possible" rather than "impossible"?

Why would IB need to send such tick to align the DailyHigh? To rectify their previous data issue (much less frequently in other datafeed providers), or just in case MC missed the DailyHigh (from individual bars?) collected so far?

Include "Unreportable Last" Trades.
If this option is checked, the small size transactions (less than 1 lot) are included.

Generate a new Tick with Zero Volume upon DailyHigh (DailyLow) update.
If the discrepancy between DailyHigh (DailyLow) and IB real time data is detected, this option allows for the generation of a new tick with the required price and zero volume to eliminate the discrepancy. When this option is disabled, the discrepancy between DailyHigh (DailyLow) and IB real time data is possible.

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 24 Jun 2021

Hi lingwuchung,

IB may send such updates as shown in the screenshot.

phantom tick.jpg: (203.32 KiB) Not downloaded yet

According to IB support this is the high of the day due to the ticket type (6, see here).

lingwuchung · Postby **lingwuchung** » 25 Jun 2021

Thanks Svetlana. If this is the case, I think MC should uncheck this setting by default for IB data feed.

Hi lingwuchung,

IB may send such updates as shown in the screenshot.phantom tick.jpg
According to IB support this is the high of the day due to the ticket type (6, see here).

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 25 Jun 2021

"Generate a new tick with zero volume upon DailyHigh (DailyLow) update" is turned OFF by default since MultiCharts 14 Release 2 (if MC has been installed on a new computer or IB settings have been reset).

Meecc · Postby **Meecc** » 25 Jun 2021

Have unticked the check box "Generate a new tick with zero volume upon DailyHigh (DailyLow) update" ,
I still see the block trade which is at price-way-off the current bid ask level.

priceWayOff.PNG: (63.1 KiB) Not downloaded yet

When look in the quote manager,
it is seen as one trade with volume of 200 lot at price 65.67 which is far way-off below the nearby trade price at 73.65.

tradeBlock.PNG: (21.12 KiB) Not downloaded yet

When look into the corresponding bid and ask ,
there is no bid-ask tick at the 65.67 price level at all.

bidNoBlockPrice.PNG: (27.27 KiB) Not downloaded yet

askNoBlockPrice.PNG: (33.68 KiB) Not downloaded yet

While on tws, we don't see trade price at 65.67 price level at all.

twsNoBlock.PNG: (67.34 KiB) Not downloaded yet

My issue is that this block trade volume is not having to do with what retail traders can trade with,
but it affects badly to the calculation of strategy signal which bases on trade price.

Is there is a way to exclude these block trade by mc pre-sending this to mc user ?

May be if price is off by > some standard deviation amount of current price and the price is no near to the current bid ask at all,
then we have the option in mc check box to not receive this kind of block trade ticks at all.

Or may be there is a way to segregate this kind of block trades within the api sent from TWS ?
Again so we have the option to switch this off and not seen this price-way-off block trade at all.
so our strategy signal calculation is not affected.

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 01 Jul 2021

Hello Meecc,

Please turn OFF Include "Unreportable Last" Trades in IB data feed settings and restart all MultiCharts components. See screenshot.

UnreportableLast.jpg: (557.54 KiB) Not downloaded yet

Here is a comment from IB API support:
Please note that historical data at IB is filtered for trade types which occur away from the NBBO such as combo legs, block trades, and derivative trades. You may read more about this on the below link.

Meecc · Postby **Meecc** » 07 Jul 2021

Thank you Svetlana,

With

turn OFF Include "Unreportable Last" Trades in IB data feed settings

, it has worked well for the past 3 days.

May be this should also be default setting ,
and with documentation that it also
solves the block trades inclusion issue, which would not be known by the name and explanation of the check box.

With best regards,

lingwuchung · Postby **lingwuchung** » 10 Jul 2021

Thank you Svetlana,

With
turn OFF Include "Unreportable Last" Trades in IB data feed settings
, it has worked well for the past 3 days.

May be this should also be default setting ,
and with documentation that it also
solves the block trades inclusion issue, which would not be known by the name and explanation of the check box.

With best regards,

He did say it is default since v14 release 2 in his 25Jun post.

Meecc · Postby **Meecc** » 10 Jul 2021

There are 2 check boxes .

1) Default-Off was said for "Generate a new tick with zero volume"

"Generate a new tick with zero volume upon DailyHigh (DailyLow) update" is turned OFF by default since MultiCharts 14 Release 2 (if MC has been installed on a new computer or IB settings have been reset).

2) was not said as Default-Off for "Include Unreportable Last Trades"

A week has past, with all 2 boxes unchecked, still not seen block trades. it's ok.

Best regards,

Fake ticks form IB TWS

Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS

Re: Fake ticks form IB TWS