FOREX Trader with IB ?

Questions about MultiCharts and user contributed studies.
User avatar
Smoky
Posts: 301
Joined: 03 Dec 2010
Has thanked: 55 times
Been thanked: 74 times

FOREX Trader with IB ?

Postby Smoky » 29 Dec 2017

hi all,

any trader to share Quotemanager setup to trade forex with IB broker ?

Thanks

TraderWalrus
Posts: 63
Joined: 13 Sep 2016
Has thanked: 30 times
Been thanked: 7 times

Re: FOREX Trader with IB ?

Postby TraderWalrus » 31 Dec 2017

IB's Forex symbols don't appear under "Forex", they appear under "Cash".
Custom settings:
Price scale: 1/100000 for EURUSD, GBPUSD and 1/1000 for USDJPY
Min Movement: 5 (can also be 1 if you want 1/10 pips accuracy - I found that keeping this setting at TWS a bit buggy though)
BigPointValue: 1

Working with IB's data is a hassle. It takes a long time to load and the pacing limitations are really strict. No historical data at all on Saturdays (maintenance day), Sundays are OK. Access to Futures and Stocks historical data only start at 2am Monday.

Today I use IQFeed for almost everything, mapped to the corresponding IB symbols.
These users thanked the author TraderWalrus for the post (total 2):
sptraderSmoky

Zheka
Posts: 129
Joined: 13 Jan 2016
Has thanked: 5 times
Been thanked: 24 times

Re: FOREX Trader with IB ?

Postby Zheka » 02 Jan 2018

Min Movement: 5 (can also be 1 if you want 1/10 pips accuracy - I found that keeping this setting at TWS a bit buggy though)

I have never encountered a problem setting a 1/10 pips accuracy in IB Gateway. Setting Min Movement to 5 will cause errors in RT, if your strategy submits a limit order with a price not rounded to half-pip. + your execution price will be suboptimal vs. using a 1/10 pip accuracy.

Working with IB's data is a hassle. It takes a long time to load and the pacing limitations are really strict. No historical data at all on Saturdays (maintenance day), Sundays are OK. Access to Futures and Stocks historical data only start at 2am Monday.

If you do not load ticks - IB data do not take a long time to load. Pacing violations loading 1min bars are really rare, and can easily be resolved by pressing "ALT-F" - which 'resets' IBGateway client and then the data continue loading quite fast again.
Saturdays used to be maintenance day, but recently it is 50/50 for FX. Cannot comment on Futures and Stocks.

IB used to supply 200-250ms data snapshots across instruments (i.e. not real ticks), but appr. 1.5 years ago improved FX data granularity to 5ms (i.e. up to 200 ticks per second) - which is practically good enough for most trading methods and MC.
https://interactivebrokers.github.io/tw ... #gsc.tab=0

TraderWalrus
Posts: 63
Joined: 13 Sep 2016
Has thanked: 30 times
Been thanked: 7 times

Re: FOREX Trader with IB ?

Postby TraderWalrus » 02 Jan 2018

Zheka wrote:
Min Movement: 5 (can also be 1 if you want 1/10 pips accuracy - I found that keeping this setting at TWS a bit buggy though)

I have never encountered a problem setting a 1/10 pips accuracy in IB Gateway. Setting Min Movement to 5 will cause errors in RT, if your strategy submits a limit order with a price not rounded to half-pip. + your execution price will be suboptimal vs. using a 1/10 pip accuracy.

The 1/10 pips setting in TWS isn't stable. It sometime reverts to the normal precision by itself (gets unchecked in the settings->display->ticker row). I have reported this problem to IB several times in the last year, they said it may be due to updates or local settings, in any case it still happens.
After a few times my orders were rejected by IB (when it happened I discovered this setting got unchecked again), I moved to trade only 0.5 pips accuracy, and perform a rounding on the prices of stop/limit orders accordingly. Since I don't scalp or use tight targets this has statistically zero effect on my trading.

Working with IB's data is a hassle. It takes a long time to load and the pacing limitations are really strict. No historical data at all on Saturdays (maintenance day), Sundays are OK. Access to Futures and Stocks historical data only start at 2am Monday.

If you do not load ticks - IB data do not take a long time to load. Pacing violations loading 1min bars are really rare, and can easily be resolved by pressing "ALT-F" - which 'resets' IBGateway client and then the data continue loading quite fast again.
Saturdays used to be maintenance day, but recently it is 50/50 for FX. Cannot comment on Futures and Stocks.

I work with minute data and it can take a very long time to load when you use data of a few years or have a few instruments. Maybe you're used to waiting or maybe I'm spoiled and can't be bothered to wait. In any case there is no comparison between the historical data speed of IB to that of services such as IQFeed.
Saturday was still maintenance day the last time I checked which was a few weeks ago.

IB used to supply 200-250ms data snapshots across instruments (i.e. not real ticks), but appr. 1.5 years ago improved FX data granularity to 5ms (i.e. up to 200 ticks per second) - which is practically good enough for most trading methods and MC.
https://interactivebrokers.github.io/tw ... #gsc.tab=0

I backtest my strategies on longer periods than 1.5 years. I suggest to everyone to do the same.

Zheka
Posts: 129
Joined: 13 Jan 2016
Has thanked: 5 times
Been thanked: 24 times

Re: FOREX Trader with IB ?

Postby Zheka » 03 Jan 2018

1. I do not know how you trade and why would you run a bulky (and somewhat buggy) TWS for execution of automated strategies.
TWS has not been designed for that - so, use at your own peril.
IB Gateway is, in contrast, a very lightweight, robust application with minimum overhead which runs for months without any intervention. And 1/10-pips setting does not change once you set it.

2. Sure, IQ Feed downloads historical data faster. It also has the advantage of delivering all ticks for all exchange-traded instruments in real-time , while IB only delivers snapshots (and there are issues with volume). Making such data not 100% usable for tick- or range bars, volume-based analysis, etc. + a bar high in RT might not be the same as the high in historical data you download later on.

However (for live trading and specifically for FX):
- IQ Feed has higher latency vs. IB in getting real-time data.
- there is no perceptible difference when backfilling 1-5 days of data
- since FX is OTC, the data you get from IQ Feed will not match the data you will trade on. One will then have to spend time to match somehow both sources to understand the quality of execution.

I have downloaded the whole of FX history from IB on the development computer - once. Updating it once a week over the weekend happens quite quickly. Production is on a VPS and there is nothing to update.

3. Increased frequency of updates in real-time suggests nothing about the availability of quality historical data from IB for backtesting. Your comment is misleading.

TraderWalrus
Posts: 63
Joined: 13 Sep 2016
Has thanked: 30 times
Been thanked: 7 times

Re: FOREX Trader with IB ?

Postby TraderWalrus » 03 Jan 2018

Zheka wrote:1. I do not know how you trade and why would you run a bulky (and somewhat buggy) TWS for execution of automated strategies.
TWS has not been designed for that - so, use at your own peril.
IB Gateway is, in contrast, a very lightweight, robust application with minimum overhead which runs for months without any intervention. And 1/10-pips setting does not change once you set it.

In some cases manual intervention may be required to resolve any issue arises - I want to have TWS there for those times. How do you handle that?

a bar high in RT might not be the same as the high in historical data you download later on.
...
3. Increased frequency of updates in real-time suggests nothing about the availability of quality historical data from IB for backtesting. Your comment is misleading.


Is this so? you have just commented (the bold part) about the difference between RT and historical data. Don't you think this can lead to misleading backtest? I can only assume you do little backtesting if you don't appreciate the importance of accurate historical data. If you believe IB kept all the available data and just provided snapshots, I suggest you check with them, or just check your data.

IB are a great choice in many respects, but historical data isn't one of them. In the last 4-hours long outage they had in their HK servers, they didn't even bother to fill in the missing FX data, and insisted everything was OK, until I proved to them with screenshots etc. there are issues and large holes in the data. Then I got an apology and the data was corrected. Just one incident that made it clear to me not to rely on their data (I worked at different times with different sites - HK, Chicago and Connecticut).

However (for live trading and specifically for FX):
- IQ Feed has higher latency vs. IB in getting real-time data.

I did not notice that at all. I did a test just now. Almost the same, with IB being actually slower by 5 ms.

- there is no perceptible difference when backfilling 1-5 days of data

Correct, but 1-5 days of data is not much.

- since FX is OTC, the data you get from IQ Feed will not match the data you will trade on. One will then have to spend time to match somehow both sources to understand the quality of execution.

The data recorded by IB in their historical server won't match what you will trade on either. IB take their FX quotes from a pool of various providers and your order may be executed on a different provider in each time. This process is completely opaque and you have no way to know where it was executed, or of any changes IB may occasionally do in their list of counter parties.

I am happy you are satisfied - I'm not. I started with IB's data, never thinking I will have any reason to pay for extra data. After a long period that included many chats, tickets and some phone calls to their support I gradually started to move away. I certainly don't enjoy paying more but things are much easier today. And, I have ample live trading records that show that my data is close enough to the actual IB executions.

Zheka
Posts: 129
Joined: 13 Jan 2016
Has thanked: 5 times
Been thanked: 24 times

Re: FOREX Trader with IB ?

Postby Zheka » 03 Jan 2018

In some cases manual intervention may be required to resolve any issue arises - I want to have TWS there for those times. How do you handle that?
Automated trading is executed on a separate VPS which only has 2 programs installed and running: trading software (MC or SC) and IBGateway, in total occupying <1GB of RAM. There has never been a situation which could not have been resolved from within the trading software itself. But, just in case, TWS can be launched on a different computer.
you have just commented (the bold part) about the difference between RT and historical data. Don't you think this can lead to misleading backtest? I can only assume you do little backtesting if you don't appreciate the importance of accurate historical data. If you believe IB kept all the available data and just provided snapshots, I suggest you check with them, or just check your data.
LOL. This and your previous comment reveals you do not really know how IB handles data.
First, there will inevitably be differences between historical and RT data - as some trades are cancelled, some ticks are lost in transmission, errors are cleaned,etc. Not a lot of them - but there will be. This concerns all data providers. Second, IB do keep all the ticks, process them on the fly and provide them from a different server. I do not "believe" - I verified. In addition, IB since ages provide "True RT 5-sec" bars. Those include all the ticks, already cleaned, and 1-min bars built from them are identical to the "historical". For the "price" of 250-300ms delay, what you trade is exactly equal to what you test on.
did not notice that at all. I did a test just now. Almost the same, with IB being actually slower by 5 ms.

This cannot physically be true, since DTN servers are located west of Chicago (Dallas, Omaha), while IB's are in NYC (Greenwich) and cross-connected to Equinix NY4/NY6 datacenters - which is where all key FX ECNs (Currenex,etc) are located. Chicago itself is 22-40ms from NY.
The data recorded by IB in their historical server won't match what you will trade on either. IB take their FX quotes from a pool of various providers and your order may be executed on a different provider in each time.
It's just great that an order can get executed by different providers each time! Similar to any other exchange (almost), IB operates an ECN where each participant can provide liquidity. IB then reports Best Bid/Offer via API. The bigger the number of providers, the more liquid the market is and the lower the spreads. Slippage on order sizes of <1-1.5mio is mostly 0 or +/-0.1-0.2 pips vs streamed Best Bids/offers.
This process is completely opaque and you have no way to know where it was executed, or of any changes IB may occasionally do in their list of counter parties.
As long as liquidity is there and slippage is nearly non-existent, I do not really care who I transact with and the theoretical counterparty risk (not at the volume I do, anyway). BUT IF you really wanted to get to such details, then you can drill down the execution report that IB provides via Account Mngt.

TraderWalrus
Posts: 63
Joined: 13 Sep 2016
Has thanked: 30 times
Been thanked: 7 times

Re: FOREX Trader with IB ?

Postby TraderWalrus » 03 Jan 2018

Zheka wrote:
In some cases manual intervention may be required to resolve any issue arises - I want to have TWS there for those times. How do you handle that?
Automated trading is executed on a separate VPS which only has 2 programs installed and running: trading software (MC or SC) and IBGateway, in total occupying <1GB of RAM. There has never been a situation which could not have been resolved from within the trading software itself. But, just in case, TWS can be launched on a different computer.

Well, I didn't work with SC but with MC I have encountered various situations that had to be resolved directly through TWS - positions not begin recognized due to reboot that was necessary for some reason, or an issue with authentication servers, or rejected exit orders, due to size restriction, or missing symbols mapping, or when opening a position in portfolio trader - when in doubt, when you need to get a reliable picture of what you hold and what happened you look at TWS, not at any trading software which is another layer.

Second, IB do keep all the ticks, process them on the fly and provide them from a different server. I do not "believe" - I verified.

I recall IB told me that they record in the historical server is what they provide - not anything else - so if they provided snapshots that's what you get in the historical server. But it can be easily checked again, when I have some time I will a comparison. Both for that and for latency issue.

The data recorded by IB in their historical server won't match what you will trade on either. IB take their FX quotes from a pool of various providers and your order may be executed on a different provider in each time.
It's just great that an order can get executed by different providers each time! Similar to any other exchange (almost), IB operates an ECN where each participant can provide liquidity. IB then reports Best Bid/Offer via API. The bigger the number of providers, the more liquid the market is and the lower the spreads.

You completely misunderstood my remark. Yes, it is good that IB uses different providers. What it means though is that your belief the data you see in historical records is the data you will trade on is just wrong. There is loose connection between the two, not more.

This process is completely opaque and you have no way to know where it was executed, or of any changes IB may occasionally do in their list of counter parties.
As long as liquidity is there and slippage is nearly non-existent, I do not really care who I transact with and the theoretical counterparty risk (not at the volume I do, anyway).

See my reply above. It's not about counterparty risk but about your conclusions regarding what historical data means for the future executions.

BUT IF you really wanted to get to such details, then you can drill down the execution report that IB provides via Account Mngt.

Yes, I did at times need to understand why slippage was in fact large, up to a few pips, and not always non-existent as you mention, I am very familiar with the account audit trail data. Again, it isn't related to how historical data will accurately or closely represent future executions.

Zheka
Posts: 129
Joined: 13 Jan 2016
Has thanked: 5 times
Been thanked: 24 times

Re: FOREX Trader with IB ?

Postby Zheka » 04 Jan 2018

Of course - data back in history cannot "represent" future executions, regardless of the number of providers.
But if an order is issued to "buy next bar at market" when backtesting in MC in Extended mode, then using "historical' bid/ask data from IB you can be pretty sure that it would indeed have been executed at the bid or ask as in the data , with negligible slippage, - and I have verified this in live trading.

Link between FX data you get via IQFeed and executions at IB is by definition not that direct. As far as I know, IB is not providing data to DTN. Even if it were, it would come to you with a delay, and this would introduce slippage.
I anyway fail to see the relevance of the "opaque and changing" provider mix to the subject.


Return to “MultiCharts”