If you use IB data with MC to daytrade, STOP  [SOLVED]

Questions about MultiCharts and user contributed studies.
dennisc
Posts: 32
Joined: 25 Feb 2014
Has thanked: 16 times
Been thanked: 9 times

If you use IB data with MC to daytrade, STOP

Postby dennisc » 17 Nov 2014

aud.usd.png
(131.86 KiB) Downloaded 920 times
For those of you who are trying to daytrade using IB data:

IB is a broker.

IB is NOT a legitimate data provider and their data is unusable for futures (and I assume stocks) daytrading purposes.

You can see just how unusable their data is by creating a tick-based chart of whatever futures contract you want to trade using IB data, and then creating the same tick-based chart using a real data feed such as the DTN IQ feed. The DTN IQ feed chart will produce roughly 4 times as many bars as the chart created using the IB feed. If you use IB data, your indicator values will not be based on reality and any setups you trade may not exist in reality.

IB is also NOT a legitimate data provider for forex daytrading purposes (even with their own Ideal Pro forex data feed!), for other reasons:

1. IB forex data is snapshot data NOT real tick-by-tick data; IB prints the price that exists every 1/5th of a second. Whether you use tick-based or time-based charts, you will NOT always see the actual high or low bid/ask or trade prices that occurred at certain times. How do you set a stop if you can't see the actual high or low? IB does have a function call available that will poll the data every 5 seconds to get the actual highs or lows, but MC isn't programmed to use it AFAIK. But even that feature means you'll be getting delayed data; and I don't know if it would even apply to tick-based charts.

2. If at the end of the week you decide to reload a tick-based chart to make sure you're up to date for the upcoming week, you'll find that instead of getting more bars of data that may have been missing because of IB's daily TWS shutdowns, you end up with fewer bars than you had before reloading! I don't mean 1 or 2 bars - I mean you'll only have 2/3 of the bars you had BEFORE you reloaded! WTF? I don't know if that's an IB issue or a MC issue, but which chart is right? How can you use IB data with MC to daytrade forex with that kind of uncertainty?

If someone knows of some different setup in QM for the IB data feed (or any other way) to solve these issues, I'd love to hear about it. Otherwise, I have to give up on IB (assuming the problem isn't a glitch in MC) and go elsewhere to daytrade forex. BTW, does the free MCFX deal with FXCM still exist since they lowered their standard account minimum?

DC
Attachments
aud.usd.png
(131.86 KiB) Downloaded 905 times

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: If you use IB data with MC to daytrade, STOP

Postby Andrew MultiCharts » 18 Nov 2014

1. IB forex data is snapshot data NOT real tick-by-tick data; IB prints the price that exists every 1/5th of a second. Whether you use tick-based or time-based charts, you will NOT always see the actual high or low bid/ask or trade prices that occurred at certain times. How do you set a stop if you can't see the actual high or low? IB does have a function call available that will poll the data every 5 seconds to get the actual highs or lows, but MC isn't programmed to use it AFAIK. But even that feature means you'll be getting delayed data; and I don't know if it would even apply to tick-based charts.

2. If at the end of the week you decide to reload a tick-based chart to make sure you're up to date for the upcoming week, you'll find that instead of getting more bars of data that may have been missing because of IB's daily TWS shutdowns, you end up with fewer bars than you had before reloading! I don't mean 1 or 2 bars - I mean you'll only have 2/3 of the bars you had BEFORE you reloaded! I don't know if that's an IB issue or a MC issue, but which chart is right? How can you use IB data with MC to daytrade forex with that kind of uncertainty?

If someone knows of some different setup in QM for the IB data feed (or any other way) to solve these issues, I'd love to hear about it. Otherwise, I have to give up on IB (assuming the problem isn't a glitch in MC) and go elsewhere to daytrade forex. BTW, does the free MCFX deal with FXCM still exist since they lowered their standard account minimum?
Hello dennisc,
  1. MC shows real-time data as it streams from IB API. Indeed IB's data is not real "every tick" data, what your MC plots as ticks on chart = snapshots, this is how IB streams data.
  2. Again, the software shows data as it is provided by API through their API for 3rd-party programs. The only settings affecting the data are located in QuoteManager. We confirm that most data vendors return different historical data (after chart is reloaded) in comparisson to rt data.
MCFX:
There are 3 MCFX versions:
  • Demo
  • Basic
  • Pro
Unfortunately the demo (free) version of MCFX is currently not working due to an issue on our server.
Currently we have 2 versions of MCFX that are available on our web page: https://www.multicharts.com/download-forex-software/
These versions requires subscription: https://www.multicharts.com/forex-tradi ... /purchase/
The offer you mentioned above is still available: https://www.multicharts.com/free-forex-charts/
MCFX data can be also used in MultiCharts. Please read the following article to learn more: https://www.multicharts.com/trading-sof ... ng_on_FXCM

At the moment FXCM brokerage is not recommend for both MultiCharts and MCFX, because their old API is used. We are planning to implement new FXCM API, most likely it will be included in MultiCharts 9.1 Release. We don't provide any support in regards to any trading problems with FXCM broker. Thank you for understanding.

dennisc
Posts: 32
Joined: 25 Feb 2014
Has thanked: 16 times
Been thanked: 9 times

Re: If you use IB data with MC to daytrade, STOP

Postby dennisc » 18 Nov 2014

Hello dennisc,
  1. We confirm that most data vendors return different historical data (after chart is reloaded) in comparisson to rt data.
At the moment FXCM brokerage is not recommend for both MultiCharts and MCFX, because their old API is used. We are planning to implement new FXCM API, most likely it will be included in MultiCharts 9.1 Release. We don't provide any support in regards to any trading problems with FXCM broker. Thank you for understanding.
Thanks for the info. Could you please list the data vendors whose data changes when reloaded? In the past I've used DTN IQ; I don't recall seeing a different-looking chart when their data was reloaded. Does the FXCM feed change after reloading? Anyone???

I'm wondering why there would be any difference after reloading; is the realtime feed so bad that the reloaded data requires correction later? I can understand correction of obvious occasional errors, but not the resulting elimination of 1/3 of the bars that showed up in the realtime feed, as with IB.

As for updating the FXCM API in MC 9.1, will it also be updated for MCFX? If so, what is the approximate date of release?

Thanks,

DC

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: If you use IB data with MC to daytrade, STOP

Postby Andrew MultiCharts » 18 Nov 2014

Thanks for the info. Could you please list the data vendors whose data changes when reloaded? In the past I've used DTN IQ; I don't recall seeing a different-looking chart when their data was reloaded. Does the FXCM feed change after reloading? Anyone???
Unfortunately no. We cannot be sure it can be replicated with any security from each data vendor. Our engineers let me know that all supported vendors at least once changed their histoircal data in comparisson to real-time data.
I'm wondering why there would be any difference after reloading; is the realtime feed so bad that the reloaded data requires correction later? I can understand correction of obvious occasional errors, but not the resulting elimination of 1/3 of the bars that showed up in the realtime feed, as with IB.
Please forward your question to IB API support team, as MC simply displays what is sent from IB API.
As for updating the FXCM API in MC 9.1, will it also be updated for MCFX? If so, what is the approximate date of release?
At the moment MCFX is not scheduled for this API update.

dennisc
Posts: 32
Joined: 25 Feb 2014
Has thanked: 16 times
Been thanked: 9 times

Re: If you use IB data with MC to daytrade, STOP

Postby dennisc » 19 Nov 2014

I'm wondering why there would be any difference after reloading; is the realtime feed so bad that the reloaded data requires correction later? I can understand correction of obvious occasional errors, but not the resulting elimination of 1/3 of the bars that showed up in the realtime feed, as with IB.
Please forward your question to IB API support team, as MC simply displays what is sent from IB API.

Before I call IB, I need to ask: when the data comes from IB for reloading, it has to go through MC to be saved; since there were more saved ticks in a MC data file before the reloading process, it would seem that the reloading process through MC deletes the previously saved data. Is there some way that MC can be set up to only fill in missing data without eliminating the previously saved data?

DC

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: If you use IB data with MC to daytrade, STOP

Postby Andrew MultiCharts » 19 Nov 2014

Before I call IB, I need to ask: when the data comes from IB for reloading, it has to go through MC to be saved; since there were more saved ticks in a MC data file before the reloading process, it would seem that the reloading process through MC deletes the previously saved data. Is there some way that MC can be set up to only fill in missing data without eliminating the previously saved data?
The answer to your question is negative.
There can be 2 different scenarios:
  1. There is no data in the symbol. Symbol is charted, historical data is cached, MC is closed, the data (that was shown on the chart) is saved to the database. Next time the same chart is opened, the saved data is brought up, the missing interval is downloaded from the vendor.
  2. There is no data in the symbol. Symbol is charted, only real-time data is received and cached, MC is closed, the data (that was shown on the chart) is saved to the database. Next time the same chart is opened, the saved data is brought up, the missing interval is NOT downloaded from the vendor.
Please note that in scenario 1 if there is a gap sent from the vendor to MC, that gap will be written to the database and will not be filled next time you open the chart, because MC will think the gap should be there (since it had been returned from the data provider). In this case the only way to fill the gap is to reload the data.
When data is reloaded, new historical data is downloaded, cached and when you close MC it replaces the same data interval in the database removing old data if there is any.
You can learn more about collecting and saving data here.

hilbert
Posts: 224
Joined: 17 Aug 2011
Has thanked: 76 times
Been thanked: 64 times

Re: If you use IB data with MC to daytrade, STOP

Postby hilbert » 19 Nov 2014

Please note that in scenario 1 if there is a gap sent from the vendor to MC, that gap will be written to the database and will not be filled next time you open the chart, because MC will think the gap should be there (since it had been returned from the data provider). In this case the only way to fill the gap is to reload the data.
Andrew - a technical question. For scenario 1, how does MC determine whether no trade has really taken place for a particular instrument and data vendor has actually sent a "gap" Vs determining that some trades have been missed? Thank You.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: If you use IB data with MC to daytrade, STOP

Postby Andrew MultiCharts » 20 Nov 2014

If you request an interval of historical data = 100 bars from your data provider and the data provider returns 99 bars (for a reason), then MC will write down the data to the database with this 1 bar gap (so-called "empty record") and will not request data to fill the gap unless you manually reload it.

If you have data for Monday in your database, don't have data for Tuesday (you did not launch the software that day) and you launch MC on Wednesday, MC will understand that the Tuesday was not filled with any record and request historical data for Tuesday (and for Wednesday if settings of your charts state so).

hilbert
Posts: 224
Joined: 17 Aug 2011
Has thanked: 76 times
Been thanked: 64 times

Re: If you use IB data with MC to daytrade, STOP

Postby hilbert » 20 Nov 2014

If you request an interval of historical data = 100 bars from your data provider and the data provider returns 99 bars (for a reason), then MC will write down the data to the database with this 1 bar gap (so-called "empty record") and will not request data to fill the gap unless you manually reload it.
1) So, this is applicable only when we are requesting historical "time bars", because we can specify 100 bars, or 100 1-minute bars. Instead, if we are requesting historical "ticks" then there is no way to know whether no trade has really taken place for a particular instrument and data vendor has actually sent a "gap" Vs determining that some trades have been missed....Correct?

2) Stated differently, when historical "ticks" are requested then MC NEVER thinks there has been any gap, even if say for 15 seconds, server doesn't send any tick for historical data....Correct?

Above discussion is valid for requesting historical data. How does MC determine that a real-time feed is working fine? For example, for IB, I assume that you test to see whether usfarm, hkfarm, usfuture etc. are all up and working fine. In case, one of them is not working, you might show a disconnection event in the QM.....correct? So, if usfuture is broken, maybe it will affect charts of all the globex instruments, and MC will request backfill when connection event happens again in QM....correct? Can it be done at the instrument level, instead of just at the farm level?

I thank you for clarifying these important points.

User avatar
Andrew MultiCharts
Posts: 1587
Joined: 11 Oct 2011
Has thanked: 931 times
Been thanked: 559 times

Re: If you use IB data with MC to daytrade, STOP  [SOLVED]

Postby Andrew MultiCharts » 20 Nov 2014

1) So, this is applicable only when we are requesting historical "time bars", because we can specify 100 bars, or 100 1-minute bars. Instead, if we are requesting historical "ticks" then there is no way to know whether no trade has really taken place for a particular instrument and data vendor has actually sent a "gap" Vs determining that some trades have been missed....Correct?
What i described in my example above is applicable every time when any kind of historical data is requested.
2) Stated differently, when historical "ticks" are requested then MC NEVER thinks there has been any gap, even if say for 15 seconds, server doesn't send any tick for historical data....Correct?
The assumption that ticks are treated differently is wrong. MC requests data in intervals. When an interval is received, the data from the interval is saved to the database. If there is a missing tick/bar in the interval, the "empty record" is also saved.
Above discussion is valid for requesting historical data. How does MC determine that a real-time feed is working fine? For example, for IB, I assume that you test to see whether usfarm, hkfarm, usfuture etc. are all up and working fine. In case, one of them is not working, you might show a disconnection event in the QM.....correct? So, if usfuture is broken, maybe it will affect charts of all the globex instruments, and MC will request backfill when connection event happens again in QM....correct? Can it be done at the instrument level, instead of just at the farm level?
In current version of MC (9.0 Release 3) only tick-based charts backfill gaps, which appears in real-time. Other charts don't do this. In my previous post i gave you the link that explains that by default MC doesn't store collected in real-time data (thus the gaps will not be stored either and will be backfilled with historical data next time you plot the chart).

hilbert
Posts: 224
Joined: 17 Aug 2011
Has thanked: 76 times
Been thanked: 64 times

Re: If you use IB data with MC to daytrade, STOP

Postby hilbert » 20 Nov 2014

1) So, this is applicable only when we are requesting historical "time bars", because we can specify 100 bars, or 100 1-minute bars. Instead, if we are requesting historical "ticks" then there is no way to know whether no trade has really taken place for a particular instrument and data vendor has actually sent a "gap" Vs determining that some trades have been missed....Correct?
What i described in my example above is applicable every time when any kind of historical data is requested.
2) Stated differently, when historical "ticks" are requested then MC NEVER thinks there has been any gap, even if say for 15 seconds, server doesn't send any tick for historical data....Correct?
The assumption that ticks are treated differently is wrong. MC requests data in intervals. When an interval is received, the data from the interval is saved to the database. If there is a missing tick/bar in the interval, the "empty record" is also saved.
I think I understood all this regarding historical data. Thanks!
Regarding real-time data, I will do some tests and create another thread with further questions. Thanks again.


Return to “MultiCharts”