OANDA Data / Chart / Bar issues

Questions about MultiCharts and user contributed studies.
beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

OANDA Data / Chart / Bar issues

Postby beck donald » 06 Mar 2018

10.0 Build 14417

In my ongoing effort to make a strategy work. To make sure the strategy is operating properly in live trading, I am running the identical chart below it. It is just updating with the data stream.

Trades take place on the live chart that do not have signals on the updating chart (Non-Traded).
and or
Signals take place on the updating chart that do not have trades taking place on the live chart.

1. I am running various minute timeframes.
2. I use extended mode Ask and Bid Data
3. I have backtesting precision set to 1 tick and have tried 1 second.
4. I enter buys as the market drops and sells as the market rises using limits to enter and exit trades.

I tracked it down to Data is missing.

Live Trading
I have trades that have been executed by OANDA at prices that are not present visually on the chart bar and when looking into the Quote Manager on a tick level or minute level do not exist. They existed at ONADA and therefore the executed the trade which appears on the Chart, and therefore live data is missing

I have updating Charts (Non-Traded) that have signals that appear at prices that either on tick or minute level do not exist in the Quote manger producing inaccurate backtests.

Please look into this, a lot of time has been spent trying to isolate this ongoing issue.

Thank you

The picture documents the problem.
Attachments
MC1.jpg
MC1.jpg (454.51 KiB) Viewed 1224 times

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 06 Mar 2018

To add to my digging.

At no immediate time surrounding this trade did the price exist in ask data

It does however show up on Oanda's chart.
Attachments
OANDA.png
OANDA.png (247.18 KiB) Viewed 1221 times

User avatar
TJ
Posts: 7016
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 984 times
Been thanked: 2008 times

Re: Data issues

Postby TJ » 06 Mar 2018

Please be specific with the details.

What price?
What price do you see on the chart?
What price do you see on whatever alternative source you have?

Is this backtesting? or live data?

Did you reload the chart?

What do you think is the problem?
Is this a dataprovider problem? or MultiCharts problem?

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 06 Mar 2018

All prices are highlighted in the pictures and explained if you read my post....however...

Buy signal live trading at 1.23295. That was the fill from OANDA and that price does not exisit on the MC chart. I use limit orders against the trend which is why it shows up. The MC range of that bar was 1.23373 High and 1.23315 Low. OANDA had a low of 1.23288 and therefore the order filled.

Live trading

No,..not reloaded. However if I make a 3rd chart after the fact, it will match the one not being traded which has been an on going issue, meaning a reloaded chart does not match a live traded chart. It does however match the live chart not traded. I run 2 live charts when looking for a problem. One trades live, the other updates. I have automated charts ranging from 10 seconds to daily bars. many have experienced the issue at different times of day and different prices since the test started on Sunday.

The problem is...
Either the data OANDA has on there end is not being sent as they use it themselves or MC is getting it and not processing it the way it is sent.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 06 Mar 2018

Digging a bit deeper I installed Oanda's MT4. That also has a lower data point that would trigger the trade. Pondering it all... MC is pulling the data directly from OANDA. If all the data is deleted and refreshed, the same chart shows up. I can't imagine MC has a filter to limit ticks, but if it does it is the issue. I think the data being sent to the servers internally at OANDA, which we access with MC is different than the data available to those using OANDA's platform or their version of MT4. I have communicated with and sent the support to show this is taking place. In todays world, the only other broker option for MC and Forex is IB. I wish MC would release a version for FOREX.COM to give us another option. For me, IB won't work. Assuming MC says they do not affect the data, if you are an OANDA client and want it fixed, contact frontdesk@onada.com and raise some hell. They need to know that clients are prepared to walk if it is not fixed. It doesn't matter how you trade manual or automatic. If you use the chart bars to figure out your trading and the bars are wrong, then your trades might be incorrect as well.

User avatar
TJ
Posts: 7016
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 984 times
Been thanked: 2008 times

Re: Data issues

Postby TJ » 06 Mar 2018

What you are describing happens quite often with broker-provided data. (ie not a paid dedicated data-provider)

The data are not tick-by-tick quotes; they are snapshots in very high frequency. Usually in 1/3 to 1/4 second intervals.
That's why in fast markets, you might see missing quotes at the peaks and bottoms.
Usually, when you refresh the data, you are pulling the quotes from the historic server, the quotes will be corrected.

If, after your data refresh, your filled trade is still different than the chart, you should contact your broker for a trade reversal.

For more information on data snapshots, you can check the small prints in your broker agreement.


ps.

MC does not filter to limit ticks. Why would they do that? MultiCharts has stated this many times in various posts. I do not have a link at hand, but you can search for it, or ask MultiCharts tech support for this question again.

Also, MultiCharts does not go to OANDA server to retrieve your data. The quotes are pushed to your computer through your account login.
ie. MultiCharts plots whatever it gets.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 06 Mar 2018

I assumed MC would not filter data. The issue surfaces because it is selective. Use MT4 or OANDA's platforms and you get a different data stream than a MC user. If it all was filtered and matched there wouldn't be an issue. When you get to the point that the chart your trading doesn't have the data on it to cause a trade, but a trade happens, thats a big problem.

hughesfleming
Posts: 208
Joined: 22 Apr 2014
Has thanked: 53 times
Been thanked: 52 times

Re: Data issues

Postby hughesfleming » 07 Mar 2018

Hi Beck,

Can you explain why IB won't work? Have you considered LMAX as a broker?

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 07 Mar 2018

Hello Hughes,

I am based in the US, LMAX is not an option. For me IB will not work because, there is a dialing in process for trade automation and with a min trade size of 25k it would be to costly to hammer out issues live. If it would all work, I would be trading a number of charts and each with multiple positions. Compounding results will be impacted because there would be longer pauses between portfolio size adjustments (when compared with OANDA at any trade size, Forex.Com at 1k). When I had been on top of it, IB seemed to update TWS alot and it causes lots of upgrading issues.
These users thanked the author beck donald for the post:
hughesfleming

Zheka
Posts: 186
Joined: 13 Jan 2016
Has thanked: 8 times
Been thanked: 42 times

Re: Data issues

Postby Zheka » 08 Mar 2018

IB would work as smooth as silk.

1) use IB Gateway, not TWS. It can stably run for months, no need to re-login daily; update it as you need.
2) set-up a second "user" for the account without (card) authentication -with trading permissions only - if you want to avoid "dialing in" during live trading for automated re-login
3) min. trade sizes at IB are "tentative", i.e. you would get a "Odd lot..." warning but this would rarely impact execution. Trade in whatever size/increments you want.
4) spreads and liquidity are better.
5) with recent upgrades to the API, tick frequency is 5ms, up to 200 ticks/s.
Still, you might see occasional differences between historical and real-time data.

Enjoy!
These users thanked the author Zheka for the post:
TJ

User avatar
TJ
Posts: 7016
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 984 times
Been thanked: 2008 times

Re: Data issues

Postby TJ » 08 Mar 2018

IMHO, for forex, IB is the way to go. A lot of the others are highway robbers.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 08 Mar 2018

I did not know you could connect without TWS. You do mean the MC gateway in broker profile? I am not clear on your item #2. Is this step the one that makes it stable?

Thank you!

Zheka wrote:IB would work as smooth as silk.

1) use IB Gateway, not TWS. It can stably run for months, no need to re-login daily; update it as you need.
2) set-up a second "user" for the account without (card) authentication -with trading permissions only - if you want to avoid "dialing in" during live trading for automated re-login
3) min. trade sizes at IB are "tentative", i.e. you would get a "Odd lot..." warning but this would rarely impact execution. Trade in whatever size/increments you want.
4) spreads and liquidity are better.
5) with recent upgrades to the API, tick frequency is 5ms, up to 200 ticks/s.
Still, you might see occasional differences between historical and real-time data.

Enjoy!

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 08 Mar 2018

crapola They do not offer FX to US clients any longer

sptrader
Posts: 696
Joined: 09 Apr 2010
Location: Colorado
Has thanked: 455 times
Been thanked: 256 times
Contact:

Re: Data issues

Postby sptrader » 08 Mar 2018

If you have Windows10 just click on the bottom left icon to see installed programs, under Interactive brokers, you'll see a drop-down menu that includes IB Gateway. It's just for trading, not much info there but it won't log you out like TWS does.

Zheka
Posts: 186
Joined: 13 Jan 2016
Has thanked: 8 times
Been thanked: 42 times

Re: Data issues

Postby Zheka » 08 Mar 2018

IB Gateway is a small gateway app from IB to connect via API.
I am not clear on your item #2. Is this step the one that makes it stable?
No, it is stable by design (and lack of GUI).
But this avoids having to type authentication codes if the app crashes in automated trading and you then restart it automatically (but this rarely happens).

Are you 100% sure they are closed for US residents? Better confirm with support...

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 08 Mar 2018

I called client services..no US accounts for FX :(

User avatar
TJ
Posts: 7016
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 984 times
Been thanked: 2008 times

Re: Data issues

Postby TJ » 08 Mar 2018

Why not trade the futures?
It is a better-controlled market than the FX.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 08 Mar 2018

Agreed but contract periods and continuous data had been an issue in the past (20 years ago), no idea about modeling in today's world. In forex limits are filled at touch. Futures you have to expect to go through the price by a tick. If there is no other choice, I would need to investigate it, but I am quite sure any kind of high speed trading (hundreds of trades per day) is out and therefore a whole lot of work goes down the drain.

Zheka
Posts: 186
Joined: 13 Jan 2016
Has thanked: 8 times
Been thanked: 42 times

Re: Data issues

Postby Zheka » 08 Mar 2018

In forex limits are filled at touch. Futures you have to expect to go through the price by a tick
In a free 'market" (ECN in case of IB or exchange in case of futures) if there is somebody on the other side - you will be filled. So, it all comes down to depth/liquidity at the time.
Now, practically speaking, you can expect decent liquidity/spread and fills in big contracts in EUR and JPY, somewhat worse in GBP, AUD and CAD. Crosses and mini-contracts have insignificant participation.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 09 Mar 2018

Zheka,

Although some time ago, it had been my experience with limit orders, is that they took place after market orders and you got in line. With MC, it will cancel and resubmit the same order with the closing of a bar. If trading at lets say with 1 minute bars, Every minute you get sent to the back of the line. It maybe mute as OANDA just said they can do nothing to help as it is a 3rd party software issue. I do not understand how there is not a point person to deal with issues so they can be resolved.

I did setup a demo with the TWS and the flow of ticks for forex is much greater. Although I can;'t trade it, I was able to see how OANDA compared. They don;t.

Thanks
Don

Zheka
Posts: 186
Joined: 13 Jan 2016
Has thanked: 8 times
Been thanked: 42 times

Re: Data issues

Postby Zheka » 11 Mar 2018

With MC, it will cancel and resubmit the same order with the closing of a bar.
AFAIK, MC cancels/resubmits a limit order, only if its price changed.
Write an email to Support, if it does not.

I was able to see how OANDA compared. They don;t.
In what sense?

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 11 Mar 2018

As you had shared... the data stream from IB is more robust and constant. For the period when OANDA had gaps (as an example) from 1.20000 to 1.20010, IB had price of 1.20000 to 1,20001 to 1.20002 to 1.20003 and so on.

wilkinsw
Posts: 447
Joined: 21 Apr 2013
Has thanked: 111 times
Been thanked: 75 times

Re: Data issues

Postby wilkinsw » 12 Mar 2018

beck donald wrote:Agreed but contract periods and continuous data had been an issue in the past (20 years ago), no idea about modeling in today's world. In forex limits are filled at touch. Futures you have to expect to go through the price by a tick. If there is no other choice, I would need to investigate it, but I am quite sure any kind of high speed trading (hundreds of trades per day) is out and therefore a whole lot of work goes down the drain.


Futures is certainly more work to compile quality historic datasets but I strongly believe is worth it.

There are sharks in all markets but unless you are a big trader with relationships avoid a broker who is making prices..... futures is the way to go IMHO.

Also, I've suffered CQG API provided post hoc data changes, like you describe. Like TJ says, a dedicated feed provider (e.g. IQfeed) will probably solve that.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Re: Data issues

Postby beck donald » 12 Mar 2018

Thank you all for taking the time to offer your experiences and suggestions. I have begun the process to evaluate Futures.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Changes at OANDA

Postby beck donald » 14 Mar 2018

While I continue to trade with OANDA and evaluate futures, I have been in contact with OANDA and here is the response... the v3Rest I guess is new.

We will be upgrading your legacy trade account(s) to a new V20 account(s) starting in May 2018. At this time the funds in your legacy account(s) and any open positions will be transferred to your new V20 account(s).

You have an active API token. If you are trading via our API there will be impacts to your trading algorithms due to this upgrade. We will give you a more specific date closer to the upgrade, however we wanted to give you as much time as possible to make changes to your trading algorithms.

Your V20 account will be unable to use the v1 REST API. Instead, please use the v3 REST API with your V20 account. There are resources available on our developer portal to help you migrate your applications. Specifically, we have reference implementations located at http://developer.oanda.com/rest-live-v20/sample-code/. Also, feel free to contact api@oanda.com for more information or if you require assistance.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Seems to be fixed

Postby beck donald » 15 Mar 2018

I deleted all data, cleared cache and reloaded fresh data using v20 and the historic data changed to match the live data when the problems took place. So it would seem that although I do not know what changed (The fellow at OANDA API did not share info), it appears to now be correct. I am running the latest MC build, V20 along with the recent MC patch. Data also loaded quicker than the last v20 I ran. That does not mean that the quantity of ticks has greatly increased, just that the API stream now seems to match the OANDA platform.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

Fix Confirmed by OANDA

Postby beck donald » 15 Mar 2018

I hope I can verify the effort worth it...

From OANDA this AM.

Dear XXXXXXXXXXXXXX

Thank you for contacting OANDA.

There was a problem regarding 3rd party trading platform. We have resolved it from our end.

beck donald
Posts: 195
Joined: 25 Jan 2008
Has thanked: 2 times
Been thanked: 5 times

OANDA Data / Chart / Bar issue continues

Postby beck donald » 16 Mar 2018

MultiCharts Version 11 build 15808.

OANDA said they fixed the problem on their end, however I had the same thing happen. I had a buy take place at 1.22684 which is lower than the low of the bar 1.22686. The bar closed at 1.22700. I will send the info to OANDA, however this is an open an ongoing problem and likely will need to test futures.


Return to “MultiCharts”