Loading missing bars of data

et1hugo · Postby **et1hugo** » 19 Sep 2022

Hi all,

I created a strategy (I guess it can be an indicator as well) to check the completeness of data on a chart for back testing purposes. The output would look as follows for SPY loading daily bars from 1/1/1993 to 19/09/2022, it computes that about 3.5% of the data over that time period is missing:

------------------------------------------------------------------------
SPY - Quality 96.52%, Received 7264, Expected 7526
------------------------------------------------------------------------
Symbol,MissingBars,StartDate,EndDate
SPY,3,12/30/1993 4:00:00 PM,1/3/1994 4:00:00 PM
SPY,2,1/16/1994 4:00:00 PM,1/18/1994 4:00:00 PM
SPY,2,4/1/1994 4:00:00 PM,4/3/1994 4:00:00 PM
SPY,1,4/27/1994 4:00:00 PM,4/27/1994 4:00:00 PM
SPY,2,6/19/1994 4:00:00 PM,6/21/1994 4:00:00 PM
SPY,2,10/9/1994 4:00:00 PM,10/11/1994 4:00:00 PM
SPY,3,11/10/1994 4:00:00 PM,11/14/1994 4:00:00 PM
SPY,2,1/15/1995 4:00:00 PM,1/17/1995 4:00:00 PM
SPY,2,4/14/1995 4:00:00 PM,4/16/1995 4:00:00 PM
SPY,2,6/18/1995 4:00:00 PM,6/20/1995 4:00:00 PM
SPY,2,10/8/1995 4:00:00 PM,10/10/1995 4:00:00 PM
SPY,3,11/9/1995 4:00:00 PM,11/13/1995 4:00:00 PM
......

Given that this data is pretty fragmented I want to add functionality to this strategy to send data requests to download the missing sets of data, I'm using FreeQuotes for this but the data vendor is not that relevant. I've added the following code to the strategy but it does not seem to work:

Code: Select all

        if (FixData)
        {
          foreach (Tuple<DateTime, DateTime> missingBarRange in _missingBarRanges) {
            InstrumentDataRequest request = Bars.Request;
            request.Subscribe2RT = true;
            DataRequest dataRequest = new DataRequest();
            dataRequest.RequestType = DataRequestType.FromTo;
            dataRequest.From = missingBarRange.Item1;
            dataRequest.To = missingBarRange.Item2;
            request.Range = dataRequest;
            DataLoader.BeginLoadData(request, OnData, null);
          }
        }

I would expect to see the data requests in the QuoteManager if it is actually working but this is not the case and the data loader also does not seem to actually work when the data is not there. I would expect it to send a request to the data vendor if the data is missing to update the local cache of data, does this code look correct or am I missing something here?

Thank you in advance for any feedback.

Kind regards
Ettienne

Svetlana MultiCharts · Postby **Svetlana MultiCharts** » 30 Sep 2022

Hi Ettienne,

DataLoader request is equivalent to building a chart. In other words, if the data already exists in the database, MultiCharts takes it from there and does not request new data. Unfortunately, there is no way to request new data from script in this case. You can try to request the data from a different provider.

et1hugo · Postby **et1hugo** » 02 Oct 2022

Thank you for the reply Svetlana,

It would be great if the platform can support that so I'll log a feature request for this. The issue is deeper lying in the sense that it is hard to determine whether the data just got missing in the replies from the data vendor or for whatever reason or whether the vendor has actual holes in their data. It would be great to be able to analyze the data get the missing bars and request them, if they are still missing after requesting one or more times it can be safely assumed that the vendor does not have the data.

I guess I can always do this outside Multicharts and import the data into the Quote Manager but it would be great if the platform can support this natively.

Kind regards
Ettienne

Loading missing bars of data

Loading missing bars of data

Re: Loading missing bars of data

Re: Loading missing bars of data