Historical Data Download Scheduler

Questions about MultiCharts and user contributed studies.
Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Historical Data Download Scheduler

Postby Mydesign » 10 Nov 2020

Hello,

I want to use the new Historical Data Download Scheduler to refresh my datas everyday as true backfill and to overwrite the eventual realtime collected datas. My data source is LMAX.

To do so I created a daily task for downloading the last 2 days, and use the Data removal option, but the wiki is a bit confusing:
Data removal options section allows to specify the types of data for the selected symbols to be removed from the MultiCharts database before downloading the historical data.
  • Delete nothing to keep the historical data for the symbols stored in the database (only missing historical data will be downloaded).
  • Delete only the data of the selected type and time interval to delete the symbols’ historical data for the selected data range, resolution and quote field.
  • Delete the data of the selected type completely to delete all the historical data stored in the database for the selected symbols.
I use the 2nd option (Delete only the data of selected...), and it seems to do what I want, but each time it creates a backup, which piles up days after days and uses a huge amount of disk space for nothing.

Am I doing things correctly and how do I disable the automatic backup ? if I choose the 1st option (Delete nothing), no backup is made, but my realtime data is not overwritten.

Thanks for your help !
Last edited by Mydesign on 20 Nov 2020, edited 1 time in total.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Historical Data Download Scheduler

Postby Mydesign » 16 Nov 2020

Anyone ?

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: Historical Data Download Scheduler

Postby Svetlana MultiCharts » 20 Nov 2020

Hi Mydesign,

To disable the automatic backup, choose the option Delete nothing.
Other options suppose that the backup is created automatically. We are going to improve this behaviour in one of the future versions (it will be possible to select the maximum number of backup files to store).
Delete nothing option does the following:
Historical Data Download Scheduler requests the data for the interval and type selected in the settings.
If the data is received from the provider, the new data replaces the data that was stored in the database before for the selected interval and type. If the data is not received from the provider, nothing changes.
Delete only the data of the selected type and time interval option does the following:
Data of the selected type and time interval is deleted.
Data backup is automatically created.
Historical Data Download Scheduler requests the data for the interval and type selected in the settings.
If the data is received from the provider, the new data is saved in the database. If the data is not received from the provider, there is a gap in the data of the selected type and time interval.
If it does not behave like described above, please reproduce the issue on one instrument. Send us logs via the main MultiCharts window -> Help -> Feedback.
In the description please add detailed description of the behavior, steps to reproduce the issue, exact time of each step.
Please select the options:
Attach a screenshot;
Attach log files…;
Include Backup of Logs from previous session;
Snapshot of the current MC setup;
Upload collected data to help desk;
uncheck other options and click Send.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Historical Data Download Scheduler

Postby Mydesign » 20 Nov 2020

Thank you Svetlana for your answer.

So according to your explanation I actually need to select Delete nothing to simply replace existing datas, which is what I want. No unwanted backup then. But for this option the wiki says: "only missing historical data will be downloaded". This is confusing :roll: How can we "replace" datas if only missing ones are downloaded ? Is this just a backfill then (filling missing gaps) ?

I have to say that chosen terminology for these options are a bit misleading...

Let's take an example:
(I assume that downloaded historical datas are complete, and previously collected datas are eventually incomplete)
(I assume that "replace" and "overwrite" mean the same)

- MC collects realtime datas (ticks) during day 1
- On day 2 scheduler is set to download historical datas (ticks) of day 1 with Delete Nothing option

Will it overwrite all previous datas or just download missing ones in this case ?

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: Historical Data Download Scheduler

Postby Svetlana MultiCharts » 04 Feb 2021

Hi Mydesign,

We updated the name of the option and description in the wiki.

Mydesign
Posts: 177
Joined: 15 Feb 2017
Has thanked: 32 times
Been thanked: 39 times

Re: Historical Data Download Scheduler

Postby Mydesign » 04 Feb 2021

Ok Thanks Svetlana.

"Refresh" is much more explicit than previous "Delete nothing" !


Return to “MultiCharts”