Download strategy for historical data suggestions please

Questions about MultiCharts .NET and user contributed studies.
picante
Posts: 3
Joined: 20 Apr 2022
Has thanked: 4 times

Download strategy for historical data suggestions please

Postby picante » 24 May 2023

Hi,
I would like to download historical data using IQFeed to allow experimenting with strategies and validating them.
My initial thoughts are to just go to QuoteManager select all Symbols and download everything (tick, minute, day & ask bid trade) for the last 10 years.
Does someone have nay thoughts on a download strategy.
Which of those options are relevant for that scenario
From what I have see this could take some time, if there something I could be doing to make sure that the data is downloaded and if the machine restarts for example I don't have to start the process again.
I know it might depend on many factors like bandwidth, processing capacity, etc, limitations on IQ Feed but could someone share experiences on how long this has taken for you.
Any other info that you think might be relevant for someone experimenting like me, it would be appreciated.
Thanks!!!

HellGhostEvocatorX
Posts: 81
Joined: 10 Feb 2022
Has thanked: 52 times
Been thanked: 11 times

Re: Download strategy for historical data suggestions please

Postby HellGhostEvocatorX » 28 May 2023

Hello Picante. If you need the data for backtest, download it exactly as you suggested.
I can't say anything about IQ-Feed, but there are also some good free data sources on the Internet that you can use to check your scenarios.

I use Ducascopy and the FreeQuote data. Ducascopy offers free data for download, there are also some free download managers that can be used partially as a demo. The disadvantage here is that you have to laboriously import the data individually into the Quote Manager.

About backtesting in general: It depends on your trading window, which data, tick, minute or day you need (as well as your personal preferences).
If you were trading on 1h, 1min data should be sufficient (of course, more accurate is always better). Remember that you would never have achieved the exact same results in live trading anyway.
And the more accurate the data, the longer calculations take.

The more precise the data (the most precise is tick), the longer the download usually takes.

Why do you want to restart your PC? If you're afraid of rebooting, transfer data to the database now and then during the download
(ctrl +B) transfer.
There is also a setting that you could change so that the data is not/otherwise cached, there was a post on the forum somewhere about that.

For pure experimentation, the FreeQuotes should be sufficient if you want to test simple trading strategies and the download is relatively quick.

As I said, I can't say anything about IQFeed, but I might be able to help with other questions.


picante
Posts: 3
Joined: 20 Apr 2022
Has thanked: 4 times

Re: Download strategy for historical data suggestions please

Postby picante » 28 May 2023

Thank you HellGhostEvocatorX for taking the time. About the PC restarting or more generally I am worried of any unexpected events that could stop QuoteManager and could leave the download process in an expected state, I am worried about windows patching for example. Thanks I didn't know about the Ctrl +B option. Another question that you might know, is there a way of validating if there are gaps in data for example.
I've been running QM for a few days and this morning it crashed, I am not sure what I have to trigger to re-download.
Cheers

HellGhostEvocatorX
Posts: 81
Joined: 10 Feb 2022
Has thanked: 52 times
Been thanked: 11 times

Re: Download strategy for historical data suggestions please

Postby HellGhostEvocatorX » 28 May 2023

Gladly. Have a look at the Windows update: for Win10 under windows update --> advanced options --> restart the device as soon as possible => off

this should fix the problem, then there is also the update pause, which unfortunately is only temporary.

I also had the problem with the updates, and I think that has solved my problem so far. Otherwise, there are command line commands that can be used to find out the reasons for a restart, but I don't know them by heart either.

HellGhostEvocatorX
Posts: 81
Joined: 10 Feb 2022
Has thanked: 52 times
Been thanked: 11 times

Re: Download strategy for historical data suggestions please

Postby HellGhostEvocatorX » 28 May 2023

you can also find the ctrl +b option in the edit tab.

under right click edit data you can check data manually, I don't know of an automated solution, since the system cannot know which data should be available and therefore cannot compare whether data is missing.

It's best to make a backup copy from time to time so that you don't have to download everything from scratch if something goes wrong. If necessary, "Raid" hard disk technology is also something for you or you use cloud services.

Do you want to download the QuoteManager program again? Or the data since the crash?

I don't think there is something like "Download since last time" in the Quote Manager, it might be something for a feature request.

Otherwise, maybe not download too long periods at once, but e.g. only the year 2000, then the year 2001... So you know better where you were last if it crashes and only need 1 year to download again.

But maybe someone else knows better :)


Return to “MultiCharts .NET”