Is there any way ( or plan to implement) avoiding the need to save all the historical data required in the Quotemanager. When you have several hundred stocks in the database, it becomes a major effort in itself just to manage the data, whereas most data providers make adjustments for splits / dividend adjustments and other corporate actions automatically.
Is there not some way that the historical data could be called directly from the 3rd party vendors API in a timely manner.
I use Bloomberg with Multicharts Pro but, as a result of the effort required to maintain the Quotemanager database being so large and prone to errors (human), it is often easier just to delete all of the history and download again. In the case of Bloomberg, this can take many hours / days for several hundred stocks if downloading many years of history. It's the only way to maintain a clean database.
It seems to me that, given that most vendors already do this in their process, there should be some way of avoiding the manual management of the database, even if there is additional cost required.
Thanks
Quotemanager Splits and dividend adjustments
-
Andrew MultiCharts
- Posts: 1587
- Joined: 11 Oct 2011
- Has thanked: 931 times
- Been thanked: 559 times
Re: Quotemanager Splits and dividend adjustments
Hello aconnery,
At the moment if you request historical data from a data provider into MultiCharts (what means you are in Online mode and the "Download Missing Historical Data" box is checked in the Preferences), this data is saved to cache and later (upon close of the software) is saved to your database. If you want to replace historical data on all charts, you can use ".rld glob" in the command line when your charts are open. This will lead to re-downloading all requested on those charts data and will replace the historical data in the database. As you mentioned above, it may take some time. Righ not we don't have plans to change the behavior. Please leave us such feature request.
At the moment if you request historical data from a data provider into MultiCharts (what means you are in Online mode and the "Download Missing Historical Data" box is checked in the Preferences), this data is saved to cache and later (upon close of the software) is saved to your database. If you want to replace historical data on all charts, you can use ".rld glob" in the command line when your charts are open. This will lead to re-downloading all requested on those charts data and will replace the historical data in the database. As you mentioned above, it may take some time. Righ not we don't have plans to change the behavior. Please leave us such feature request.
Re: Quotemanager Splits and dividend adjustments
Getting historical data from your database should be in general be faster compared to getting it thru bloomberg API everytime. So, I think current implementation is better.Is there not some way that the historical data could be called directly from the 3rd party vendors API in a timely manner.
I use Bloomberg with Multicharts Pro but, as a result of the effort required to maintain the Quotemanager database being so large and prone to errors (human), it is often easier just to delete all of the history and download again. In the case of Bloomberg, this can take many hours / days for several hundred stocks if downloading many years of history. It's the only way to maintain a clean database.
As far as redownloading all the data that can take several hours/days, I suggest instead of downloading all the historical data, maybe you can just delete the data for current day and download the data for today only in the evening via bloomberg. This should be much faster (I guess a matter of minutes).
Btw, how large is your database?
Re: Quotemanager Splits and dividend adjustments
Many thanks for your input.
My database is around 1500 stocks with about 25 years of daily data. I only use end of day data for my tests, so at least that's not too large a requirement.
I agree with your point that saving in the database will always be much quicker, but the challenge is that when there's any corporate action, it's difficult to monitor 1500 stocks to ensure that the historical data is correct e.g. a 2:1 stock split will naturally create orders of the wrong size and potentially generate incorrect signals if the data is not adjusted.
I suppose one possibility would be for MC to have a function that can check the local database for all stocks with, for example, the last 10 days of data from the vendor with an alert list of 'mis-matches' i.e. where the vendors data does not agree with that saved in the database it is possibly due to a split or adjustment of some sort. This way it would be a flag which could pinpoint which stocks require a full historical download and those which do not. At least this would be a partial solution to minimise corporate actions risk to a portfolio.
Any thought appreciated.
My database is around 1500 stocks with about 25 years of daily data. I only use end of day data for my tests, so at least that's not too large a requirement.
I agree with your point that saving in the database will always be much quicker, but the challenge is that when there's any corporate action, it's difficult to monitor 1500 stocks to ensure that the historical data is correct e.g. a 2:1 stock split will naturally create orders of the wrong size and potentially generate incorrect signals if the data is not adjusted.
I suppose one possibility would be for MC to have a function that can check the local database for all stocks with, for example, the last 10 days of data from the vendor with an alert list of 'mis-matches' i.e. where the vendors data does not agree with that saved in the database it is possibly due to a split or adjustment of some sort. This way it would be a flag which could pinpoint which stocks require a full historical download and those which do not. At least this would be a partial solution to minimise corporate actions risk to a portfolio.
Any thought appreciated.
-
Andrew MultiCharts
- Posts: 1587
- Joined: 11 Oct 2011
- Has thanked: 931 times
- Been thanked: 559 times
Re: Quotemanager Splits and dividend adjustments
Please leave us such feature request.I suppose one possibility would be for MC to have a function that can check the local database for all stocks with, for example, the last 10 days of data from the vendor with an alert list of 'mis-matches' i.e. where the vendors data does not agree with that saved in the database it is possibly due to a split or adjustment of some sort. This way it would be a flag which could pinpoint which stocks require a full historical download and those which do not. At least this would be a partial solution to minimise corporate actions risk to a portfolio.
Any thought appreciated.
Re: Quotemanager Splits and dividend adjustments
The "problem" is that MC saves the data on a local database. As long as the data are saved to the database, MC doesn't know something has to be changed. That's where the feed and/or the user comes in.
First of all, the very very slow data delivery via Bloomberg API is not a problem on the MC side. It is just how fast the Bloomberg server "want" to deliver data in a specific time. Look at CQG. I can load all contracts till 2002 in the ES in max. 2 minutes (Min/Daily chart) in parallel to other symbols.
Second of all, it's easy to make your own calendar with alerts to figure out any and every split or dividend adjustments. Either way, if you are running strategies through dividend season, you always should be prepared. Please still be aware, anything & everything with MC happens local !!! So it's up to the user, mostly.
Third and last the overall problem is the MC database by itself and the idea behind it. May you remember the database with the GolbalServer from TS2000i where you where able to adjust data "just like that" on your own and local and without loading everything again and again. So if you could have access to the db and write a macro/script or something like that to adjust all the data by a specific diff, quotient etc. for a specific symbol and at a specific time , than it would be helpful for you. Not just for you !!! Imagine how many contracts you have to have for the "Custome Future" feature to get always well adjusted continues futures. And sometimes it calculates just wrong. Instead of having just 8 symbols with the same amount of data, you have to have 507 symbols for the same amount and same target what you want. Even the export of specific data for a specific time is close to impossible if you want it adjusted for a date 2-3 yrs ago .... How to adjust ? I wish you could just load the data like in the GS of TS2000i and adjust the data by CLICK CLICK CLICK to adjust the saved data and ready it is. I thought it would have changed in the last yrs, but since yesterday I know it better
So, all in all .... the db is the problem. But in your case it is not MC's fault that Bloomi API is so slow and it's just up to you to prepare yourself for the dividend and split days. Don't take it personally. PLEASE !!!
Thank you.
Ben
EDIT: Further you could export the data for the symbols you need to adjust on this day via sudy and already adjusted when you write it into a file. Than you delete the data in the db, close it, open it again and import the data from this file. Ok there are busy times, but earning money is not easy
Study :
First of all, the very very slow data delivery via Bloomberg API is not a problem on the MC side. It is just how fast the Bloomberg server "want" to deliver data in a specific time. Look at CQG. I can load all contracts till 2002 in the ES in max. 2 minutes (Min/Daily chart) in parallel to other symbols.
Second of all, it's easy to make your own calendar with alerts to figure out any and every split or dividend adjustments. Either way, if you are running strategies through dividend season, you always should be prepared. Please still be aware, anything & everything with MC happens local !!! So it's up to the user, mostly.
Third and last the overall problem is the MC database by itself and the idea behind it. May you remember the database with the GolbalServer from TS2000i where you where able to adjust data "just like that" on your own and local and without loading everything again and again. So if you could have access to the db and write a macro/script or something like that to adjust all the data by a specific diff, quotient etc. for a specific symbol and at a specific time , than it would be helpful for you. Not just for you !!! Imagine how many contracts you have to have for the "Custome Future" feature to get always well adjusted continues futures. And sometimes it calculates just wrong. Instead of having just 8 symbols with the same amount of data, you have to have 507 symbols for the same amount and same target what you want. Even the export of specific data for a specific time is close to impossible if you want it adjusted for a date 2-3 yrs ago .... How to adjust ? I wish you could just load the data like in the GS of TS2000i and adjust the data by CLICK CLICK CLICK to adjust the saved data and ready it is. I thought it would have changed in the last yrs, but since yesterday I know it better
So, all in all .... the db is the problem. But in your case it is not MC's fault that Bloomi API is so slow and it's just up to you to prepare yourself for the dividend and split days. Don't take it personally. PLEASE !!!
Thank you.
Ben
EDIT: Further you could export the data for the symbols you need to adjust on this day via sudy and already adjusted when you write it into a file. Than you delete the data in the db, close it, open it again and import the data from this file. Ok there are busy times, but earning money is not easy
Study :
Code: Select all
Input: Filename("D:/*.csv"), Add_Factor(0), Dec(2), Vers(1);
Input: StartDate(1140101), EndDate(1141008);
vars: init(true),out_str("");
If Date > StartDate and Date <= EndDate then
begin
if vers = 1 then // Tick Charts
begin
if init then
begin
init = False;
FileDelete(Filename);
FileAppend(Filename, "Date, Time, Close, Volume" + NewLine);
end;
out_str=elDateToString(Date) + ",";
out_str=out_str + NumToStr(Time_S,0) + ",";
out_str=out_str + NumToStr(C+Add_Factor,Dec) + ",";
out_str=out_str + NumToStr(Ticks,0) + ",";
// out_str=out_str + NumToStr(upTicks,0) + ",";
// out_str=out_str + NumToStr(downTicks,0) + ",";
FileAppend(filename,out_str + NewLine);
end;
if vers = 2 then // Minute or Daily Charts
begin
if init then
begin
init = False;
FileDelete(Filename);
FileAppend(Filename, "Date, Time, Open, High, Low, Close, Volume" + NewLine);
end;
out_str=elDateToString(Date) + ",";
out_str=out_str + NumToStr(Time_S,0) + ",";
out_str=out_str + NumToStr(O+Add_Factor,Dec) + ",";
out_str=out_str + NumToStr(H+Add_Factor,Dec) + ",";
out_str=out_str + NumToStr(L+Add_Factor,Dec) + ",";
out_str=out_str + NumToStr(C+Add_Factor,Dec) + ",";
out_str=out_str + NumToStr(Ticks,0) + ",";
// out_str=out_str + NumToStr(upTicks,0) + ",";
// out_str=out_str + NumToStr(downTicks,0) + ",";
FileAppend(filename,out_str + NewLine);
end;
end;
Re: Quotemanager Splits and dividend adjustments
Aconnery, MC not have Stock split's, all popular datafeed IQFeed, ESignal have split's information.
QuoteManager not have external packet file import, for example tick and minutes with stock's data, not have data redownloader if you have error in base in certain period.
Ps. New real-time portfolio trading may be have nice result with splited stock in previous day
QuoteManager not have external packet file import, for example tick and minutes with stock's data, not have data redownloader if you have error in base in certain period.
Ps. New real-time portfolio trading may be have nice result with splited stock in previous day
-
Andrew MultiCharts
- Posts: 1587
- Joined: 11 Oct 2011
- Has thanked: 931 times
- Been thanked: 559 times
Re: Quotemanager Splits and dividend adjustments
Hello Tcust,Aconnery, MC not have Stock split's, all popular datafeed IQFeed, ESignal have split's information.
QuoteManager not have external packet file import, for example tick and minutes with stock's data, not have data redownloader if you have error in base in certain period.
Ps. New real-time portfolio trading may be have nice result with splited stock in previous day
Please leave us such feature request as i kidly asked in my previous post.