TS Data not storing Data slow to load

Questions about MultiCharts and user contributed studies.
User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

TS Data not storing Data slow to load

Postby Mark Brown » 03 Feb 2017

I have the local data storage box checked yet every time I load a chart it takes a long time to populate. Using TS data.

User avatar
Angelina MultiСharts
Posts: 260
Joined: 28 Dec 2016
Has thanked: 28 times
Been thanked: 66 times

Re: TS Data not storing Data slow to load

Postby Angelina MultiСharts » 06 Feb 2017

Hello Mark,

1. When you close MultiCharts and TS, please make sure that no processes remain in the Details tab of Task Manager;
2. To speed up the process it is recommended to have a chart for this symbol with the same resolution and data range plotted in TS;
3. Please also check that TS download scheduler (TS->View tab->Launch Download Scheduler) has not queued up MC data requests.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 11 Oct 2018

DATA IS NOT BEING STORED OR THE STORED DATA IS NOT LOADING. NEVER HAS ON ANY BUILD OF MC.

I SEE MANY PEOPLE HAVING THIS ISSUE YOU KEEP CLOSING THE THREAD WHEN YOU GOING TO FIX THIS?

DATA SHOULD BE STORED AND LOADED QUICKLY - MC HAS NEVER LOADED A CHART QUICKLY EVER!

DATA IS CHECKED TO BE SAVED - TS IS RUNNING ALL THAT YOU CONTINUALLY HERE FROM USERS WHO TELL YOU THIS PROBLEM AND YOU DO NOTHING.

MC IS NOT PULLING DATA FROM STORAGE. OR IT IS NOT STORING DATA.

automaton
Posts: 47
Joined: 04 Apr 2017
Has thanked: 2 times
Been thanked: 4 times

Re: TS Data not storing Data slow to load

Postby automaton » 12 Oct 2018

Mark

You think maybe your being a bit abrasive??

Based on all your complaints and dissatisfaction, I think it’s time you start developing your own custom platform.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

BACKFILLING DATA BUG!

Postby Mark Brown » 17 Oct 2018

Henry you can lock all the threads you want but the problem of backfilling data will not go away. If you can not get your team to fix this problem then just say so. Maybe we could crowdfund and hire some good programmers to help you. This problem MUST BE FIXED.

IF DATA IS BEING STORED WHY IS IT TAKING SO LONG TO PLOT?

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: BACKFILLING DATA BUG!

Postby hughesfleming » 17 Oct 2018

Hi Mark,

Are you using any kind of antivirus? Windows defender can slow things down significantly if the correct directories an not excluded. I update about a 100 charts everyday from IQFeed and it is quick. It is much quicker on the colocated server compared to doing the same thing at home so network speed does make a difference. Even then it is not something that is a big deal.

9/10 it is the antivirus that is giving people problems.

Alex

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018

Mark

You think maybe your being a bit abrasive??

Based on all your complaints and dissatisfaction, I think it’s time you start developing your own custom platform.
i am a bit abrasive because i have licensed two mc's and i have now forund them to be as riddled with bugs as where i come from "ts".

automated trading is not possible with mc

storing data locally has never worked because the charts load to slow. even "ts" beats out of mc for loading a chart.

so yea i am a little pissed.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: TS Data not storing Data slow to load

Postby Henry MultiСharts » 17 Oct 2018

Hello Mark,

I believe there is some misunderstanding. There are no outstanding issues with saving/loading the data from storage.
Nobody is locking any threads. Please specify the exact "closed threads with unresolved issues" you are referring to.
If you experience an issue - please come to our Live Chat to demonstrate it remotely.
Going forward please avoid using all caps when posting messages. Our discussion forum is for constructive dialogue.
Posting subjective unjustified critics or expressing your personal opinion in an offensive manner will not be tolerated.
Duplicated posts/threads related to the same subject and not providing any of the requested/new/constructive information will be deleted.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018

do yourself a favor and search backfilling - see the post of people having slow data loading issues? the data is not being pulled from a locally stored database is it?

the data is being pulled each and every time from ts server. no data is being stored - if it is being stored then why is it so slow to make a chart?

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018

Hello Mark,

1. When you close MultiCharts and TS, please make sure that no processes remain in the Details tab of Task Manager;
2. To speed up the process it is recommended to have a chart for this symbol with the same resolution and data range plotted in TS;
3. Please also check that TS download scheduler (TS->View tab->Launch Download Scheduler) has not queued up MC data requests.
if data is being stored locally > why would i also need to have a ts chart up of the same resolution? please answer that.

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: TS Data not storing Data slow to load

Postby hughesfleming » 17 Oct 2018

Mark, why don't you try and get an IQFeed demo up and running and see if you have the same issues that you do with TS data. Disable windows defender or any other antivirus temporarily. If I were having your problems, I would even consider a fresh install of Windows on a different machine. You could rent a colocated Windows 2012R2/2016 instance for a month and see if that works differently than your local machine.

I know you are frustrated but not everyone is having your problems. There is something specific about your setup that is causing your headaches.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018

Mark, why don't you try and get an IQFeed demo up and running and see if you have the same issues that you do with TS data. Disable windows defender or any other antivirus temporarily. If I were having your problems, I would even consider a fresh install of Windows on a different machine. You could rent a colocated Windows 2012R2/2016 instance for a month and see if that works differently than your local machine.

I know you are frustrated but not everyone is having your problems. There is something specific about your setup that is causing your headaches.
iq feed has no history so of course it will load like quickly for you. probably you use daily or standard time intervals, i use tick and range charts.

ts has a setting to build range bars as 1 second to minutes - mc has no such settings so they build each range bar for 1 tick level.

i guess mc does not save tick data and so they have to load all the tick data each time a chart is loaded.

i do not have windows defender or any anti virus on my computer it's behind a mikrotik firewall.

sptrader
Posts: 742
Joined: 09 Apr 2010
Location: Texas
Has thanked: 483 times
Been thanked: 274 times
Contact:

Re: TS Data not storing Data slow to load

Postby sptrader » 17 Oct 2018

Mark, why don't you try and get an IQFeed demo up and running and see if you have the same issues that you do with TS data. Disable windows defender or any other antivirus temporarily. If I were having your problems, I would even consider a fresh install of Windows on a different machine. You could rent a colocated Windows 2012R2/2016 instance for a month and see if that works differently than your local machine.

I know you are frustrated but not everyone is having your problems. There is something specific about your setup that is causing your headaches.
iq feed has no history so of course it will load like quickly for you. probably you use daily or standard time intervals, i use tick and range charts.

ts has a setting to build range bars as 1 second to minutes - mc has no such settings so they build each range bar for 1 tick level.

i guess mc does not save tick data and so they have to load all the tick data each time a chart is loaded.

i do not have windows defender or any anti virus on my computer it's behind a mikrotik firewall.
"iq feed has no history" ?

How Much Historical Data Is Available?
180 calendar days of tick (every trade)
10+ years of 1 minute historical bars
15+ years of daily historical bars

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: TS Data not storing Data slow to load

Postby hughesfleming » 17 Oct 2018

Are you sure it is not your firewall setup? Did you do it yourself?

https://forum.mikrotik.com/viewtopic.php?t=98291

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018

Are you sure it is not your firewall setup? Did you do it yourself?

https://forum.mikrotik.com/viewtopic.php?t=98291
answer this question -

is data being stored locally or not? if it is then what would a firewall have to do with anything?

my mikrotik setup is not causing historical data from being pulled off my hard drive where mc says it's being stored.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 17 Oct 2018


How Much Historical Data Is Available?
180 calendar days of tick (every trade)
ts can make a 1 point range bar chart of the es going back to 97 in a flash once you have created it once.

mc on the other hand struggles to make a 1 point range bar chart of the es just one year.

this is because ts can compile the ticks and mc can not - they have chosen to not address the backfilling bug by fixing it but rather ignoring it.

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: TS Data not storing Data slow to load

Postby hughesfleming » 17 Oct 2018

Data is being stored locally. If it has to backfill then the new data comes from IQFeed. If I close a tick chart and re-open it....the chart comes up instantly.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 18 Oct 2018

Data is being stored locally. If it has to backfill then the new data comes from IQFeed. If I close a tick chart and re-open it....the chart comes up instantly.
how much data are you loading? because mine and others are waiting for up to and over 10 minutes for data to load.

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: TS Data not storing Data slow to load

Postby hughesfleming » 18 Oct 2018

For this test.... 1000 tick bars, 10000 bars. 18 seconds to backfill on a freshly started MC. This is to a datacentre in Frankfurt so you should be able to do better being in the US.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 18 Oct 2018

For this test.... 1000 tick bars, 10000 bars. 18 seconds to backfill on a freshly started MC. This is to a datacentre in Frankfurt so you should be able to do better being in the US.
here is what i know "if data is being stored locally as they claim" even your chart should not take 18 seconds to load.

even ts beats 18 seconds to load 100,000 bars so there is something bad wrong with mc back-filling as numerous people have complained right here on the forums many many times without mc offering anything other than load the same chart up in ts. lol

hughesfleming
Posts: 275
Joined: 22 Apr 2014
Has thanked: 70 times
Been thanked: 72 times

Re: TS Data not storing Data slow to load

Postby hughesfleming » 18 Oct 2018

Mark, you are missing the point. Only some of that data is in the database. It had to get the new data from IQFeed. This was not a chart I use for my trading. It was to give you an example using tick data and to give you an idea how long it could take to get new data from IQFeed. Data that is in the database loads instantly....please read that again....INSTANTLY. Regardless 18 seconds is far away from your 10 minutes.

You need to look at your setup. Something is not working and no one here is claiming to have your problems. This forum would be bursting with complaints if everyone had the same issues that you do.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: TS Data not storing Data slow to load

Postby Henry MultiСharts » 19 Oct 2018

Mark, I’m afraid that your understanding of MultiCharts operation is incorrect. Let me clarify it for you:

Once historical data is downloaded from the data feed server and plotted on the chart - it is stored in the cache. When you close MultiCharts, QuoteManager and Portfolio Trader - cached historical data is saved to the local database automatically:
https://www.multicharts.com/trading-sof ... s_Database
For TS data provider the corresponding option “Save data to local database” also needs to be enabled:
http://www.multicharts.com/trading-soft ... adeStation
The option comes into effect after restarting all MultiCharts processes.

Since version 9.0 (in 64 bit edition only) the operation speed of MultiCharts has been increased by using RAM Cache (in 64 bit MC only). It means there is no file cache and if the application is shutdown incorrectly (crash, freeze, PC power outage, etc) the data from the cache will be lost.
In MultiCharts 12 we have added an option to “Flush Cached Data to Database” manually. You can find it in Multicharts->File menu or QuoteManager->Edit menu.
If you want to get back to file cache you need to run the registry editor and go to the following path:
HKEY_CURRENT_USER\Software\TS Support\MultiCharts64\Storage\UseFileCache
By default UseFileCache = 0 (file cache is disabled). You need to set UseFileCache = 1 to enable the file cache. When the file cache is enabled MultiCharts will try to restore the data on the next run in case the application crashed.

You can view the locally stored data using the Edit Data option of QuoteManager:
https://www.multicharts.com/trading-sof ... iting_Data
The time it takes to load the data from the database depends on the number of instruments your request, resolution, data range, and your hardware.
You can test loading the data from the local database by running MultiCharts in Offline mode:
https://www.multicharts.com/trading-sof ... erver_Mode
In Online mode receiving the missing data from the data provider depends on the same factors + the speed of the data provider returning the data. That was suggested to you to use the TS download scheduler for checking if TS adds data requests from MC to the queue, slowing data the data download procedure.
That was also suggested to run the same chart in TS in order to be able to reload it, if TS does not provide the data for it to MultiCharts, or in case the data has gaps.
Once again - if you experience an issue, or you think that the data is loading slow, please come to our Live Chat to demonstrate it remotely.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 15 Nov 2018

I am afraid you do not understand - MY DATA IS NOT BEING STORED. I was using TS before there was MultiCharts and I know TS inside and out ask Dennis. I know all about everything you posted, I understand all about what you are saying.

MY DATA IS NOT BEING STORED - OR - MULTICHARTS IS NOT PULLING THAT DATA TO CHART.

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: TS Data not storing Data slow to load

Postby wilkinsw » 16 Nov 2018

I am afraid you do not understand - MY DATA IS NOT BEING STORED. I was using TS before there was MultiCharts and I know TS inside and out ask Dennis. I know all about everything you posted, I understand all about what you are saying.

MY DATA IS NOT BEING STORED - OR - MULTICHARTS IS NOT PULLING THAT DATA TO CHART.
Mark, i think it’s fair to say it’s now down to you to prove your claim. Upper case won’t help your cause bud. ;)

wilkinsw
Posts: 662
Joined: 21 Apr 2013
Has thanked: 154 times
Been thanked: 104 times

Re: TS Data not storing Data slow to load

Postby wilkinsw » 16 Nov 2018

Henry (or other users),

If i export a qmd file will it include the most recent data that’s held in the cache? Or will it only hold the data since the last restart (or last flush, if Mc12).

Will

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: TS Data not storing Data slow to load

Postby TJ » 16 Nov 2018

Henry (or other users),

If i export a qmd file will it include the most recent data that’s held in the cache? Or will it only hold the data since the last restart (or last flush, if Mc12).

Will
You must exit MC in order to flush the cache to the database.

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: TS Data not storing Data slow to load

Postby Henry MultiСharts » 20 Nov 2018

MY DATA IS NOT BEING STORED - OR - MULTICHARTS IS NOT PULLING THAT DATA TO CHART.
Mark,

None of the other users experience anything similar. As previously mentioned - please come to our Live Chat for the remote support.

Kel
Posts: 19
Joined: 27 Nov 2017
Has thanked: 15 times
Been thanked: 5 times

Re: TS Data not storing Data slow to load

Postby Kel » 01 Nov 2020

Henry - I experience the same problem Mark Brown is having all the time - I'm saving 20 years of 1 minute data for system development from TS - I understood that cached data was stored to local database when I exit the chart/workspace but every time I reopen my workspace with the chart I go through the same process which can take up to an hour even when I prioritize backfiling from the TS side. I would like to open other charts in other workspaces using the saved data as well.

Has this been resolved or if is not a problem what I would like to see please, is some simple straight foward steps to accomplish this that is
After I backfill my chart -
1) how do i save the cache locally
2) where is that data stored so I can confirm that.
3) is there anything special to do when I start up MC and load that chart again (without have to log into TS if possible)

Thanks,

Ed

p.s. I have posted on this issue before. Mark Brown is very experienced trader and knowledgeable (based on reading posts and knowing he developed odd ball system and released that to the public years ago). I have been trading via Neoticker for a long time and develop software for the aerospace industry professionally - While I know I make mistakes and sometimes miss understand very basic concepts I believe it should be a simple straight forward process and I am happy to discover it is.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 01 Nov 2020

Henry - I experience the same problem Mark Brown is having all the time - I'm saving 20 years of 1 minute data for system development from T
mc is really just a charting program with a packaged translation database they call quotemanager, which is not a real data server.

mc will never fix this problem because they will not admit that it exist, you can send them videos, they will login and try and help you. they will delete your post asking for help or close the thread claiming its fixed. but they will never listen to you because you and i are the only two user that have complained about this i guess.

mc is a pretty charting package it is not much beyond that and never will be a real trading platform. they don't understand that data is number one because without it charting doesn't matter. you can't rely on mc for real trading, the order entry system has bugs, and charting can't chart data it can't pull from the translation database. running ts and mc side by side you can see what a joke mc is even though it's 64bit as compared to ts 32bit.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: TS Data not storing Data slow to load

Postby TJ » 01 Nov 2020

Henry - I experience the same problem Mark Brown is having all the time - I'm saving 20 years of 1 minute data for system development from TS - I understood that cached data was stored to local database when I exit the chart/workspace but every time I reopen my workspace with the chart I go through the same process which can take up to an hour even when I prioritize backfiling from the TS side. I would like to open other charts in other workspaces using the saved data as well.

Has this been resolved or if is not a problem what I would like to see please, is some simple straight foward steps to accomplish this that is
After I backfill my chart -
1) how do i save the cache locally
2) where is that data stored so I can confirm that.
3) is there anything special to do when I start up MC and load that chart again (without have to log into TS if possible)

Thanks,

Ed

p.s. I have posted on this issue before. Mark Brown is very experienced trader and knowledgeable (based on reading posts and knowing he developed odd ball system and released that to the public years ago). I have been trading via Neoticker for a long time and develop software for the aerospace industry professionally - While I know I make mistakes and sometimes miss understand very basic concepts I believe it should be a simple straight forward process and I am happy to discover it is.
Can you verify you have done the following steps:

After exiting MultiCharts,
before you shut down the computer,
check the TaskManager to make sure you do not see any MultiCharts modules on the list.
ie MultiChars have completed its operation, including saving your data to the harddisk, and have exited the computer.


Image
Attachments
MC TaskManager.jpg
(134.53 KiB) Not downloaded yet

Kel
Posts: 19
Joined: 27 Nov 2017
Has thanked: 15 times
Been thanked: 5 times

Re: TS Data not storing Data slow to load

Postby Kel » 01 Nov 2020

TJ - I looked in quote manager and opened the "edit data window" a symbol I was interested in and loaded 1 minute data for a symbol (ES.D) I was interested in and I did have the data go all the way back to early 2000. I then exited Multicharts and made sure nothing was running including the TS server. I the restarted MC (but did not reboot my computer) and logged into TS choosing to work off line. I then opened the charts with the symbols I wanted and it loaded - it took awhile may 10 minutes in all. I did watch the Event log in the quote manager window - and it was making requests for 1 minute data - since I chose working offline I assume the data came from the saved cache.

Anyway the charts loaded much faster than when I have backfilled directly from TS.

So John Brown it seems to work as least the way I did it - still on the slow side - but my guess within 10 minutes (I did not time it)

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: TS Data not storing Data slow to load

Postby TJ » 02 Nov 2020

TJ - I looked in quote manager and opened the "edit data window" a symbol I was interested in and loaded 1 minute data for a symbol (ES.D) I was interested in and I did have the data go all the way back to early 2000. I then exited Multicharts and made sure nothing was running including the TS server. I the restarted MC (but did not reboot my computer) and logged into TS choosing to work off line. I then opened the charts with the symbols I wanted and it loaded - it took awhile may 10 minutes in all. I did watch the Event log in the quote manager window - and it was making requests for 1 minute data - since I chose working offline I assume the data came from the saved cache.

Anyway the charts loaded much faster than when I have backfilled directly from TS.

So John Brown it seems to work as least the way I did it - still on the slow side - but my guess within 10 minutes (I did not time it)
If you want to work off-line,
you need to go to Preferences and select to run the Data Server in Offline mode.

You can access the Preferences from either the Quote Manager, or in the MultiCharts program.

Image
Attachments
preferences offline.jpg
(68.97 KiB) Not downloaded yet

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 19 Nov 2020

so my box looks different than your box 14 - build 23905

so what your saying tj is you have a choice of being on line or being off line ?

btw none of the little help buttons work at all they give no info or help - all i know is everytime i load mc it has to load all the data from the data vendor ts. it never stores any data - so then what is the use of flushing the data.

Image


also notice i have save data checked here so why will it not save the data for faster reload next time it's opened?

Image



ps i see the phantom trade thing has been fixed supposedly after adding 100's of contracts onto my overnight order, they never admitted to the bug just figured out to fix it. whatever i just want the slow data loading thing fixed now.

next i want someway to use mc to auto trade if it has to send a hot key out to a real trading platform.

User avatar
Mark Brown
Posts: 181
Joined: 29 Nov 2016
Has thanked: 111 times
Been thanked: 17 times

Re: TS Data not storing Data slow to load

Postby Mark Brown » 23 Nov 2020

again today running ts and mc side by side range bars can not keep up on mc platform. i have to reload maybe this is a problem unique to range bars?


Return to “MultiCharts”