Save data - question

Questions about MultiCharts and user contributed studies.
User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Save data - question

Postby arnie » 14 Feb 2009

Hi.

I'd like an information regarding the way MC saves data.

Imagine that we call 20.000 5 minutes bars.
Does MC saves all this data automaticaly so we can access to it in offline mode or do we must have QuoteManager opened?

When MC is opened he's saving all data received from the opened charts, correct?

Is we want to collect the data without MC opened we connect the symbol in QM, correct?

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 14 Feb 2009

Hi arnie,

My understanding is this. Maybe someone can correct or refine this.

MC stores the data in a memory buffer while it is collecting data real time. It does not save it to disk until you close MC down. In some cases if MC aborts and gives you a proper error popup it will save it then as well.

I am not sure if you can collect data without the workspace open. I am not sure why you would want to do this anyway.

If you do not run MC for a few days, when you start it up again, it will backfill and catch up on what you missed. I am not sure if a portion of this is saved to disk before it starts collecting the real time data in memory if all of it is stored in memory until you close MC down.

The data is stroed in three files in this directory
C:\Documents and Settings\All Users\Application Data\TS Support\MultiCharts\Databases\

the three files are
FBPORTFOLIO.GDB
TSCACHE.GDB
TSSTORAGE.GDB

If you have a one user license it is okay to copy these files to another machine for offsite use and run MC on that machine instead to access these file. Notice I said "instead".

I am going to search for a post which will give you an udpate this with some details as to what is in these files.

John.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 14 Feb 2009

Below is the URL for the post I found by putting one of the database files into the search you find at the very top of this forum. Marina gives some detailed information in there. You might find more info if you read the other posts found by that search. I used "FBPORTFOLIO.GDB" for the search. It is a good way to get this sort of info on MC. For example Do a search on "reserve words" and you will find some extra commands such as "recalculate" which is not in the help. If you do this search and go down far enough you should find the document listing those extra commands.

http://forum.tssupport.com/viewtopic.ph ... tfolio+gdb

Here is a good post from the second page found searching on "reserve words"
http://forum.tssupport.com/viewtopic.ph ... erve+words

John.

drwar
Posts: 218
Joined: 31 Jul 2005

Postby drwar » 14 Feb 2009

John and Arnie

"I am not sure if you can collect data without the workspace open. I am not sure why you would want to do this anyway."

You can collect data without the workspaces open but then you have will have to turn any such symbols online in QM.

"If you do not run MC for a few days, when you start it up again, it will backfill and catch up on what you missed."

You would have to open up a chart with every symbol you collect in order for this to catch everything up.

J~

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 14 Feb 2009

Hi drwar

Am I corect here. Is what you are saying is that if one scans a lot of stocks for say long term hold trades but at the same time scalps one or two all day and has only two charts open daily for these then one might want to have the ones they are scanning turned on for download in the QM. If true, I guess one could run the mass download early in the morning before volume picks up enough for scalping during the day. Or maybe do the mass download after normal heavy volume during the day.

John.

drwar
Posts: 218
Joined: 31 Jul 2005

Postby drwar » 14 Feb 2009

[quote]Hi drwar

Am I corect here. Is what you are saying is that if one scans a lot of stocks for say long term hold trades but at the same time scalps one or two all day and has only two charts open daily for these then one might want to have the ones they are scanning turned on for download in the QM. If true, I guess one could run the mass download early in the morning before volume picks up enough for scalping during the day. Or maybe do the mass download after normal heavy volume during the day.

John.[/quote]

Turning them on is not a very good way to handle it. Because you need to shut them off when you want to trade and it becomes a lot of work.

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Postby arnie » 15 Feb 2009

Thank you all.

So, if MC saves all data when we close it, then, it's possible that in a few years we will have years of intraday data stored and ready to be analised in offline mode, correct?

So, imagine that we only daytrade in two products, ES futures and EURUSD cross. We have intraday charts of these two products and therefore, we know that their data are being collected and saved.

Now, we also do business in a couple of stocks. If we don't want to have their charts opened, the best way is to connect them in QM so we can build a database for later. Correct?


John, you said:
If you have a one user license it is okay to copy these files to another machine for offsite use and run MC on that machine instead to access these file. Notice I said "instead".

Does this mean that it's possible the install MC in two computers with just one licence?

I'd like to install MC also in my laptop so I could run offline tests while the desktop was running the realtime feed.

Is this possible?

Regards

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 15 Feb 2009

Hi arnie,

I'd like to install MC also in my laptop so I could run offline tests while the desktop was running the realtime feed.
No. With a one machine license they mean "you are only suppose to run one copy of MC at a time" It allows you to run it on a different machine but you are not suppose to be running it on two machines at the same time. If you want to run it on two machines at the same time you have to ask them what they want for this. I don't know since I do not backtest myself and have no need to run two copies at the same time. You need to respect this agreement too since it seems TSS have devised a way to know you are running it on two machines and they could shut if down on you. I don't think it is wise to go against this for many reasons including because if they go belly up this is not good for any MC users (existing and future). So most MC users won't support those who break this agreement anyway.

So, if MC saves all data when we close it, then, it's possible that in a few years we will have years of intraday data stored and ready to be analised in offline mode, correct?
Yes. I think with some data vendors you can buy historic data. I think even IB might have it.


Now, we also do business in a couple of stocks. If we don't want to have their charts opened, the best way is to connect them in QM so we can build a database for later. Correct?
Yes, that was my question to drwar. I think he answered it pretty clearly. You can do it (I have no idea how to do this myself) but he said it is a lot of work since during the trading day you want it shut off and you have to switch off all the other symbols. Sounds like a good feature request to me.

John.

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Postby arnie » 15 Feb 2009

No. With a one machine license they mean "you are only suppose to run one copy of MC at a time" It allows you to run it on a different machine but you are not suppose to be running it on two machines at the same time. If you want to run it on two machines at the same time you have to ask them what they want for this. I don't know since I do not backtest myself and have no need to run two copies at the same time. You need to respect this agreement too since it seems TSS have devised a way to know you are running it on two machines and they could shut if down on you. I don't think it is wise to go against this for many reasons including because if they go belly up this is not good for any MC users (existing and future). So most MC users won't support those who break this agreement anyway.

I asked this because, with Metastock, I'm was able to install it in two computers with only one license. Equis sales department told me at the time of purchase that could install it in two computers.

To run MC at the same time in two computers with realtime datafeed I'd need to have two realtime datafeeds subcriptions, something that I'm certain will never happen.

The purpose to have MC installed also in my laptop is for testing data offline and when I'm not at home.

All of this talk comes from what I've read in the thread that you posted above where Marina spoke about copying the files to another computer. This sentence leads us to speculate if it's possible to install MC in another computer with a single license or you need to pay for two licenses.

Anyway, this is not the main issue of this thread.

Regarding receiving stock quotes in QM, I confess I didn't understand what drwar said:
Turning them on is not a very good way to handle it. Because you need to shut them off when you want to trade and it becomes a lot of work.
If you have QM running 24/7 there should be no problem in collecting the data.

Why do you need to turn off the symbols?

Is there a problem having a chart opened in MC and that same symbol turned on in QM?

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 15 Feb 2009

For me I copy the files to the notebook, then shut MC down on the desktop (actually turn off the machine to save power) and use MC on the notebook offsite to maybe study the day, study my calls, my system or do some programming (whatever). I just get tired of being in the house since I am there all day trading. TSS has no problem with this use of MC.

I myself will probably never understand what drwar said since I follow the advice of these great traders.

Larry Williams said "focus is very important".
Linda Bradford said "consistency is very important".
John Hayden "never loose more than 1% on any one trade".

To implement this advice:

Focus: I focus only on two symbols (mostly one).

Consistent: My routine is well honed and never changes. Every day is the same. Nothing surprises me any more. Starting MC automatically opens one workspace for each symbol. No need to learn this QM trick. I use a checklist to stay consistent and I do it very very fast before every trade. Changing my trading system (to avoid human errors mostly) takes 5minutes max (no elaborate EL coding).

1%: I diversify by taking a max loss of 1% on any one trade, most of which are winners since I know the symbols so well and stay consistent in my approach via the checklist.

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Postby arnie » 15 Feb 2009

So what you're saying is that you have your MC (with 1 license) installed in two computers.

Good. The only difference is that my trading computer runs 24/7 though I restart it every weekend to clean memory and also clean possible viruses, bad registry entries and all sort of problems that windows can produce :wink:

So to run MC on my laptop I need to close it on my desktop or both can be open at the same time though the laptop will only work with offline data?

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 15 Feb 2009

...I am not sure if you can collect data without the workspace open. I am not sure why you would want to do this anyway.
...
I think it can. But you have to right click on the symbol to connect to the datafeed.

One easy way to test it out is to open QM only, then connect the symbol by tight click on it.
You can then see if data is coming through. If it does, you will see, both beside the symbol and in the log, how many quotes you are receiving.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 15 Feb 2009

... my trading computer runs 24/7 though I restart it every weekend to clean memory and also clean possible viruses, bad registry entries and all sort of problems that windows can produce :wink:
...
Be careful.
MC collects the data in a cache.
The data is written to the database when you EXIT the program.

if you experience a power outage, all the data in the cache are gone. MC has to re-download all the data you have collected since last reboot.


[Update on 20110722:]

the latest version of MultiCharts saves the real time data into a cache in the harddisk,
ie. even if there is a power interruption, the data you have collected is not lost; it can be recovered during restart of MultiCharts.

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Postby arnie » 15 Feb 2009

Be careful.
MC collects the data in a cache.
The data is written to the database when you EXIT the program.

if you experience a power outage, all the data in the cache are gone. MC has to re-download all the data you have collected since last reboot.
:shock:

Although re-download all data isn't a big issue, though it would be time consuming, there should be an option to save all collected data 'til that point.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 15 Feb 2009

Hi arnie,

Why do you run 24/7. Do you trade the overnight emini or have a big stack of backtesting prepared (or both). Maybe it would be worth betting a second license it it is backtesting.

I agree, it would be nice to save the data (maybe at lunch when time is not an issue). Having said that they may never change it since that would make running two computers with one license easier.

John.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 15 Feb 2009

Actually I just remember that there is software that comes with my UPS that can shut down applications if there is a power outage. I never installed it but maybe I should check into it. I run an IB bracket on all my trades so for me it would be a good thing. For those expecting MC to take you out maybe it would not help except to save cache to the database.
Last edited by bowlesj3 on 15 Feb 2009, edited 1 time in total.

User avatar
arnie
Posts: 1594
Joined: 11 Feb 2009
Location: Portugal
Has thanked: 481 times
Been thanked: 514 times

Postby arnie » 15 Feb 2009

Hi arnie,

Why do you run 24/7. Do you trade the overnight emini or have a big stack of backtesting prepared (or both). Maybe it would be worth betting a second license it it is backtesting.

I agree, it would be nice to save the data (maybe at lunch when time is not an issue). Having said that they may never change it since that would make running two computers with one license easier.

John.
Hi John.

I run my computer 24/7 because, first of all, it's better for the hardware. All components are working at the same temperature and this way I'm enhance the performance and the life of the computer.

Then of course I have access to the overnight trades in ES, NQ and forex markets.

Is not a question of trading during the overnight session, it's more a question of just turning on the monitors and having all the information already there.

I don't see why it would be difficult to save the data or what that has to do with having MC installed in two computers.
It's just a save option for the collected data like the one that exists for the workspace.

Regards.
Fernando

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 15 Feb 2009

You snuck a post in just a split second after I sent my post. See my prior post for the UPS info.

User avatar
JoshM
Posts: 2195
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1544 times
Been thanked: 1565 times
Contact:

Re:

Postby JoshM » 22 Jul 2011

... my trading computer runs 24/7 though I restart it every weekend to clean memory and also clean possible viruses, bad registry entries and all sort of problems that windows can produce :wink:
...
Be careful.
MC collects the data in a cache.
The data is written to the database when you EXIT the program.

if you experience a power outage, all the data in the cache are gone. MC has to re-download all the data you have collected since last reboot.
Thanks TJ - important point, and didn't know that. :)

How many symbols and fields do you guys collect with MultiCharts, and what is your memory usage when collecting real-time data?

I'm asking because I'm wondering how many symbols I can collect (Tick: Trade, Bid, Ask and Minute: Trade) simultaneously. Theoretically MultiCharts has enough memory to collect a lot of symbols, but do the incoming ticks get timestamped by MultiCharts or is this provided by the data feed? (Zen-Fire in my case)

I was thinking that, if MultiCharts timestamps the incoming ticks, collecting multiple symbols can give a lag, which results in wrong ticks in the database (is that assumption correct?). I don't see any way to test those timestamps of MultiCharts and data feed - so I'd like to hear your opinions on this.

Besides that, are there any values in the "Quotes collector status" (that thing besides the clock in the MultiCharts status bar) that are alarming, and suggesting that the tick data integrity being saved to the database is "lagging" price action? I assume any value > 1 seconds is troubling?

Thanks for any insights,

Regards,
Josh

sptrader
Posts: 742
Joined: 09 Apr 2010
Location: Texas
Has thanked: 483 times
Been thanked: 274 times
Contact:

Re:

Postby sptrader » 22 Jul 2011

... my trading computer runs 24/7 though I restart it every weekend to clean memory and also clean possible viruses, bad registry entries and all sort of problems that windows can produce :wink:
...
Be careful.
MC collects the data in a cache.
The data is written to the database when you EXIT the program.

if you experience a power outage, all the data in the cache are gone. MC has to re-download all the data you have collected since last reboot.
*****************************************************************************
** Every RT trader should have their computer on a UPS , so in the event of a power failure, you have time to properly shut-down MC, so no data is lost.

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: Re:

Postby TJ » 22 Jul 2011

... my trading computer runs 24/7 though I restart it every weekend to clean memory and also clean possible viruses, bad registry entries and all sort of problems that windows can produce :wink:
...
Be careful.
MC collects the data in a cache.
The data is written to the database when you EXIT the program.

if you experience a power outage, all the data in the cache are gone. MC has to re-download all the data you have collected since last reboot.
*****************************************************************************
** Every RT trader should have their computer on a UPS , so in the event of a power failure, you have time to properly shut-down MC, so no data is lost.
note:

the latest version of MultiCharts saves the real time data into a cache in the harddisk,
ie. even if there is a power interruption, the data you have collected is not lost; it can be recovered during restart of MultiCharts.


Return to “MultiCharts”