Hello
Before starting to code a routine to write to a text file, I want to make sure that in real life it could be possible.
The goal is to add a function to an indicator that is use by all my charts that under a given condition will write info to a text file. Since I may have 50 to 70 charts open and running under same instrument with different time scale, many of them are subject to have to write to the same text file at the same time.
Is there a risk of congestion? I so, any suggestion on how to prevent it
A good weekend to all
Martin
Writing to a text file: Risk of congestion??? [SOLVED]
Re: Writing to a text file: Risk of congestion???
You could write it with the fileappend command as a CSV file (ending in a new line of course) to do a fast test. I suspect there would be no problem. I have written from two or three charts and had no problem that I am aware of. However it was not critical data. it was debugging data. The only problem if it is critical is there is no return code to see if the write failed. You could maintain a count in a GV and each write would grab the count, add to it, update the GV with the new value then do the write with the count on the file as one of the fields. You would edit this count from excel or something. At least you would know if it was correct. They may get out of sequence however but I think that would be very rare.
- TJ
- Posts: 7746
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1033 times
- Been thanked: 2225 times
Re: Writing to a text file: Risk of congestion???
It should be not be a problem, because today's computer uses multi-core CPU; housekeeping duties (eg writing to text file) are handled by one core, you still have the other cores to perform your calculations.Hello
::
Is there a risk of congestion? I so, any suggestion on how to prevent it
::
Martin
A SSD would definitely help.
- TJ
- Posts: 7746
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1033 times
- Been thanked: 2225 times
Re: Writing to a text file: Risk of congestion???
ps. you can always write a test code, load it on 70 charts, then test run it on a papertrading account.
- Smoky
- Posts: 522
- Joined: 03 Dec 2010
- Location: Thailand
- Has thanked: 99 times
- Been thanked: 125 times
Re: Writing to a text file: Risk of congestion???
No need to do the test I have done for you, 10 charts with 15 indicators Writing in the same CSV file tick by tick, you will be suprised at how fast you fill your SSD and all the datas are clean (in my opinion this is one feature that works best)
Re: Writing to a text file: Risk of congestion???
Thanks SmokyNo need to do the test I have done for you, 10 charts with 15 indicators Writing in the same CSV file tick by tick, you will be suprised at how fast you fill your SSD and all the datas are clean (in my opinion this is one feature that works best)
The end goal is to have that file be written and read by both MC & Excel. MC to write and Excel Read and erase selected data in the file
OK for the SSD
Martin
- TJ
- Posts: 7746
- Joined: 29 Aug 2006
- Location: Global Citizen
- Has thanked: 1033 times
- Been thanked: 2225 times
Re: Writing to a text file: Risk of congestion???
SSD has limited write/erase life,
you will destroy the disk in short order if you are to write and erase repeatedly.
You should use RAM disk if the data is not needed permanently.
You can always save permanent data to harddisk if desired.
you will destroy the disk in short order if you are to write and erase repeatedly.
You should use RAM disk if the data is not needed permanently.
You can always save permanent data to harddisk if desired.
- Smoky
- Posts: 522
- Joined: 03 Dec 2010
- Location: Thailand
- Has thanked: 99 times
- Been thanked: 125 times
Re: Writing to a text file: Risk of congestion??? [SOLVED]
Sorry but MC write protect the CSV file, you can only read or make a copy to work,
The end goal is to have that file be written and read by both MC & Excel. MC to write and Excel Read and erase selected data in the file
Martin
but no writing/erasing with Excel
when you open this CSV file (in real time MC is writing) open office ask you to make a copy or to open it in read only mode
Re: Writing to a text file: Risk of congestion???
OK, My plan was to use .txt file. I will postpone the project.Sorry but MC write protect the CSV file, you can only read or make a copy to work,
The end goal is to have that file be written and read by both MC & Excel. MC to write and Excel Read and erase selected data in the file
Martin
but no writing/erasing with Excel
when you open this CSV file (in real time MC is writing) open office ask you to make a copy or to open it in read only mode
Martin
- JoshM
- Posts: 2195
- Joined: 20 May 2011
- Location: The Netherlands
- Has thanked: 1544 times
- Been thanked: 1565 times
- Contact:
Re: Writing to a text file: Risk of congestion???
While true, this shouldn't have to be a real concern: with writing 10GB/day, a 80GB SSD has over 18 years till write exhaustion, and larger SSD drives take even longer (source).SSD has limited write/erase life,
you will destroy the disk in short order if you are to write and erase repeatedly.
- JoshM
- Posts: 2195
- Joined: 20 May 2011
- Location: The Netherlands
- Has thanked: 1544 times
- Been thanked: 1565 times
- Contact:
Re: Writing to a text file: Risk of congestion???
There is a risk of congestion because it seems that MultiCharts batches the write to file commands. Which is understandable because, as I understand it, `FileAppend` opens and closes the file every time. So the `FileAppend` from Symbol A will need to wait till Symbol B has completed its `FileAppend` process. If there are only a few symbols, this works good, but with plenty a "traffic jam" occurs.The goal is to add a function to an indicator that is use by all my charts that under a given condition will write info to a text file. Since I may have 50 to 70 charts open and running under same instrument with different time scale, many of them are subject to have to write to the same text file at the same time.
At a certain point (on my pc with 8 charts and around 20 instances of the indicator below), the file is updated in (quick) steps but not fluently any more.
In some cases, there is also a line missing from the output, so writing to a file continuously does not seem very reliable, especially if you have a lot of charts (in my opinion).
Example indicator:
Code: Select all
{
Indicator: Example_WriteToFileAtTheSameTime
MultiCharts forum discussion topic: Writing to a text file: Risk of congestion?,
http://www.multicharts.com/discussion/viewtopic.php?f=1&t=46648
3-6-14
Note: Turn the 'Skip Identical Ticks' option off.
}
Variables:
fileName("C:\Temp\mcWriteTest.txt"),
stringToWrite("");
// No processing on non-realtime data
if (GetAppInfo(aiRealTimeCalc) <> 1) then
#return;
// Generate string
stringToWrite = Text(
Formattime("HH:mm:ss.", ComputerDateTime),
NumToStr(MilliSecondsFromDateTime(ComputerDateTime), 0),
";",
GetSymbolName,
";", NumToStr(Close, 5), ";",
NumToStr(Random(100), 0),
NewLine);
FileAppend(fileName, stringToWrite);
Code: Select all
18:08:13.699;EUR/USD;1.36220;3
18:08:13.711;EUR/USD;1.36220;61
18:08:13.711;EUR/USD;1.36220;88
18:08:13.726;EUR/USD;1.36220;26
18:08:13.738;EUR/USD;1.36220;76
18:08:14.487;EUR/USD;1.36221;39
18:08:14.488;EUR/USD;1.36221;57
18:08:14.503;EUR/USD;1.36221;2
18:08:14.519;EUR/USD;1.36221;2
18:08:14.534;EUR/USD;1.36221;23
18:08:14.548;EUR/USD;1.36221;84
18:08:15.234;USD/CAD;1.09140;64
18:08:15.248;USD/CAD;1.09140;45
18:08:16.731;AUD/CAD;1.01003;24
18:08:16.749;AUD/CAD;1.01003;52
18:08:17.481;GBP/USD;1.67504;74
18:08:17.496;GBP/USD;1.67504;96
18:08:18.885;EUR/USD;1.36221;85
18:08:18.885;EUR/USD;1.36221;3
18:08:18.900;EUR/USD;1.36221;46
18:08:18.900;EUR/USD;1.36221;10
18:08:18.915;EUR/USD;1.36221;71
18:08:18.929;EUR/USD;1.36221;71
18:08:19.244;USD/JPY;102.44600;54
18:08:19.264;USD/JPY;102.44600;90
18:08:19.388;USD/JPY;102.44600;78
18:08:19.401;USD/JPY;102.44600;97
18:08:19.994;EUR/USD;1.36221;86
18:08:19.994;EUR/USD;1.36221;5
18:08:20.17;EUR/USD;1.36221;94
18:08:20.32;EUR/USD;1.36221;9
18:08:20.45;EUR/USD;1.36221;90
18:08:20.58;EUR/USD;1.36221;93
18:08:20.289;GBP/CHF;1.50199;86
18:08:20.303;GBP/CHF;1.50199;60
18:08:20.541;USD/CAD;1.09140;60
18:08:20.553;USD/CAD;1.09140;70
18:08:20.695;GBP/USD;1.67504;47
18:08:20.709;GBP/USD;1.67504;36
18:08:20.724;GBP/USD;1.67504;11
18:08:20.742;AUD/NZD;1.09762;41
18:08:20.749;GBP/CHF;1.50203;72
18:08:20.756;GBP/CHF;1.50203;85
18:08:20.756;AUD/NZD;1.09762;30
18:08:20.788;EUR/USD;1.36218;40
18:08:20.790;EUR/USD;1.36218;34
18:08:20.792;AUD/CHF;0.82980;55
18:08:20.802;AUD/CHF;0.82980;1
18:08:20.803;EUR/USD;1.36218;49
18:08:20.805;GBP/USD;1.67504;90
18:08:20.815;EUR/USD;1.36217;61
18:08:20.815;GBP/USD;1.67504;82
18:08:20.816;EUR/USD;1.36217;74
18:08:20.828;GBP/USD;1.67504;45
18:08:20.828;EUR/USD;1.36218;77
18:08:20.840;EUR/USD;1.36217;14
18:08:20.851;GBP/CHF;1.50203;1
18:08:20.853;GBP/CHF;1.50203;13
18:08:20.853;EUR/USD;1.36218;70
18:08:20.867;EUR/USD;1.36217;53
But instead of just identifying an issue , I'd also like to offer three potential workarounds:
1) How many times per second (or per minute) would you like to update the file? If this is not continuous (say, every 5 seconds), perhaps you could create a global variable string that each indicator appends to. Then create one indicator that, once every five seconds, retrieves the GV string and write it to the file (and then reset it). But in that situation I'm note sure if there is absolutely no data loss.
2) Perhaps you could create a file for each symbol, and then read each file in a separate worksheet in Excel . Then you'll need one 'master worksheet' that combines them all.
3) You could batch the write to file. For example, don't write every tick update to the file immediately, but store all ticks updates in a string, and as soon as you have a certain amount of price updates, write this whole string in one go to the file. In that case, you bring down the amount of `FileAppend` open/close actions considerably. (This is the option I'd personally use).
But these three points are based on the assumption that you need to write to file a very often (i.e., continuously). If that is not the case, the odds are much smaller than in my example above that this happens at the same time.
So to summarise, if the `FileAppend` actions do not happen on the same millisecond (give or take 2-3 milliseconds), I would not be concerned about the risk of congestion.
The best test would be to use an indicator, like the one above, and open the file that is written to and see how fluently it updates. On my pc, time gaps of around 4-5 milliseconds were still being written fluently, but as soon as the write actions happened on the same millisecond, gaps started occurring and the file was updated in 'steps'.
(Edit: If the size of your string increase, so does the risk of congestion. When testing, write a string of the same length (or longer) to see how it would perform).
- Smoky
- Posts: 522
- Joined: 03 Dec 2010
- Location: Thailand
- Has thanked: 99 times
- Been thanked: 125 times
Re: Writing to a text file: Risk of congestion???
As I tell you, if you use this syntax there are not congestion ...
I think print command is buffered
Code: Select all
Vars: Enrg(""), Enrg2(""), Enrg3(""), FileLog("C:\CSV\LOG.csv");
Enrg = dat + ";" + hour +";" +order +";"+ numtostr(close,5) +";" ....+
print(file(Filelog),Enrg+Enrg2);