A Native C# Solution for Global Storage

Questions about MultiCharts .NET and user contributed studies.
L Lewis
Posts: 2
Joined: 24 Aug 2013
Has thanked: 1 time
Been thanked: 20 times

A Native C# Solution for Global Storage

Postby L Lewis » 03 Sep 2013

Global storage is sometimes useful in MultiCharts .NET as it is with regular MultiCharts. This has been discussed in other threads, and it is certainly possible to use TS's GlobalVariable.dll (a version is already supplied with the EasyLanguage version of MultiCharts) or other external dll for this purpose with MultiCharts .NET as discussed in this thread: viewtopic.php?f=19&t=11315

Since MultiCharts .NET makes use of C#, there are several ways to implement a global storage solution within C# itself without resorting to GlobalVariabe.dll . A native C# solution has the advantage of being convenient for the user to modify as desired. After some thought, I decided to implement an approach using the ConcurrentDictionary class, one of several thread-safe collections added by the System.Collections.Concurrent namespace introduced in .NET Framework 4. This has worked out well for me thus far, so I thought I'd share the technique and my Globals class for the MC .NET community to use.

A ConcurrentDictionary works by specifying key/value pairs just like the regular generic Dictionary type available prior to .NET 4. We can create a static Globals class to store such dictionaries for access from any indicator or strategy in MultiCharts .NET. As an example, to create a ConcurrentDictionary called NamedInteger for storing integers that will be accessed by a string name, we could do it in the following way inside the Globals class:

Code: Select all

static ConcurrentDictionary<string, int> NamedInteger = new ConcurrentDictionary<string, int>();
Later in some other code, if we want to make use of this Globals class and add to the NamedInteger dictionary an entry for some number named "Length" having a value of 4, we could simply do this:

Code: Select all

Globals.NamedInteger["Length"] = 4;
To retrieve this value, assuming it already exists in the dictionary, we could simply do this:

Code: Select all

int myLength = Globals.NamedInteger["Length"];
If we want to create a dictionary to store integers that are accessed by a number index instead of a string name, we can create it like this:

Code: Select all

static ConcurrentDictionary<int, int> NumberedInteger = new ConcurrentDictionary<int, int>();
The ConcurrentDictionary class has a rich set of methods for setting and accessing dictionary values in order to account for different situations. For example, if we want to set myLength based on the dictionary entry stored with the "Length" key as above, but we don't know whether or not the "Length" key exists, then we could use the following technique to access the dictionary value if the key exists but use a default value if it doesn't:

Code: Select all

int myLength = 10; // Default value
int retrievedValue = 0;
if (Globals.NamedInteger.TryGetValue("Length", out retrievedValue))
{
myLength = retrievedValue;
}
For a complete reference on the ConcurrentDictionary class, see the MSDN Library entry here: http://msdn.microsoft.com/en-us/library/dd287191.aspx

The C# Globals class I've created stores a set of dictionaries that provide global storage for the same data types supported by TS's GlobalVariable.dll (int, float, double, bool, and string). These dictionaries are completely independent of any MC .NET types, but I've placed the class in the PowerLanguage namespace for easy access without requiring a "using" clause. The dictionaries were created in a manner similar to the examples shown above, but using properties for accessing the dictionaries. Since each property simply returns a ConcurrentDictionary, the syntax for using a property for access is identical to the examples shown above for directly accessing a dictionary itself. There are five dictionaries accessed by string name, and five that are accessed by a numeric index:

NamedInteger
NamedFloat
NamedDouble
NamedBool
NamedString
NumberedInteger
NumberedFloat
NumberedDouble
NumberedBool
NumberedString

The attached "globals.zip" file contains the Globals class in a couple of alternative forms that you can use to install it for MC .NET use. Unzip the file to any convenient temporary folder, then choose one (not both at the same time!) of the following approaches:

1. Within the MC .NET editor, import the "Globals.pln" file. The Globals class will then be imported as a MC .NET "function", although it is not a function. We're just using this as a means to sneak some code into MC .NET for compilation, as described in the "MultiCharts .NET Programming Guide" on page 86 (see steps under item #2 on that page). As noted in the Guide, such "functions" will always be colored in red in the MC .NET editor, indicating not compiled. This is OK according to the Guide, since we're just using the file for code storage. I recommend using "Recompile All" (CTL-F7) any time you modify the Globals class or any indicators and signals using the class. Having the actual code for the Globals class, as we accomplish with this approach, is useful so that you can see how it is implemented and also to experiment with changes to it if desired. However, as noted on page 87 of the Guide, using a MC .NET function file for storing general code can have some undesired compilation side effects in some situations. If those side effects turn out to be a problem for you, then use approach #2.

2. Instead of approach #1, you can simply use the Globals.dll assembly provided in the zip file (or just create it yourself using Visual Studio). Just place the assembly in any folder you like (the MC .NET installation folder is convenient), and add a reference to it from the MC .NET editor before you use it. It works with both 32-bit and 64-bit MultiCharts and Windows.

Be aware that using any kind of global storage solution in a multi-threaded environment requires careful thought. Although the ConcurrentDictionary type is specifically designed to be thread-safe and optimized for multi-threaded use, being thread-safe only means that the collection itself is safe for multi-threaded access. If you are allowing code from multiple threads to read and write to the exact same dictionary keys within a dictionary, you may also need to work out your own thread synchronization code and special locks. However, for the most common usage cases for global storage in MC .NET, you should be able to use the dictionaries provided by the Globals class and use the built-in ConcurrentDictionary methods without needing to add any complex synchronization code.

I've found the Globals class to be a good global storage solution for my purposes in MultiCharts .NET, and I've successfully used it in a recent project when re-engineering EasyLanguage code to C#, where the EasyLanguage code had used TS's GlobalVariable.dll for global storage.

In a separate post to this thread, I'll provide an example of how to use the Globals class to accomplish sharing of settings between a strategy and an indicator.
Attachments
globals.zip
Globals class
(4.81 KiB) Downloaded 661 times

L Lewis
Posts: 2
Joined: 24 Aug 2013
Has thanked: 1 time
Been thanked: 20 times

Re: A Native C# Solution for Global Storage

Postby L Lewis » 03 Sep 2013

Globals class example: sharing strategy settings with an indicator."

The sharing of a strategy's settings with a cooperating indicator is one of the more useful roles for global storage in MC .NET (as it is with regular MultiCharts and TS). I believe MultiCharts 9.0 Beta 1 is going to add a capability for plotting directly from strategies. I don't know if it will also provide a way for an independent indicator to access properties of a strategy running in the same chart. In any event, this situation can be easily handled by using global storage. Historically the TS GlobalVariable.dll has been widely used for this purpose in TS and in regular MultiCharts. The native C# Globals class that I posted at the beginning of this thread works well for this use-case, and this post will present an example.

For the example, I've built on the "TrendPower" tutorial strategy and indicator from Chapter 3 of the "MultiCharts .NET Programming Guide". Most folks who are using MC .NET will be familiar with the TrendPower tutorial. Keep in mind that TrendPower is not a real technical indicator - it was created by the authors of the Guide just to demonstrate coding techniques for MC .NET. It's ideal for my purposes in this demo code.

First, make sure you have installed the Globals class using one of the two approaches I described in my initial post in this thread. Then, import the attached "GlobalsDemoTrendPower.pln" file via the MC .NET editor. You should then be able to see the GlobalsDemoTrendPower strategy and GlobalsDemoTrendPower indicator in navigation pane of the MC .NET editor.

In the GlobalsDemoTrendPower strategy, a SaveGlobalProperties option is provided to save selected input property settings to the global dictionaries for later retrieval and use by the GlobalsDemoTrendPower indicator. The global dictionary saves are accomplished within each property's setter function. There are three properties that are saved to global storage: FastLength, SlowLength, and StrongLevel. In this example, the global name (dictionary key) for each property was created by appending to the property name a chart-dependent string ID via Environment.ChartWindowHWND.ToString(). In this way, multiple instances of the GlobalsDemoTrendPower strategy running in separate charts and perhaps running at different time intervals can save unique settings that can later be retrieved by an indicator running in the same chart. If you want to have indicators be able to retrieve values saved by a strategy running in a different chart, you can modify the code for both strategy and indicator to simply have the user enter a unique desired name for each globally shared property instead of tying the name to ChartWindowHWND.

To save values to the Globals class dictionaries from the strategy, I've used the ConcurrentDictionary AddOrUpdate method. This method has a couple of overloads. The one I've used provides for creating a new dictionary entry if the dictionary key does not exist and then setting it to a specified value. If the key already exists, the value is updated based on a user-specified delegate function that can use the key and existing value (the k, v arguments) in the computation of a new value. In my simple example, since the delegate function does nothing more than return the same specified value used when the key does not exist, we could have just simply used dictionary[key] = value instead of using the AddOrUpdate method. However, having the AddOrUpdate example in place allows you to replace my simple delegate value assignment with a more complicated code block if you like.

Note that the AddOrUpdate method is thread-safe just like all the other ConcurrentDictionary methods. However, it is not atomic since the delegate is invoked outside of the dictionary's internal lock. This approach was implemented intentionally by the .NET Framework 4 designers so that any unknown code would not block all threads (see notes at http://msdn.microsoft.com/en-us/library/dd997369.aspx). In this use-case and most others that we would be interested in for MC .NET use, this behavior is fine, and is just how we would want it to work.

The GlobalsDemoTrendPower indicator has a UseGlobalProperties option to allow it to use the global property values saved by the strategy instead of the local values set manually by the user. The indicator is just a consumer for the global dictionary values and does not save any values to the dictionaries. When a user manually sets a value for a property having a globally shared option, this value is saved in the local property backing store and is only used by the getter function when UseGlobalProperties == false.

In the indicator, when UseGlobalProperties == true, a snapshot of property values from the global dictionaries is saved to a special set of property backing stores just for globals each time StartCalc is called. From then on, each globally shared property uses the saved snapshot value instead of accessing the global dictionary each time the property getter function is called. In this way, execution speed is almost the same regardless of whether or not global sharing is used.

There is an option in the indicator called DoPeriodicRefresh, along with a selection for RefreshIntervalSeconds. If DoPeriodicRefresh is set to true, the indicator will recalculate the entire series periodically at the interval specified by RefreshIntervalSeconds. Without this, if historical data is being used, even if both strategy and indicator are set to use global sharing, the indicator will not respond to property changes in the strategy because after the last bar has been calculated and plotted there is no event forcing the indicator to re-plot its values. Even with real-time data, a recalculation of the entire plot is normally only done when needed. So enabling DoPeriodicRefresh will ensure that if you make changes to one or more globally-shared properties in the strategy, the changes will soon be picked up by the indicator and reflected in the recalculated plot. You can also force the indicator to recalculate its values by turning it off and then on again (click the indicator name twice) without using the periodic refresh.

To see how the strategy and indicator work together, insert them both in a chart. In the strategy set SaveGlobalProperties = true and in the indicator set UseGlobalProperties = true and set DoPeriodicRefresh = true. Next, make changes to any of the three globally shared properties in the strategy. You should see the plot in the indicator pane change in response. Note that the displayed setting for FastLength, SlowLength, and StrongLevel in the indicator's status text will show the manual setting from the input dialog box, and not the one picked up from global storage. Currently, as far as I know, there is not a way to force the input dialog box setting to change to reflect a property value that has been modified in the code. A capability for such synchronization in the future would be a welcome addition to MC .NET.

I hope the Globals class and the example provided here will be useful to you in designing your strategies and indicators. Enjoy!
- Larry
Attachments
GlobalsDemoTrendPower.pln
(5.99 KiB) Downloaded 821 times

gztanwei
Posts: 29
Joined: 15 Aug 2015
Has thanked: 6 times
Been thanked: 5 times

Re: A Native C# Solution for Global Storage

Postby gztanwei » 05 Oct 2015

Hi

Any disadvantage for just declaring one dictionary of object, and then cast the objects when getting them?

private static ConcurrentDictionary<string, object> namedObject = new ConcurrentDictionary<string, object>();

This way all parameters are put into one place. Code can be more reusable.
Casting may hurt performance but if we are only to store and read parameters from it, performance should be manageable.

But maybe I didn't think it through, so just to throw out this question. :)

User avatar
jwebster503
Posts: 24
Joined: 13 Mar 2014
Has thanked: 9 times
Been thanked: 14 times

Re: A Native C# Solution for Global Storage

Postby jwebster503 » 06 Oct 2015

...

Any disadvantage for just declaring one dictionary of object, and then cast the objects when getting them?

private static ConcurrentDictionary<string, object> namedObject = new ConcurrentDictionary<string, object>();

...
Hi, gztanwei.

The "only" disadvantage that I can think of for this involves type safety. One of the large ways that C# and VB.NET are there to help you is that each variable has a type which is strongly enforced by the compiler. There are several major ways you would run into issues, which would snowball in complexity and rapidly make your code much, much harder to maintain after you go back to look at what you wrote 6 months ago. As a professional programmer of over 25 years, let me caution you to please not underestimate this -- in my opinion, code maintainability is THE ABSOLUTE NUMBER ONE PRIORITY of a coder. This alone strongly suggests not to follow what you suggest.

However, if you do, here are couple things what you would want to be on the lookout for if you simply had a single dictionary of object. Keep in mind there are probably many more -- this is just off the top of my head:
  • 1. Every type has its own helper methods and becomes drastically less useful if you try to access it as an object. For example, you can get the Months component of a date, or the length of a string. You rightly suggest you could just cast it if you wish -- there is a minor performance hit, and the major probability of bugs for when you forget to cast it, or probably cast it incorrectly.

    2. Let's say you're counting the number of ticks in the volume of a month, but initially cast that number as an int, which probably would overflow towards the end of the month. You realize this, then change it to a long. Hope you don't forget to change all occurrences of this! :) If it were an int and you change it to a long, the compiler will warn you everywhere you'd need to worry about possible loss of data, but using object, you'll get less assistance.
There are several alternatives if you really want to emphasize code reuse which wouldn't impair type safety:
  • Create a "Composite Dictionary" object that has a bunch of ConcurrentDictionaries of specific types inside, one for Int, another for Date, a third for String, etc. The Composite object would have adder methods like "AddInt( string key, int value )", and this would add to the Int dictionary for you. a RetrieveInt( string key ) would return the int. This way, all of your global dictionaries are contained in one class, but you still have strong typing.

    You could slightly improve on the Composite Dictionary using generics, so you have Add<T>( string key, T value), and inside switch on your supported types "if (T is int)" add to the Int dictionary, etc. You run into issues when you retrieve the value, though, because you need to know the type. But, adding would be nice -- Add( "some bool value", true ) followed by Add( "last week", DateTime.Today.AddDays( -7 ) ), etc.

    Probably my favorite is to better understand what you're going to be reusing, and come up with a custom class which holds your data together in larger chunks. This is harder to do, but will probably give you the most mileage in the end. A simple scenario would be that you have common configuration variables about how far back you look, what timeframe bars you look at, length of moving average, etc. Create a class called "MySettings" or such with these values as properties. Give the class the ability to save itself to ConcurrentDictionaries, and the ability to load itself. This way the many different studies you're writing (to make good use of that reuse) don't need to deal with the ConcurrentDictionaries at all -- just call MySettings.Retrieve(), and then just use the values.
Ultimately, I strongly suggest you don't do this. But, if you do decide to, and you're very careful, you COULD get away with it. But, with all the complexity that I need to deal with throughout the entire day, I'm always looking for ways to make my life easier, not add tedium where it really isn't necessary.

Either way, good luck!

Jeff

jimpe606
Posts: 6
Joined: 19 Aug 2013

Re: A Native C# Solution for Global Storage

Postby jimpe606 » 17 Feb 2016

Hi,

with global variables / storage, is it possible to share information between MC Portfolio Trader and Scanner?

I'm looking for a solution to use Portfolio Trader with different time frames, e.g. getting indicator values based on daily data from scanner, use it together with lower time frame from portfolio trader and manages trades before end of session.

Regards,
Jimmy


Return to “MultiCharts .NET”