notes on Global Variable
Posted: 26 Feb 2008
This is a little long so here is a bit of organization to help you decide if it is worth reading.
1a/ Short Background history leading to work to test GV speed.
1b/ Background Specifics.
2/ A method to test the Named GV speed.
3/ Questions about how the Named GVs work and what can be done to make it faster.
4/ What about numbered GVs instead of named GVs.
1a/ General Background:
I am a discretionary trader. I do not attempt to program a full trading system and thus use MC primarily as a work load reduction tool. I am an application programmer, I use MS-Access VBA as part of my overall trading setup. I use MS-Access as a control software to control MC and also IB’s Traders Work Station. I also obviously get data from MC via named global variables for this process. After 7 months of working with MC and MS-access I have my trading system software basically done and I have finally started to notice enough delay in getting data from MC to MS-Access through Named GV methods that I am having to now both look for ways to speed up the EL code, reduce unneeded EL code, and also ask myself if any new code will just cause more slowdown with little or no value in return which contributes to increased income.
1b/ Background Specifics:
My EL system has to count ticks and feed these counts to MS-Access so it knows that certain data is available for the “trigger wait mode” portion of my trading cycle. MS-Access has a timer function in every form. In one of my forms I use the timer to go after the tick count that the EL code puts out. It does this every second. If a parameter setting of 10 seconds passes and it does not get a tick count MS-Access gives me a popup warning. This warning happens a lot with my current level of system code and 10 seconds is a lot. I also see the last prices changing in the MC chart but I do not get the ticks in MS-Access even though I see them.
2/ Method to test:
Okay so there is a problem. However before I go looking for code to get rid of or make more efficient I need a good way to measure if these changes are working. So I plan on writing out a print statement in the EL code at the exact same point that I put the tick count out to the GV. I put the current time on this print statement along with the actual current tick count. In MS-Access I use the GV_GetNamedInt command to get the tick and I write out a log record exactly when I get it with the tick count and the current time. After running for a while I run in MS-Access a matching record batch process that is sorted by tick count and when the two logs (the EL print statements and the MS-Access log writes) get a matching tick number it does a subtraction of the current times on each to get the number of seconds or milli second difference. At the end of the batch run I have the average, max and min delays giving me a feel for how bad is it. Now I can start putting in changes and see how much each change helps. I can also run this test on a stripped down single chart workspace to see what the best possible result could be. I have a list of 10+ possible improvements to my EL code to make and I can measure each improvement and the total improvement.
3/ Questions about how the GVs work:
To the best of my knowledge the named GV calls should be executing code to do a binary search to get the info. This would seem to be the fastest method to get the data out of the named tables. In order to do this the named items must be sorted. The fastest sort for tables of 200+ elements is to the best of my knowledge what is called a “comb sort”. I think it is actually faster than the shell sort. So the question is “do new elements get loaded in and sorted with a comb sort or maybe an even better one”. Maybe even faster, are they loaded in with a binary search to find the proper position with a subsequent mass calculated move that shifts everything down using an assembler type low level move statement. Anyway, what I currently do, just in case it is doing a binary search, is I preload all my GVs at the very start of the trading day in sorted sequence. I use MS-Access for that. This way the searches should be faster and the dll code should not have to do any insertions during trading time.
4/ What about numbered GVs instead of named GVs.
I am thinking that numbered GVs may be a much faster approach since they have a direct reference not needing any form of search and I could use MS-Access to keep all this organized as new ones come in with future programming/trading enhancements. Maybe that is the best approach now that my trading systems are basically done and I am not likely to make too many changes to it (no major ones).
Thanks for any comments or suggestions.
John.
1a/ Short Background history leading to work to test GV speed.
1b/ Background Specifics.
2/ A method to test the Named GV speed.
3/ Questions about how the Named GVs work and what can be done to make it faster.
4/ What about numbered GVs instead of named GVs.
1a/ General Background:
I am a discretionary trader. I do not attempt to program a full trading system and thus use MC primarily as a work load reduction tool. I am an application programmer, I use MS-Access VBA as part of my overall trading setup. I use MS-Access as a control software to control MC and also IB’s Traders Work Station. I also obviously get data from MC via named global variables for this process. After 7 months of working with MC and MS-access I have my trading system software basically done and I have finally started to notice enough delay in getting data from MC to MS-Access through Named GV methods that I am having to now both look for ways to speed up the EL code, reduce unneeded EL code, and also ask myself if any new code will just cause more slowdown with little or no value in return which contributes to increased income.
1b/ Background Specifics:
My EL system has to count ticks and feed these counts to MS-Access so it knows that certain data is available for the “trigger wait mode” portion of my trading cycle. MS-Access has a timer function in every form. In one of my forms I use the timer to go after the tick count that the EL code puts out. It does this every second. If a parameter setting of 10 seconds passes and it does not get a tick count MS-Access gives me a popup warning. This warning happens a lot with my current level of system code and 10 seconds is a lot. I also see the last prices changing in the MC chart but I do not get the ticks in MS-Access even though I see them.
2/ Method to test:
Okay so there is a problem. However before I go looking for code to get rid of or make more efficient I need a good way to measure if these changes are working. So I plan on writing out a print statement in the EL code at the exact same point that I put the tick count out to the GV. I put the current time on this print statement along with the actual current tick count. In MS-Access I use the GV_GetNamedInt command to get the tick and I write out a log record exactly when I get it with the tick count and the current time. After running for a while I run in MS-Access a matching record batch process that is sorted by tick count and when the two logs (the EL print statements and the MS-Access log writes) get a matching tick number it does a subtraction of the current times on each to get the number of seconds or milli second difference. At the end of the batch run I have the average, max and min delays giving me a feel for how bad is it. Now I can start putting in changes and see how much each change helps. I can also run this test on a stripped down single chart workspace to see what the best possible result could be. I have a list of 10+ possible improvements to my EL code to make and I can measure each improvement and the total improvement.
3/ Questions about how the GVs work:
To the best of my knowledge the named GV calls should be executing code to do a binary search to get the info. This would seem to be the fastest method to get the data out of the named tables. In order to do this the named items must be sorted. The fastest sort for tables of 200+ elements is to the best of my knowledge what is called a “comb sort”. I think it is actually faster than the shell sort. So the question is “do new elements get loaded in and sorted with a comb sort or maybe an even better one”. Maybe even faster, are they loaded in with a binary search to find the proper position with a subsequent mass calculated move that shifts everything down using an assembler type low level move statement. Anyway, what I currently do, just in case it is doing a binary search, is I preload all my GVs at the very start of the trading day in sorted sequence. I use MS-Access for that. This way the searches should be faster and the dll code should not have to do any insertions during trading time.
4/ What about numbered GVs instead of named GVs.
I am thinking that numbered GVs may be a much faster approach since they have a direct reference not needing any form of search and I could use MS-Access to keep all this organized as new ones come in with future programming/trading enhancements. Maybe that is the best approach now that my trading systems are basically done and I am not likely to make too many changes to it (no major ones).
Thanks for any comments or suggestions.
John.