New subject: auto maxbarsback adjustment message needed.

Questions about MultiCharts and user contributed studies.
bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

New subject: auto maxbarsback adjustment message needed.

Postby bowlesj3 » 11 May 2009

Things got off topic a bit near the end. However the very last post has a suggestion for what was actually occuring and a specific message to use to warn the user if MC thinks that the MaxBarsBack setting should be increased above its current setting.

Original post below:

I have been painfully reminded today that if you apply a study to a chart that has a search back in time via offset (either sequential search or binary search), and you forget that you have to manually set the maxbarsback to a much higher value based upon how far back you are likely to search, then MC will override your code and cause it to do strange things such as this. It will make the code below (which should only execute once) execute more than once.

if currentbar = 1 then
begin
{stuff that should only execute one time at the first bar}
end;

So the suggestion is to have MC create a popup warning since it would be a lot better than what it is doing now which is to create unpredictable results for the trader. It can send the trader on a wild bug finding goose chase if they are new to this quirk (or have temporarily forgotten about this as I did). It could also create a serious problem for them if they do set maxbarsback to 1000 and on a few occasions their program needs more than that. In my case it is not a big deal but for some traders it might be.

Thanks,
John.
Last edited by bowlesj3 on 13 May 2009, edited 1 time in total.

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 12 May 2009

It seems I have solved the problem above on the 10 second bars only be setting the MaxBarsBack to a value of 10,000 to match the number of bars of the symbol itself. What is interesting is the fact that I could only set MaxBarsBack to a value of no more than about 1200 on the 1 minute bars but on the 10 second bars I can set it way higher. I think there is a way to increase the maximum size for it. I should do a search to find it. It must be different for different bar compressions.

By the way, for anyone wanting to do similar tests here is how I figured out that the code above was repeating. I put these commands within that code and before I turn on the EL code I have these fields initialized. The string field below shows me the date and time that the command above is repeated. I actually changed the code above to be based upon code shown farther down. Either one should limit that code to be executed only once at the start before all other code.


Code: Select all


   tempstring = GV_GetNamedString("Arw--TempString1","");
   tempstring  = tempstring  + " " + numtostr(date,0)   + " " + numtostr(Time_s,0)   + " / "  ;
    value2 = GV_SetNamedString("Arw--TempString1",tempstring);

   value1 = GV_GetNamedInt("Arw--Temp1",0);
   value1 = value1 + 1;
   value1 = GV_SetNamedInt("Arw--Temp1",value1);



Code: Select all


if FirstBarHit = "N" then
   Begin
   FirstBarHit = "Y";
   end;


bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 12 May 2009

Actually it turns out I had to back off the MaxBarsBack setting down to 8000 from 10000 so the study has some room to start before the current bars. The good news is it still seemed to fix the problem of having the (currentbar = 1) code being reexecuted more than the normal 1 time we would expect. I don't understand the problem but all is well is seems for now at least. The study is now working properly on the 10 second bars. 8000 is a really large setting. Way more than the size of the binary searches back in time that the study requires. Actually the binary search is such that the start point of the search is bar 1 anyway so it can never exceed the max bars back setting. So maybe there is something strange in the data that is triggering this problem and not actually the offsets going too far back. Only this rather strange problem made it so a maxbarsback setting of 8000 is required.

So in the end maybe MC needs a trap installed by TSS to determine if CurrentBar = 1 is executing more than one time. For now I have my own trap with an alert based upon the GV code above. I didn't mention the actual number of times that code was executed. When set to auto, it was 4 times when it should have only been once.

User avatar
RobotMan
Posts: 375
Joined: 12 Jul 2006
Location: Los Altos, California, USA
Has thanked: 31 times
Been thanked: 12 times
Contact:

Postby RobotMan » 12 May 2009

Hi John,

You have probably forgotten more about coding than I will ever learn - Believe me, I'm not trying to insult you... :)
But do you ever use "If CurrentBar >= MaxBarsBack then begin" after any code that fills any arrays or sets variables? I use that whenever I don't want to start a plot on price that begins at 0 when CurrentBar = 1.

sheepishly submitted....

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 12 May 2009

Hi Bob,

I have done a lot of coding in normal languages (and even assembler when I started - actually machine language a tiny bit just to have a tiny thrill) but EL is a special kind of quirky animal that can really through the best of programmers because it just isn't normal. The more normal programming a programmer does the more likely they will get into a normal thinking rut and it can be hard to see outside of this box. Having said that for sure what I describe above is a bug of some sort. At this point it is a TSS problem.

Anyway, as far as using "If CurrentBar >= MaxBarsBack then begin" and "I don't want to start a plot on price that begins at 0 when CurrentBar = 1" I have never used that. My understanding of MaxBarsBack is that it works this way. If, when using 1 minute bars, the very first bar showing on your chart is at time 0001 today and the next bar is at time 0002 today and this pattern continues to time 0014 (which is the standard RSI setting) then a simple RSI study would set maxbarsback to 14 so that currentbar=1 would actually be the bar at the time 0014 (I just tested it by forcing the maxbarsback to 14 on a simple RSI study and putting in a print statement (see code below) and my understanding appears to be correct). In other words the maxbarsback is a built in delay factor for the startup of a study so it automatically delays a study like the RSI enough so that you do not have a problem with its plot. You only get a problem if your chart is actually showing less than 14 bars (as the daily bars futures does in the SXF symbol I use when the rollover first occurs) . In that case MC gives an error of some sort. So if you are having a problem with a study at currentbar = 1 then all you have to do is force the MaxBarsBack up 1 digit at a time until the problem goes away.

Here is how my study works (how it uses the binary search).

The problem:
I do not trust swing code to get the swings correct. I wrote one that works better than the one that comes with MC but still it just can never be smart enough to figure out the subwaves that can occur. I figure my eyes will always be able to out-do-it by a mile and a hell of a lot faster too. It isn't just the subwaves but the acceleration and deceleration (which have all sorts of shapes) that the eyes can so easily detect. Going against early acceleration in the market is not a good thing and writing swing code to detect it with 100% certainty I figure is real hard to do. Why bother. It can never out do the eyes. I use the auto swing code that I wrote still but in a different way now (it helps me confirm things).

The solution:
So what I do to overcome this weakness is I manually put arrows on the chart where I believe the tops and bottoms of the waves are as I see it (I force it to be correct no matter what and it is extremely fast too!). My study searches back using a binary search, finds the arrow and gets whatever info I want from that bar using the backward offset which the binary search came up with when it found the arrow. So I set the binary search to search from the currentbar=1 to the lastBarOnTheChart. Interestingly if I want the binary search to be able to search back to the very first bar on the chart I need to set the maxbarsback to zero which would make the currentbar=1 be the actual first bar you see on the chart. That did not occur to me until today actually. In other words a study that uses the binarysearch needs a smaller maxbarsback if you want to search more data. It is not like a plot study at all. If I was to set the maxbarsback to 10000 which is the size of the data on my chart then the binary search could not get any data to search. So I figure that I fixed the problem of CurrentBar=1 getting tripped 4 times by using the really high MaxBarBack Setting to cause my study to not see these strange bars that were triggering the problem for whatever reason. It is funny, I never fully realized why a maxbarsback setting of 8000 fixed the problem until right now as I write this. So I guess I have to thank you Bob for getting me to think about it a little more. Maybe this could be thought of as "Writing Therapy for the troubled EL programmer that is about to go crazy with all its quirks :-)".

John.

Here is the code that helped me prove how the MaxBarsBack actually works. Actually make this all that is in your study and call it MaxBarsBackTest. Run it with a setting of MaxBarsBack=0 and the date and time printed will match your very first bar. Keep adding 1 to the maxbarsback setting and each time the bar will be the next bar from the very start of your chart. Now for the first time I can go from being 99.9% certain I understand it to 100% certain I understand it.

Code: Select all

if CurrentBar = 1 then
   Begin


       Print( File("C:\ALog_MaxBarsBackTest.txt"),
                  "Date" , " " ,
                  Date, " " ,
                  
                   "Time_s" , " " ,
                  time_s, " " ,
                  
                     " ");
    
   
 
     end;

Last edited by bowlesj3 on 13 May 2009, edited 5 times in total.

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 12 May 2009

It just occured to me that maybe MaxBarsBack would be better named DelayStudyStart. In other words, delay my study from starting to execute until there is enough data available to provide the amount of data it requires. Thus the currentBar=1 will be a later bar on the chart if the MaxBarsBack/DelayStudyStart is set to a higher value.

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 13 May 2009

It may be that the original problem I was having was that for some reason my code was searching back farther than the automatic maxbarsback setting and also searching back farther than the initial 500 and 1000 settings I chose. I did have to make a change to my code to get it to work on the 10 second bars even though it was working on the 1 minute bars before. I made this change after I increased the MaxBarsBack to 8000 actually and I have not gone back to test my theory I just discussed here. However if MC is coming up with an initial MaxBarsBack setting then on the fly deciding that it has to do a recalculate with a larger maxbarsback and thus a new currentbar = 1 date and time (which is what it appears to have been doing) then I think the user needs to get a popup warning saying exactly that. Specifically:

MaxBarsBack adjustment Warning:
Study whatever maxbarsback is exceeded and being increased to 799. Initial currentbar = 1 date and times 1090503 and 105730 will now be 1090503 and 115230).

The reason they need the warning is because the GV values will get all screwed up if the currentbar = 1 code has to be reexecuted and the study will not work. The popup warning would give the trader the needed info to speed up their debugging process. Not only that if this happens live the popup will notify them and maybe prevent financial losses.

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 13 May 2009

Okay, now I fixed the post.

I thought I would try an experiment to determine a few things.

1/ I wanted to see if I could use an offset in square brackets to reference back to the very first bar on the chart during the "if CurrentBar = 1 then" code when maxbarsback was set to a value of greater than zero. This worked. I was able to do that. It is something I never had any need for and probably never will. I was just curious.

2/ Next I added one more print statement to see if I could recreate the error where the "if CurrentBar = 1 then" code was being repeated up to 4 times (in other words MC is somehow doing a recalculate). I tried to get this to happen by purposely trying to reference a bar prior to the first bar in the chart (see the code below where there is a comment in this regard). However I could not and the code below describes the error that I got. It is the "Not enough series length" error.


So the conclusion is I have no idea :roll: why the study was somehow forcing the "if CurrentBar = 1 then" code to repeat. All I know is setting max bars back to 8000 fixed the problem so maybe is was some sort of bad data doing it.

Thanks,
John.




Code: Select all


variables:
   tempstring("");
   
   
 

if CurrentBar = 1 then
   Begin
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_SetNamedInt", lpstr, int ;
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_GetNamedInt", lpstr, float ;

   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_SetNamedString", lpstr, lpstr ;
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", lpstr, "GV_GetNamedString", lpstr, lpstr ;

       Print( File("C:\Access\ALog.txt"),
                  "Date" , " " ,
                  Date, " " ,
                  
                   "Time_s" , " " ,
                  time_s, " " ,
                  
                     " ");

       Print( File("C:\Access\ALog.txt"),
                  "Date[maxbarsback]" , " " ,
                  Date[maxbarsback] , " " ,
                  
                   "Time_s[maxbarsback] " , " " ,
                  time_s[maxbarsback] , " " ,
                  
                   "maxbarsback", " " ,
                  maxbarsback, " " ,
                      " ");
                      
   {
   If I add this print statement in I get an error "Not enough series length".
   I was trying to see if I could recreate the error where the "if CurrentBar = 1 then" code was repeated more than one time
   by referencing a bar prior to the very first bar showing on the chart. So this seems to prove that something strange
   was causing the "if CurrentBar = 1 then" to be executed more than once.
   }
       Print( File("C:\Access\ALog.txt"),
                  "Date[maxbarsback + 1]" , " " ,
                  Date[maxbarsback + 1 ] , " " ,
                  
                   "Time_s[maxbarsback + 1 ] " , " " ,
                  time_s[maxbarsback + 1 ] , " " ,
                  
                   "maxbarsback", " " ,
                  maxbarsback, " " ,
 
                   "maxbarsback + 1", " " ,
                  maxbarsback + 1, " " ,
                       " ");
 
     {This code is used to determine if a recalculate is being triggered}
   tempstring = GV_GetNamedString("Arw--TempString1","");
   tempstring  = tempstring  + " " +
                     numtostr(date,0)   + " " +
                     numtostr(Time_s,0)   + " / "  ;
    value2 = GV_SetNamedString("Arw--TempString1",tempstring);

   value1 = GV_GetNamedInt("Arw--Temp1",0);
   value1 = value1 + 1;
   value1 = GV_SetNamedInt("Arw--Temp1",value1);
      
 
     end;




User avatar
RobotMan
Posts: 375
Joined: 12 Jul 2006
Location: Los Altos, California, USA
Has thanked: 31 times
Been thanked: 12 times
Contact:

Postby RobotMan » 13 May 2009

Wow, John! This is a lot of work. It is nice of you to document it for "posterity" even though I don't use any code at the moment that needs such drastic evaluation of maxbarsback. It is nice to be able to come to the forum and do searches when having trouble understanding how something works and finding someone's posts that clear it up or help.
Just thought I would say Thanks!

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 13 May 2009

Hi Bob,

Actually most of the work was just typing it up but typing is fast so not that bad. All code is just simple cut and paste.

I am curious about your statement below.

I use that whenever I don't want to start a plot on price that begins at 0 when CurrentBar = 1.


When would a study start at 0 on currentbar = 1? Would it not be just a matter of increasing the maxbarsback setting manually to fix this? If it did work you could store this as default with the study so new applications would have the default maxbarsback setting.

John.

User avatar
RobotMan
Posts: 375
Joined: 12 Jul 2006
Location: Los Altos, California, USA
Has thanked: 31 times
Been thanked: 12 times
Contact:

Postby RobotMan » 13 May 2009

Hi John,

I only try stuff until I kinda get a solution, then I quit. I don't know why I use the code, I just know that the difference between the two JPGs is that line of code. I could've used "barnumber >= MBB", but I ended up using "Currentbar>=MBB". Infact, as I open up VIDYA to take a peek, I used 2*MBB.

Although I *will* go back and hand calculate, on paper or Excel, stuff to make sure that the software is doing what I expect it to do. And I *never* use the built in indicators until I re-write them using code I know for sure works right. And I *will* research all the different ways to do it wrong and put in 100's of hours of mind numbing coding - I am basically, sloppy and lazy during the last 5% and say, "good enough". I'd rather be trading. :)
Attachments
with.jpg
with.jpg (106.77 KiB) Viewed 415 times
without.jpg
without.jpg (98.69 KiB) Viewed 409 times

User avatar
TJ
Posts: 6521
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 957 times
Been thanked: 1883 times

Postby TJ » 13 May 2009

you guys' analytical skills are amazing!

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 14 May 2009

Code: Select all

I only try stuff until I kinda get a solution, then I quit.


I do that a lot too. Reading it made me laugh but seriously I don't think it should have because I actually think it is suppose to be that way. I didn't know things about MaxBarsBack until I had this problem.

I would rather trade too.

John.
Last edited by bowlesj3 on 14 May 2009, edited 1 time in total.

bowlesj3
Posts: 1977
Joined: 21 Jul 2007
Has thanked: 155 times
Been thanked: 395 times

Postby bowlesj3 » 14 May 2009

I think I figured out the problem.

Short version:
When you execute an offset on a variable that exceeds the maxbarsback setting, MC forces a recalculate using a new MaxBarsBack setting so it can allocate more table size for that variable. It does this a few times then eventually gives you a popup at about 8 tries I think. So it is during this recalculate that the CurrentBar = 1 code is repeated.


My situation:
My code which needed an adjustment to run under the 10 second bars must have had a bug during the early stages of the conversion that was causing the maxbarsback value to be exceeded by an offset trying to look past it.

The script below:
To determine this I modified my test study above into a new version shown below. There are some comments included within the study to explain things. If anyone wants to use this study to do their own tests you need a way to manually reset the GVs before each time you turn on the study for the testing. I use my database program for this. You could write a special study to do this from MC and turn it on and off to reset those variables. I also display them from the database program. From EL I guess you would create a separate study with a print statement to print the GV values to see how many times the currrentbar = 1 code was executed and what the values in the string GV were set to.

The conclusion:
if you exceed the maxbarsback setting with an offset which is too large then currentbar = 1 code will get repeated and if there are GV initialization settings in there (or increments) this will create a problem. The thing is MC will quietly do this maybe 8 times before it gives you a very vague message to let you know something is wrong. In my case it did it only 4 times and not knowing of this (or suspecting it) a fair bit of debugging work was created as well as the followup experiments you see in this post to try and understand it. Lots of fun.

Followup notes:

Last nights test was not good enough because I did the test such that it went off the end of the actual bars on the chart. The test below is similar but is executed starting only on the last bar on the chart. This way I could see several recalculates as documented by the string global variable.

It is probably a good idea to test the offset amount to see if it is greater than the maxbarsback setting before doing the offset reference. The other option is to do what I was doing below and test if currentbar = 1 code is executed more than one time and just bump up your maxbarsback if needed. Either way the problem is if you create an alert it may not get seen due to the built in MC recalculate. I guess you could bypass the offset reference if maxbarsback is exceeded so that the alert popup would be noticed (you have to increase the maxbarsback setting and do a rerun anyway if this happens).

I am assuming that close[offset] is limited to the maxbarsback in addition to variables that the user sets up.

John.

Code: Select all


inputs: NumberOfRecalculates(5), {If you set this high enough you eventually get a popup error}
    ExceedMaxBarsBackAmt(1); {this is the amount to exceed maxbarsback by to force currentbar = 1 to repeat due to a recalculate}

variables:
    tempstring("");
   
if CurrentBar = 1 then
   Begin
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_SetNamedInt", lpstr, int ;
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_GetNamedInt", lpstr, float ;

   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", int, "GV_SetNamedString", lpstr, lpstr ;
   DefineDLLFunc:  "C:\Program Files\TS Support\MultiCharts\GlobalVariable.dll", lpstr, "GV_GetNamedString", lpstr, lpstr ;

       Print( File("C:\Access\ALog.txt"),
                  "Date" , " " ,
                  Date, " " ,
                  
                   "Time_s" , " " ,
                  time_s, " " ,
                  
                     " ");

       Print( File("C:\Access\ALog.txt"),
                  "Date[maxbarsback]" , " " ,
                  Date[maxbarsback] , " " ,
                  
                   "Time_s[maxbarsback] " , " " ,
                  time_s[maxbarsback] , " " ,
                  
                   "maxbarsback", " " ,
                  maxbarsback, " " ,
                      " ");
                      
     
     {
     this code is to determine of the currentbar = 1 code has been performed more than one time if an offset
     beyond the maxbarsback setting has been attempted.
                You can not use the print statement above since the printout will be restarted with each new recalculate.
     }
   tempstring = GV_GetNamedString("Arw--TempString1","");
   tempstring  = tempstring  + " " +
                     numtostr(date,0)   + " " +
                     numtostr(Time_s,0)   + " / "  ;
    value2 = GV_SetNamedString("Arw--TempString1",tempstring);

   value1 = GV_GetNamedInt("Arw--Temp1",0);
   value1 = value1 + 1;
   value1 = GV_SetNamedInt("Arw--Temp1",value1);
      
 
     end;
 
if LastBarOnChart_s then
   begin

   value1 = GV_GetNamedInt("Arw--Temp1",0);
   if value1 < NumberOfRecalculates then
      begin
       Print( File("C:\Access\ALog.txt"),
            "Date[maxbarsback + ExceedMaxBarsBackAmt]" , " " ,
            Date[maxbarsback + ExceedMaxBarsBackAmt], " " ,
            
             "Time_s[maxbarsback + ExceedMaxBarsBackAmt]" , " " ,
            time_s[maxbarsback + ExceedMaxBarsBackAmt] , " " ,
            
             "maxbarsback", " " ,
            maxbarsback, " " ,
 
             "maxbarsback + ExceedMaxBarsBackAmt ", " " ,
             maxbarsback + ExceedMaxBarsBackAmt, " " ,

               " ");
      end;
 
     end;



Return to “MultiCharts”