ADE and optimization  [SOLVED]

Questions about MultiCharts and user contributed studies.
evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

ADE and optimization  [SOLVED]

Postby evdl » 25 Jul 2012

I have a strategy that uses ADE and backtesting works (although ADE slows down the backtesting). But when I try to do an optimization it will start the optimization but after about 10 minutes it has not found any possible scenario and the ADE will come up with a error, saying there is no collection with that ID. This happens only with optimization.

Before I go debug the strategy to find the possible error. Is it even possible to do an optimization with ADE?
Attachments
Error ADE.png
(5.56 KiB) Downloaded 5114 times

SP
Posts: 465
Joined: 06 Feb 2006
Has thanked: 36 times
Been thanked: 286 times

Re: ADE and optimization

Postby SP » 25 Jul 2012

Did you put the sender code/indicator (ie ADE Save OHLCV) on the same chart as the strategy? That cause the problem on optimization, they need to be on seperate charts.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 25 Jul 2012

The receive code is embedded in my strategy, that I am trying to optimize. And this on the chart of the symbol I want to trade. (this is in case if I use MC main program to optimise)

If I use the portfolio backtester I have the strategy on the chart and ofcourse I use it in the portfolio backtester.

The sender code writes the data to a file. And the receive code reads it from this file.

The sender code is on a different symbol and chart. Because I want to relate the signals with price data of this different symbol.

Backtesting of the strategy with the embedded receive code is working in mc main program and also in portfolio backtester. In the main mc program the optimizing starts but fails after some minutes with the mentioned error. And optimizing in portfoliobacktester give the same error immediatly when I start the optimizing.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 25 Jul 2012

Send code on the dow jones indices:

Code: Select all

[LegacyColorValue = TRUE];

Inputs:
UseFile(ADE.UseFile); // default UseFile is returned by ADE.UseFile function

Vars:
Class("%change"), // class name for our test data
DOWJmap(MapSN.new), // we will use this map to retrieve data with ADE.GetBarInfo
WriteOk(true),
Close_yesterday(0),
Open_today(0),
Open_gap(0),
Change_since_open(0),
Change_since_closeyesterday(0);

// If UseFile is true, load any previously stored data from the file on the first bar.
if CurrentBar = 1 and UseFile then
Value1 = ADE.OpenMap(Class, "DOWJ", ADE.BarInterval);

// Calculate % change
Close_yesterday = closeD(1);
Open_today = OpenD(0);

Change_since_open = round(((close/open_today)-1)*100,3);;
Change_since_closeyesterday = round(((close/close_yesterday)-1)*100,3);
Open_gap = round((open_today - close_yesterday),3);

// Store the indicator values in InfoMap so that we can pass it to ADE.PutBarInfo
Value1 = MapSN.Put(DOWJmap, "% since open", change_since_open);
value1 = MapSN.Put(DOWJmap, "% since C yesterday", change_since_closeyesterday);
value1 = MapSN.Put(DOWJmap, "open today", open_today);
value1 = MapSN.Put(DOWJmap, "close yesterday", close_yesterday);
value1 = MapSN.Put(DOWJmap, "open gap", open_gap);
value1 = MapSN.Put(DOWJmap, "close", close);

// Store the data for this symbol and bar interval.
// ADE.PutBarInfo will copy the information from our InfoMap into the appropriate DataMap.
Value1 = ADE.PutBarInfo(Class, "DOWJ", ADE.BarInterval, ADE.BarID, DOWJmap);

// If UseFile is true, save the data to the file on the last bar.
if LastBarOnChart and BarStatus(1) = 2 and UseFile and WriteOk then begin
Value1 = ADE.SaveMap(Class, "DOWJ", ADE.BarInterval);
WriteOk = false; // prevent repeated writes on new bars
end;
And the receive code that is in my strategy:

Code: Select all

// BEGIN ADE -------------------------------------------------------------------------------------
// If Usefile is true, load any previously stored data from the file on the first bar.
if CurrentBar = 1 and UseFile then

Value5 = ADE.OpenMap(Class, "DOWJ", Interval);

// Retrieve the data for this symbol and bar interval.
// ADE.GetBarInfo will copy the information from the appropriate DataMap into our InfoMap.

value5 = ADE.GetBarInfo(Class, "DOWJ", Interval, ADE.BarID, DOWJmap);

// Fetch the values from the "_" Map into variables

// DOWJ values
Change_since_open_dowj = MapSN.Get(dowjMap, "% since open");
Change_since_closeyesterday_dowj = MapSN.Get(dowjmap, "% since C yesterday");
Open_today_dowj = MapSN.Get(dowjmap, "open today");
Close_yesterday_dowj = MapSN.Get(dowjmap, "close yesterday");
open_gap_dowj = MapSN.Get(dowjmap, "open gap");

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 27 Jul 2012

I have been searching in my code for errors. But I can't get it to optimize. But maybe someone from multicharts can confirm if it is possible to optimize with ADE and load data from csv files in the chart or in portfoliobacktester.

As said, backtesting and the creating of the ADE files works.

Possible problem can be the multiple read attempts of the csv files when optimizing. If someone can confirm it is not possible, would be much appreciated.

Thank you.

Edwin

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: ADE and optimization

Postby Henry MultiСharts » 30 Jul 2012

Hello Edwin,

Please send me the signals you are using (with dependent functions), the workspaces, export instruments (with data) from QuoteManager, your ADE.dll, exact steps to reproduce the issue to support@multicharts.com

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 31 Jul 2012

Thanks Henry,

I will send you an email.

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 05 Aug 2012

i use standard ADE Save OHLCV and nothing is write in "C:\ADE\Data\...".

No error message.

help welcome

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 06 Aug 2012

Hello Smoky.

Did you install the ADE on the standard location in C:\ or did you install it another directory.

If so, you have to change the path accordingly in the function ADE.directory in the PLE editor.
And also look in the ADE.usefile if this is set to TRUE.

this code works:

Code: Select all

[LegacyColorValue = TRUE];

Inputs:
UseFile(ADE.UseFile); // default UseFile is returned by ADE.UseFile function

Vars:
Class("OHLCV"),
Vol(0),
WriteOk(true);

// If UseFile is true, load any previously stored data from the file on the first bar.
if CurrentBar = 1 and UseFile then
Value1 = ADE.OpenMap(Class, GetSymbolName, ADE.BarInterval);

// Calculate correct total volume regardless of bar type
Vol = IFF(BarType < 2, UpTicks + DownTicks, Volume);

// Store the data for this symbol and bar interval
Value1 = ADE.PutOHLCV(GetSymbolName, ADE.BarInterval, ADE.BarID, Open, High, Low, Close, Vol);

// If UseFile is true, save the data to the file on the last bar.
if LastBarOnChart and BarStatus(1) = 2 and UseFile and WriteOk then begin
Value1 = ADE.SaveMap(Class, GetSymbolName, ADE.BarInterval);
WriteOk = false; // prevent repeated writes on new bars
end;

Plot1(0); // include a Plot statement so we can use this indicator in RadarScreen
You have to put this indicator on the symbol you like to have the OHLC file of. Also be sure that the barinterval is the same as the chart resolution. The code above works with 1 minute data. If you want other intervals you may have to change the code.

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 06 Aug 2012

I try this code and i have this message ...when a new candle is drawing

Image

Thanks for your help ;)

ADE is on c:\

pehaps not last ADE/EL packages ?

seven 64 bits tests with Multicharts 32 & 64 bits

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: ADE and optimization

Postby TJ » 06 Aug 2012

I try this code and i have this message ...when a new candle is drawing
Image
Thanks for your help ;)
ADE is on c:\
pehaps not last ADE/EL packages ?
seven 64 bits tests with Multicharts 32 & 64 bits
You must use the 64 bit add on for the 64 bit MultiCharts.

MultiCharts 64 bit resources
viewtopic.php?f=16&t=10094

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 07 Aug 2012

Thanks TJ,

I put the 3 dll in "C:Programmes\TS Support\MultiCharts64" folder and same message :(

in this folder my Elcollections.dll is 622Ko long, the wrong package ? #POST 2

I use Bamboo ADEsetup.exe and ELCsetup.exe to install the addon.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 07 Aug 2012

Best you can install the files yourself and not use an exe installer.

First put the elcollections.dll (the 64bit version, 622kb) in the TS support directory (the 64bit version in the 64bit MC TS support folder).

Then copy the ADE files to the desired directory (you can use the default C:\ADE directory). Follow the instructions in the Alldataeverywhere.doc (for example create necessary directories).

Then import the elcollections.eld in the powerlanguage editor. After that import the alldataeverywhere.eld. It will ask you to overright some functions which you imported with the elcollections.eld. You can overwrite these. Then compile all the uncompilled functions. (takes some minutes).

Then use the OHLC save indicator and put this on the symbol of your choice. (You may have to change the ADE.usefile setting to true).

This should work.

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 07 Aug 2012

Thanks Evdl thanks buton doesn't work

"Fatal error: Class 'messenger' not found in /usr/www/users/tss/multichartscom/discussion/includes/functions_thanks.php on line 511" LoL

I made all step by step like you explain to me and I have this message

Image

I have fist a message ELCollections.dll not found I rename it in lower case letter elcollections.dll
and the same message ..

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 07 Aug 2012

Are you using the 32bit and 64bit version of MC together? And do you maybe have the ADE files directories of both versions directed to the same ADE directory. I mean in the ADE.directory function.

If so, you can try to change the directory in the ADE.directory function and have different locations for the ADE files for the 32bit and 64bit version of MC.

And you say that you got a message first that it couldn't find the elcollections.dll? Did you place this in this directory: C:\Program Files\TS Support\MultiCharts64\

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 07 Aug 2012

Now, I only use 64bits, ade.usefile is true, ade.directory is "C:\ADE"

I don't undestand why it doesn't work :(

yes first i put it in TS support, after in right place C:\Programmes\TS Support\multicharts64\

just to find lower case letter issue on the elcollections.dll name ...

In french seven 64 bits, 64 bits programs are on "C:\Programmes" folder

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 07 Aug 2012

Maybe a longshot. I have the dutch version of W7. And my program directory is just named as the english versions of W7.

Maybe that is the problem. The person who made the elcollections.dll may have made a reference in the dll to the program files directory and not the programmes directory as in your french W7.

I don't know if this the problem, and if this can be fixed in the dll. But you can try to make this directory path yourself.

So make C:\Program Files\TS Support\MultiCharts64\.

And put there the elcollections.dll file. And give it another try. Longshot, but worth a try.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 08 Aug 2012

Henry replied to me about the optimization not working with ADE. If you have a quad core like me (or any other multiple core processor). The solutions is to limit the processor cores that are used.

I received two files from MC that edit the register of windows. One file is to limit the use of all cores and one file is to reactivate all cores again. With the use of one core the optimization with ADE works.

Please keep in mind that with one core you increase optimization time. If all cores are not activated again, also the optimization without ADE is effected and will take longer.

It will only effect optimization and not backtesting or MC itself.

According to Henry probably the ADE files do not support multiple threads (or my suspicion also the elcollections.dll) during optimization and therefore can not use all cores. I don't have the knowledge to look into dll. Hopefully someone can update these files to use all cores. Would be much appreciated.

But for now I know that my ADE code was not the problem and can stop debugging.

Thank you to MC team and Henry for the response.
Attachments
All_Thread_ForOptimize.zip
(523 Bytes) Downloaded 571 times

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: ADE and optimization

Postby Henry MultiСharts » 08 Aug 2012

Now, I only use 64bits, ade.usefile is true, ade.directory is "C:\ADE"
Are you able to create a new text file or folder in C:\ADE ?

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 08 Aug 2012

Now, I only use 64bits, ade.usefile is true, ade.directory is "C:\ADE"
Are you able to create a new text file or folder in C:\ADE ?
Sorry, no text file in "C:\ADE\Data\..."

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: ADE and optimization

Postby TJ » 08 Aug 2012

Now, I only use 64bits, ade.usefile is true, ade.directory is "C:\ADE"
Are you able to create a new text file or folder in C:\ADE ?
Sorry, no text file in "C:\ADE\Data\..."
check your access rights.

are you running MultiCharts as administrator?

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 09 Aug 2012

Yes runing as administrator, with UAC disable, nothing writting on C:\ADE\Data

SP
Posts: 465
Joined: 06 Feb 2006
Has thanked: 36 times
Been thanked: 286 times

Re: ADE and optimization

Postby SP » 10 Aug 2012

My mistake, I edited the message so no one gets confused.
Last edited by SP on 11 Aug 2012, edited 1 time in total.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 10 Aug 2012

Hi SP,

The code that I posted earlier (is the ADE Save OHLCV indicator, which is part of the ADE install), will generate a CSV file with the OHLC values of the symbol you put the indicator on, in the directory that is in the ADE.directory function.

I just tried it again, and it generates a file, so I think there is no need to add print arguments.

The ADE plot OHLCV indicator will save the info in the dll for further use on other charts.

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 10 Aug 2012

Exactly, evdl was right !

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 14 Aug 2012

Multicharts's support find why ADE can't writte a file in c:\ade\data\ !

ADE Save OHLCV use GetSymbolName to make the filename, but with "EUR/USD" we have a bad "/" in the name !

In ADE Save OHLCV

Code: Select all

Vars : sym("");

sym = GetSymbolName;
if MidStr(sym, 4,1)="/" then sym = LeftStr(sym, 3) + MidStr(sym, 5, StrLen(sym)-4);
and replace all GetSymbolName with the sym var.

Thanks for your help ;)

works well with multicores, UAC, ....

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: ADE and optimization

Postby TJ » 14 Aug 2012

Thanks for the update.
Much appreciated.

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 14 Aug 2012

Hi Smokey,

Although the topic is set to solved. Which it is , but I like to ask one last thing. In the last sentence you say:
works well with multicores, UAC, ....
The reason why I started this topic was that I couldn't do optimizing with ADE. Backtesting ok, but no optimizing. Until I limit the cores of the processor.

Just to leave no question mark for me and other readers. You mean works well with multicores, when backtesting and not optimizing, right?

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 14 Aug 2012

Hi Smokey,

Although the topic is set to solved. Which it is , but I like to ask one last thing. In the last sentence you say:
works well with multicores, UAC, ....
The reason why I started this topic was that I couldn't do optimizing with ADE. Backtesting ok, but no optimizing. Until I limit the cores of the processor.

Just to leave no question mark for me and other readers. You mean works well with multicores, when backtesting and not optimizing, right?
for this time, i can write data to c:\ade\data (he he), too early for backtesting and optimizing but I'll tell us news about my work with ADE ;)

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 14 Aug 2012

Ok thanks. And good trading to you.

tekram
Posts: 96
Joined: 26 May 2009
Has thanked: 6 times
Been thanked: 18 times

Re: ADE and optimization

Postby tekram » 20 Aug 2012

You'll probably need to get MC to support ELCollections2.dll and int64 data type.

As you may know, Marshall (aka Bamboo; author of ADE and ELC) has recently started working at the Plantation known as TS; he was previously only a TS user and not an employee. There have been reported problems with the prior versions of ELCADE with multicore support. There is now a multicore compatible beta version of ELCADE, version 2, available on the TS beta forum. This installer requires an activated TS 9.1 in order to install the new ELCollections2.dll (see the link below to get to the TS forum and installer).

You may want to vote for this feature if you depend on MC for optimization:
https://www.multicharts.com/pm/viewissu ... no=MC-1097
I have a strategy that uses ADE and backtesting works (although ADE slows down the backtesting). But when I try to do an optimization it will start the optimization but after about 10 minutes it has not found any possible scenario and the ADE will come up with a error, saying there is no collection with that ID. This happens only with optimization.

Before I go debug the strategy to find the possible error. Is it even possible to do an optimization with ADE?

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 20 Aug 2012

You may want to vote for this feature if you depend on MC for optimization:
https://www.multicharts.com/pm/viewissu ... no=MC-1097


Ok for me vote +1 :D

evdl
Posts: 401
Joined: 19 Jan 2011
Location: Netherlands
Has thanked: 85 times
Been thanked: 124 times

Re: ADE and optimization

Postby evdl » 21 Aug 2012

Thanks Tekram for the info.

+1 vote from me.

waveslider
Posts: 222
Joined: 16 Oct 2011
Has thanked: 66 times
Been thanked: 20 times

Re: ADE and optimization

Postby waveslider » 06 Oct 2013

MC please support "int64" so that we can use ELcollections2

thanks

User avatar
Smoky
Posts: 507
Joined: 03 Dec 2010
Location: Thailand
Has thanked: 97 times
Been thanked: 115 times

Re: ADE and optimization

Postby Smoky » 06 Oct 2013

waveslider could you give us where we can found ELcollections2 ?

sorry no TS forum acces :(

waveslider
Posts: 222
Joined: 16 Oct 2011
Has thanked: 66 times
Been thanked: 20 times

Re: ADE and optimization

Postby waveslider » 06 Oct 2013

I guess it doesn't really matter - since it doesn't work with MC!
you have to have an active copy of TS installed to install it also..
I got it from a friend

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: ADE and optimization

Postby Henry MultiСharts » 17 Oct 2013

MC please support "int64" so that we can use ELcollections2
thanks
Please vote for the corresponding feature request: Support int64 data type as in EasyLanguage, ELCollection2.dll

javamarket
Posts: 55
Joined: 10 Jul 2014
Has thanked: 10 times
Been thanked: 18 times

Re: ADE and optimization

Postby javamarket » 18 Jul 2014

+1 on this addition

The 'vote' doesn't seem to stick and the request does not seem to be reviewed.

TW
Posts: 31
Joined: 14 Dec 2009
Has thanked: 5 times
Been thanked: 1 time

Re: ADE and optimization

Postby TW » 20 Feb 2015

Will MC team schedule ADE for 64bits in near future?

User avatar
Henry MultiСharts
Posts: 9165
Joined: 25 Aug 2011
Has thanked: 1264 times
Been thanked: 2957 times

Re: ADE and optimization

Postby Henry MultiСharts » 24 Feb 2015

Will MC team schedule ADE for 64bits in near future?
Please see 64 bit version of ELCollections.dll


Return to “MultiCharts”