Hello,
I develop strategies which are traded with portfolio trader (no intrabar order generation)
I would like to link my algos to a python machine learning classifier.
The process would be this one.
1/ Entry conditions are met
2/ The strategy send an array of values to the python program
3/ the python program convert the array to an image
4/ the image is evaluated by the model classifier
5/ the classifier return to the strategy the tag (good or bad) of the image
6/ based on the tag value, the entry is allowed or not.
The hard part for me is to make communication between multicharts and python. i really dont know how to handle it.
Hope you can help.
Thx
MJ
Link multichart signal to python classifier [SOLVED]
-
- Posts: 275
- Joined: Apr 22 2014
- Has thanked: 70 times
- Been thanked: 72 times
Hello MJ,
You can try sending your array to sharedvar server. You can then read your array into python calling the dll using python ctypes. I have done this and it works. It is probably faster to write the result to a text file and read it . This way you can also back test. You can call your code at end of each bar using Lua.
Good luck with your project.
Alex
You can try sending your array to sharedvar server. You can then read your array into python calling the dll using python ctypes. I have done this and it works. It is probably faster to write the result to a text file and read it . This way you can also back test. You can call your code at end of each bar using Lua.
Good luck with your project.
Alex
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi,
Thank you for your reply.
I didn't know sharedvar, i'll give it a try.
so far instead of sending data from MC to python, i'm writing a csv file.
What i dont how to do is to trigger the python script to open it and send an answer (writing a new csv file).
An other concern is , with all the process outside MC ( file opening, model query , file writing), is there a way to make MC script to wait until it receive the answer to open or not the trade.
Regards,
MJ
Thank you for your reply.
I didn't know sharedvar, i'll give it a try.
so far instead of sending data from MC to python, i'm writing a csv file.
What i dont how to do is to trigger the python script to open it and send an answer (writing a new csv file).
An other concern is , with all the process outside MC ( file opening, model query , file writing), is there a way to make MC script to wait until it receive the answer to open or not the trade.
Regards,
MJ
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi Alex,Hello MJ,
You can try sending your array to sharedvar server. You can then read your array into python calling the dll using python ctypes. I have done this and it works. It is probably faster to write the result to a text file and read it . This way you can also back test. You can call your code at end of each bar using Lua.
Good luck with your project.
Alex
i'm performing test of sharedvar.
I can send MC data to the sharedvar, but for some reason, i can read them with python . it seems the handle value send by MC is incorect.
so i cant read svgetBool, svgetDouble, svgetInt, svgetString.
regards,
MJ
- Attachments
-
- sharedvar.png
- (202.69 KiB) Downloaded 432 times
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi smoky,
You can find the code below.
I can see the values on the sharedvar monitor application, but i can't get all them when i query the server from python.
I cant understand why, but it seems handle int value returned is incorrect when used in the python.
You can find the code below.
I can see the values on the sharedvar monitor application, but i can't get all them when i query the server from python.
I cant understand why, but it seems handle int value returned is incorrect when used in the python.
Code: Select all
external: "SharedVar-a64.dll", int, "svInit" ,string;
external: "SharedVar-a64.dll", bool, "svServerRunning";
external: "SharedVar-a64.dll", bool, "svSetString" ,int,string,string;
external: "SharedVar-a64.dll", string, "svGetString",int,string;
external: "SharedVar-a64.dll", string, "svVersion";
external: "SharedVar-a64.dll", bool, "svSetValue",int,string,double;
external: "SharedVar-a64.dll", int, "svRealmCount";
var : myserv(false),intrabarpersist initval(-1),strbool(False),varstr(""),db(False),intrealm(0);
once begin
cleardebug;
initval = svInit("MC-" +getsymbolname);
end;
myserv = svServerRunning();
strbool = svSetString(initval,"Close","Test sharedvar");
db = svSetValue(initval,"MCChart handle value",initval);
intrealm = svRealmCount();
print(initval);
print("Handle value for " + getsymbolname + " : " + NumToStr(initval,0));
plot1(initval,"graph");
-
- Posts: 275
- Joined: Apr 22 2014
- Has thanked: 70 times
- Been thanked: 72 times
Hi Mj,
I will try and post a simple example later today. It has been a while since I looked at this. You are probably running into an issue with C types and Python types.
regards,
Alex
I will try and post a simple example later today. It has been a while since I looked at this. You are probably running into an issue with C types and Python types.
regards,
Alex
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi,Hi Mj,
I will try and post a simple example later today. It has been a while since I looked at this. You are probably running into an issue with C types and Python types.
regards,
Alex
I 've 2 issues with SharedVar app.
MC side (sharedvar-a64.dll):
I ve converted all dll function to powerlanguage function. the only one that wont work as expected is svGetString.
Inside a study i use the svSetString function, i can see the value on sharedvar app, i immediately (next code line) call svGetString for the same variable and get weird characters.
Code: Select all
//********print**********
Get string from variable named Close :Ðp).
Get string from variable named Un nom :Ðp).
While mc study is running, i run my notebook which create a Realm name : "Python reader".
When i loop through Realms value, i get MC realm and Python realm names =>Ok
But when want to see the variable inside each realm, i'm only get python realm variable names. => Not Ok I'm stuck .
Hope you can help.
MJ
-
- Posts: 275
- Joined: Apr 22 2014
- Has thanked: 70 times
- Been thanked: 72 times
Hi MJ,
Hopefully this will help. I am running this Python code inside of Rapidminer as I prefer to use that for many machine learning tasks especially in the early phase of testing ideas. This requires that I use dataframes but the code should be straight forward.
In this example, I am exporting six data points from Multicharts and generating an ID and datetime stamp.
On the Python side, the code looks like this....
The only tricky part is this part which is dealing with the differences between python types and c types. This probably where you are having your problem.
The end result is a nicely formatted table ready for machine learning.
I hope this helps.
Regards,
Alex
Hopefully this will help. I am running this Python code inside of Rapidminer as I prefer to use that for many machine learning tasks especially in the early phase of testing ideas. This requires that I use dataframes but the code should be straight forward.
In this example, I am exporting six data points from Multicharts and generating an ID and datetime stamp.
Code: Select all
//Generate Realms
OutputDateString = FormatDate(OutputDateFormat, ELDateToDateTime(Date));
bartime = FormatTime("HH:mm:ss", DateTime );
once begin
BarID = svinit("ID_"+NumtoStr(barinterval,0));
VOpenLink = svinit("VOpen_"+NumtoStr(barinterval,0));
VHighLink = svInit("VHigh_"+NumtoStr(barinterval,0));
VLowLink = svInit("VLow_"+NumtoStr(barinterval,0));
VCloseLink = svInit("VClose_"+NumtoStr(barinterval,0));
SSHighLink = svInit("VSSHigh_"+NumtoStr(barinterval,0));
SSLowLink = svInit("VSSLow_"+NumtoStr(barinterval,0));
svRemoveAll(BarID);
svRemoveAll(Closelink);
svRemoveAll(VOpenLink);
svRemoveAll(VHighLink);
svRemoveAll(VLowLink);
svRemoveAll(VCloseLink);
svRemoveALl(SSHighLink);
svRemoveAll(SSLowLink);
end;
if barstatus(0) = 2 then
begin
svSetValue(BarID,OutputDateString+" "+bartime,barnumber);
svSetValue(VOpenLink,OutputDateString+" "+bartime,VOpen);
svSetValue(VHighLink,OutputDateString+" "+bartime,VHigh);
svSetValue(VLowLink,OutputDateString+" "+bartime,VLow);
svSetValue(VCloseLink,OutputDateString+" "+bartime,VCLose);
svSetValue(SSHighLink,OutputDateString+" "+bartime,SSHigh);
svSetValue(SSLowLink,OutputDateString+" "+bartime,SSLow);
end;
Code: Select all
import pandas as pd
from ctypes import *
def rm_main():
lib = cdll.LoadLibrary("SharedVar-w64.dll")
linkID =(lib.svInit("ID_65"))
link1 =(lib.svInit("VOpen_65"))
link2 =(lib.svInit("VHigh_65"))
link3 =(lib.svInit("VLow_65"))
link4 =(lib.svInit("VClose_65"))
link5 =(lib.svInit("VSSHigh_65"))
link6 =(lib.svInit("VSSLow_65"))
lib.svGetDouble.restype = c_double
lib.svVarName.restype = c_wchar_p
MaxBars = lib.svVarCount(linkID)
data=[]
for x in range(1,MaxBars):
BarDateTime = lib.svVarName(linkID, x)
varID = lib.svGetInt(linkID,BarDateTime)
varValue1 = lib.svGetDouble(link1,BarDateTime)
varValue2 = lib.svGetDouble(link2,BarDateTime)
varValue3 = lib.svGetDouble(link3,BarDateTime)
varValue4 = lib.svGetDouble(link4,BarDateTime)
varValue5 = lib.svGetDouble(link5,BarDateTime)
varValue6 = lib.svGetDouble(link6,BarDateTime)
data.append((varID,BarDateTime,varValue1,varValue2,varValue3,varValue4,varValue5,varValue6))
cols = ['ID','DateTime','VOpen','VHigh','VLow','VClose','SSHigh','SSLow']
df = pd.DataFrame(data, columns=cols)
df = df.sort_values(['ID'], ascending=True)
return(df)
Code: Select all
lib.svGetDouble.restype = c_double
lib.svVarName.restype = c_wchar_p
I hope this helps.
Regards,
Alex
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi Alex,Hi MJ,
Hopefully this will help. I am running this Python code inside of Rapidminer as I prefer to use that for many machine learning tasks especially in the early phase of testing ideas. This requires that I use dataframes but the code should be straight forward.
In this example, I am exporting six data points from Multicharts and generating an ID and datetime stamp.
On the Python side, the code looks like this....Code: Select all
//Generate Realms
OutputDateString = FormatDate(OutputDateFormat, ELDateToDateTime(Date));
bartime = FormatTime("HH:mm:ss", DateTime );
once begin
BarID = svinit("ID_"+NumtoStr(barinterval,0));
VOpenLink = svinit("VOpen_"+NumtoStr(barinterval,0));
VHighLink = svInit("VHigh_"+NumtoStr(barinterval,0));
VLowLink = svInit("VLow_"+NumtoStr(barinterval,0));
VCloseLink = svInit("VClose_"+NumtoStr(barinterval,0));
SSHighLink = svInit("VSSHigh_"+NumtoStr(barinterval,0));
SSLowLink = svInit("VSSLow_"+NumtoStr(barinterval,0));
svRemoveAll(BarID);
svRemoveAll(Closelink);
svRemoveAll(VOpenLink);
svRemoveAll(VHighLink);
svRemoveAll(VLowLink);
svRemoveAll(VCloseLink);
svRemoveALl(SSHighLink);
svRemoveAll(SSLowLink);
end;
if barstatus(0) = 2 then
begin
svSetValue(BarID,OutputDateString+" "+bartime,barnumber);
svSetValue(VOpenLink,OutputDateString+" "+bartime,VOpen);
svSetValue(VHighLink,OutputDateString+" "+bartime,VHigh);
svSetValue(VLowLink,OutputDateString+" "+bartime,VLow);
svSetValue(VCloseLink,OutputDateString+" "+bartime,VCLose);
svSetValue(SSHighLink,OutputDateString+" "+bartime,SSHigh);
svSetValue(SSLowLink,OutputDateString+" "+bartime,SSLow);
end;
The only tricky part is this part which is dealing with the differences between python types and c types. This probably where you are having your problem.Code: Select all
import pandas as pd
from ctypes import *
def rm_main():
lib = cdll.LoadLibrary("SharedVar-w64.dll")
linkID =(lib.svInit("ID_65"))
link1 =(lib.svInit("VOpen_65"))
link2 =(lib.svInit("VHigh_65"))
link3 =(lib.svInit("VLow_65"))
link4 =(lib.svInit("VClose_65"))
link5 =(lib.svInit("VSSHigh_65"))
link6 =(lib.svInit("VSSLow_65"))
lib.svGetDouble.restype = c_double
lib.svVarName.restype = c_wchar_p
MaxBars = lib.svVarCount(linkID)
data=[]
for x in range(1,MaxBars):
BarDateTime = lib.svVarName(linkID, x)
varID = lib.svGetInt(linkID,BarDateTime)
varValue1 = lib.svGetDouble(link1,BarDateTime)
varValue2 = lib.svGetDouble(link2,BarDateTime)
varValue3 = lib.svGetDouble(link3,BarDateTime)
varValue4 = lib.svGetDouble(link4,BarDateTime)
varValue5 = lib.svGetDouble(link5,BarDateTime)
varValue6 = lib.svGetDouble(link6,BarDateTime)
data.append((varID,BarDateTime,varValue1,varValue2,varValue3,varValue4,varValue5,varValue6))
cols = ['ID','DateTime','VOpen','VHigh','VLow','VClose','SSHigh','SSLow']
df = pd.DataFrame(data, columns=cols)
df = df.sort_values(['ID'], ascending=True)
return(df)
The end result is a nicely formatted table ready for machine learning.Code: Select all
lib.svGetDouble.restype = c_double
lib.svVarName.restype = c_wchar_p
DataTable.JPG
I hope this helps.
Regards,
Alex
Thanks for your reply.
With your code, i understand what i wasn't doing right.
I didn't know i had to init the realm on both MC and Python with the same string parameter. Now it works like a charm.
Thank you again for your great help.
Now i can connect MC to python using SharedVar or Lua dll and it's great.
BTW, contacted fx1 about the dvGetString function for the Sharedvar-a64.dll et they confirmed the issue.
Regards,
MJ
-
- Posts: 275
- Joined: Apr 22 2014
- Has thanked: 70 times
- Been thanked: 72 times
Great MJ, I am glad that you got it working. Enjoy the rest of your weekend. Give Rapidminer a try. It is especially useful for validating your time series models. Once you are done with your testing, you can then call your python code directly. It is very easy use and is a huge time saver.
Regards,
Alex
Regards,
Alex
-
- Posts: 114
- Joined: Jul 12 2015
- Has thanked: 20 times
- Been thanked: 16 times
Hi,
i'll give Rapidminer a try, this tool looks great.
Regards,
MJ
i'll give Rapidminer a try, this tool looks great.
Regards,
MJ
-
- Posts: 275
- Joined: Apr 22 2014
- Has thanked: 70 times
- Been thanked: 72 times
Hi @wilkinsw,
After 30 days, the trial turns into a community edition. A limit of 10,000 rows and single threaded. It is still extremely capable. You can also run R and Python within your process, deal with databases etc. It is an amazing tool. Single threaded is not the limitation that you might think it is. Many machine learning operators like neural nets lack repeatability when multithread. That is not what you want when back testing so you would have to switch it off anyway.
regards,
Alex
After 30 days, the trial turns into a community edition. A limit of 10,000 rows and single threaded. It is still extremely capable. You can also run R and Python within your process, deal with databases etc. It is an amazing tool. Single threaded is not the limitation that you might think it is. Many machine learning operators like neural nets lack repeatability when multithread. That is not what you want when back testing so you would have to switch it off anyway.
regards,
Alex