Suggestion: PowerEditor Speed

Questions about MultiCharts and user contributed studies.
User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Suggestion: PowerEditor Speed

Postby Bruce DeVault » 04 Feb 2010

PowerEditor seems to be come very sluggish when editing long files such as 10,000 lines or longer, especially if syntax highlighting is turned on using the registry setting. This seems to indicate a design that's having to cycle through the entire file to determine context to do the highlighting. If there's any way this could be redone to somehow be much quicker, it would be helpful. Notepad or Wordpad, for instance, by comparison, are lightning fast on the same files, opening and scrolling in a split second, but on a state of the art I7 machine it's taking almost a minute to load up some files in PowerEditor.

janus
Posts: 835
Joined: 25 May 2009
Has thanked: 63 times
Been thanked: 104 times

Re: Suggestion: PowerEditor Speed

Postby janus » 05 Feb 2010

Yes, I've experienced the same issue. To get around it I've decided to write all my complex rules in another language and access it from a much simplified study in MC as a DLL. That way I've made it more portable so if I ever decide to use another package (unlikely at this stage) I won't have to rewrite all the rules. It runs a lot faster as well as compiles faster (I use FreeBasic and FBEdit or FBIde but there are many other programming languages and development tools available). I can't recall the performance difference but I think it's 2 or 3 times faster.

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 05 Feb 2010

Thanks for your comment. I do understand re: accessing DLLs, and do development of DLLs extensively. But, I would much like to see TS Support look at this to see what they can't do to make the PowerLanguage editor more efficient, as there's clearly a design issue at play. In most other areas of the platform, it seems as if a great deal of effort has gone into performance optimization, so I have no doubt they can fix this.

Spaceant
Posts: 254
Joined: 30 May 2009
Has thanked: 1 time
Been thanked: 3 times

Re: Suggestion: PowerEditor Speed

Postby Spaceant » 06 Feb 2010

Yes, I've experienced the same issue. To get around it I've decided to write all my complex rules in another language and access it from a much simplified study in MC as a DLL. That way I've made it more portable so if I ever decide to use another package (unlikely at this stage) I won't have to rewrite all the rules. It runs a lot faster as well as compiles faster (I use FreeBasic and FBEdit or FBIde but there are many other programming languages and development tools available). I can't recall the performance difference but I think it's 2 or 3 times faster.
Hi Janus,

I am new to this in developing a DLL. To a starter, what language will you recommend? And where can I find the resources / information that I can develop a DLL?

Thanks in advance!

Sa

janus
Posts: 835
Joined: 25 May 2009
Has thanked: 63 times
Been thanked: 104 times

Re: Suggestion: PowerEditor Speed

Postby janus » 06 Feb 2010

I am new to this in developing a DLL. To a starter, what language will you recommend? And where can I find the resources / information that I can develop a DLL?
Sa
I think the easiest one to use is FreeBasic or PowerBasic - both are very similar. The first is free as the name implies but the latter has some nicer features for really serious programmers. Perhaps start with FreeBasic. I use FBIde as the IDE, which is very basic but enough for my purposes. FBEdit is more sophisticated as it's very similar to Visual Studio. It's good if you want to develop Windows gui based applications. I use the manual approach by writing a lot of my Windows apps using FBIde. I wouldn't recommend it for a novice. If you just want to write rules for processing and not interested in fancy windows gui front ends then FBIde is sufficient, although the fancier FBEdit might be better suited if you prefer point and click/ drag and drop development tools. I'm old school and like to type my code in manually. I'm not used to thinking in 3 dimensions, which is how I think most VS like developers think. I'm more a 2-d person when it comes to programming. Having said all that, if I ever had to write a serious Windows gui app I think I spend the time learning how to work in a VS like environment. I would also use VB.net to take advantage of some useful MS features, like SQL database integration. However, i doubt I will ever need to as my focus now is trading, and I have better things to do in my spare time.

DLL resources are available on MC's online help & tutorial area http://www.tssupport.com/multicharts/tutorials/. Also, the FreeBasic forum http://www.freebasic.net/forum/ has many useful tips and hints on writing DLL's. I'll post a simple DLL example later.

janus
Posts: 835
Joined: 25 May 2009
Has thanked: 63 times
Been thanked: 104 times

Re: Suggestion: PowerEditor Speed

Postby janus » 06 Feb 2010

As promised here's some sample DLL code using FreeBasic. The screen shots are how the FB looks in FBIde and FBEdit. As I said FBIde is the simplest to use, but FBEdit is better for Windows gui developers as it's a lot like Visual Studio in terms of look and feel.
Attachments
study.txt
(459 Bytes) Downloaded 373 times
example_code.txt
(1 KiB) Downloaded 368 times
FBIde_screenshot.png
(23.79 KiB) Downloaded 1474 times
FbEdit_screenshot.png
(55.21 KiB) Downloaded 1491 times

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Postby TJ » 06 Feb 2010

Janus:

Thanks for sharing your knowledge in programming DLL.

Best regards
TJ

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 07 Feb 2010

Thank you for your comments fellows but please help keep this thread on track which is discussion of the speed of PowerEditor in editing large files.

While DLL / BASIC examples are helpful for those looking for that, let's start a new thread for that with an appropriate topic so it's easier to find those examples, and so this issue doesn't get side-tracked indefinitely by a discussion that is no longer about PowerEditor speed with large files.

The subject of this thread is something that TS Support can directly address, and it's in everyone's interest for them to understand this is an actionable request and one that would benefit many.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 07 Feb 2010

I noticed the same thing about the editor but I don't think it is any slower than the MC 2.1.999.999 version I was just using last December. It would still be nice if it could have that problem fixed by my approach to having it fixed is a lot more extensive. If some of my ideas below were put in there would be no need to write 10,000 lines which is a very large module. I just checked my largest which is 3,000 (I have about 8 to 10 studies per chart and my average is probably 500 lines with maybe 3 over 1000). Having said that I am a discretionary trader (and will always be since my approach is too complex to fully program) so I do not need as much code. Most of my code is in the database program and it actually does a syntax check on the fly and is faster than the MC editor. I probably have a few 5,000 line modules in there but I never check for size. After having said 10,000 lines is a lot, I figure if I could somehow fully automate my approach it would be 100,000 lines of code at least. I worked as a full time programmer for 19 years and I saw a few programs up at 12,000 lines. Two were my programs. That is unusual. I used about 11 languages used over those 19 years. Unfortunately EL is not that great a language for making things modular (you must have some very deep nested if statements or have started using a bypass switch at the start of select if statements to avoid getting too deep in nested ifs - something I just used in a study). VBA is nice in that you can define a variable as public and get at it from anywhere within that database without having to put it in the string of variables at the back of a function (that really slows programming down if that is the only way you can pass data to the function). El is the only language I have used that forces me to do that. GVs are not a great work around for public variables which are much easier to work with than than GVs. If I could define a variable in MC as public to a chart or public to a workspace or public to MC that would be a major improvement (assuming MC did not slow down at all). Then I would only use GVs for passing data outside of MC to another program. Actually my 3,000 line module would have been much smaller if public variables could be defined. I remember I tried to make a major part of it a function and I ended up with about 60 variables I had to pass to it and I just gave up and duplicated the code (copy and paste is much easier for programming than maintaining 60 variables in a function).

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 07 Feb 2010

I am not opposed to structural enhancements to EasyLanguage to make it easier to do structured programming. But, let's not lose track of the fact that there's a problem with the editor and it would presumably not be that hard for them to fix it (since NotePad and WordPad work fine in this regard and edit large files instantly), whereas radically reinventing EasyLanguage would take some consideration and would best be the subject for other threads specifically about the evolution of the language.

tekram
Posts: 96
Joined: 26 May 2009
Has thanked: 6 times
Been thanked: 18 times

Postby tekram » 22 Feb 2010

In TS 8.8, we are providing you with a new advanced EasyLanguage programming editor that we have named the TS Development Environment (TDE).
Image
Attachments
TS8.8TDE.gif
(59.39 KiB) Downloaded 1444 times

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 22 Feb 2010

Yes, the TS 8.8 TDE is pretty good - it's a big improvement.

User avatar
RobotMan
Posts: 375
Joined: 12 Jul 2006
Location: Los Altos, California, USA
Has thanked: 31 times
Been thanked: 13 times
Contact:

Postby RobotMan » 22 Feb 2010

That looks pretty sweet. I like the dictionary drag and drop, too.
I also miss a split window in PLE.

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 22 Feb 2010

On the subject of PowerEditor speed, I loaded up the new TS Development Environment, and pasted copies of the ADX indicator by holding down ^V until it was about 20,000 lines long. At that point, I did some experiments with scrolling, hitting ctrl-home, ctrl-end, etc. and everything worked flawlessly and immediately, with scrolling 20,000 lines taking way less than a second. Syntax coloring worked perfectly for all 20,000 lines. So, we know this can be done - it's just a matter of fixing MultiCharts' PowerLanguage editor. It's clearly a design problem, and it needs to be fixed.

janus
Posts: 835
Joined: 25 May 2009
Has thanked: 63 times
Been thanked: 104 times

Postby janus » 23 Feb 2010

Yes, PLEditor has some serious bugs when editing large files. Also, the undo feature breaks sometimes and one ends up undoing only part of a previous edit. It can get so bad that I just give up and restore the original file.

On another matter, I like to see run-time errors display where the error occurred, that is the study or function name and the line number. If necessary this can be turned off at compile time to avoid performance issues. For example if I get an array bound error, there's no indication of which array was the cause, let alone where. I then have to insert print statements to find where the error occurs. In this day and age of modern compilers, we shouldn't have to resort to such primitive methods to debug code.

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 23 Feb 2010

I agree with this. Whenever possible, at least, the line number of the source code causing the error should be included in the error report.

User avatar
RobotMan
Posts: 375
Joined: 12 Jul 2006
Location: Los Altos, California, USA
Has thanked: 31 times
Been thanked: 13 times
Contact:

Postby RobotMan » 23 Feb 2010

Amen to Janus.

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 24 Feb 2010

TS Support, based on these test results of newly released 8.8 TDE, clearly it's possible to have an editor with syntax checking etc. that works quite well for large files. What would it take to get good results when editing large files on the PowerLanguage editor?

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 25 Feb 2010

10,000 lines of code!!!!! :shock: :shock: :shock: :shock:

Wow. All of my best strategies have fewer then 200. I guess I'm a believer that the markets aren't that complicated... at least the way I trade it.

One thing I have learned is there are many ways to skin a cat. So I'm not going to say you're crazy for thinking money can be made with that much complexity.

Do you mind giving a little info about this 10,000 line strategy? Is this some sort of super complex arb spreader?

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 26 Feb 2010

The most common reason EL strategies get very long is the desire to write something very general that can be used to explore/leverage a number of different ideas within a common framework. It's a simple design decision whether or not to put a number of things into one strategy, or to have separate strategies, and different users have different preferences in this regard.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 26 Feb 2010

Hi flipflopper,

Are your short strategies generally average based strategies? I am thinking of ones based off MacD crossovers, multi moving averages and the like. I find these can be very short but they enter the market way to late. I now use wave projections, bounces, break detection, and forward stall detection. These types of strategies can get very complex. My approach is so complex that it is far to hard to program the whole thing (or at the very least not worth the effort when I can do it so much better visually). It works very well but how well depends on who is trading it. Generally discretionary traders remain so because their systems are too complex to fully program. I try to automate as many of these segements as I can to ease the work load and make it so I do not miss things.

The other thing is some programmers use a lot of indenting, commenting and white space. This adds up fast. With the more complex strategies it also becomes more important to do this.

John.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 26 Feb 2010

Thanks for the responses. In my very short experience of automating systems I have found that when I have a good profitable strategy when I try to add too many ideas it seems to always erode the systems profitability.

I have been a discretionary trader full time for 3 years and have constantly found that the market is not as clever as I think I am. I usually does the same things over and over again.

The reason I think most traders don't last and even ones that are successful end up blowing up is because the markets seem to have about 4 different modes they go into. Each lasting between a 12-18 months. Once out of every 5 years or so we get a really easy market that discretionary traders can get rich on. When the party is over they don't realize it and out of frustration and ego feel they have to be right and average into a killer.

The reason I'm saying all this is because I feel it can't be known when the markets are going to change and when they are going to work for your strategy. I have automated the systems I have found to make a good amount of money and will let them sit in the markets until another great run occurs. If I could find a way to identify the different markets and adapt strategies I would but I just feel when I try to do this my simple strategies would have made more money anyways. In the end I guess that's what its all about.

flipflopper
Posts: 261
Joined: 28 Feb 2008
Has thanked: 2 times
Been thanked: 1 time

Postby flipflopper » 26 Feb 2010

Hi flipflopper,

Are your short strategies generally average based strategies? I am thinking of ones based off MacD crossovers, multi moving averages and the like. I find these can be very short but they enter the market way to late. I now use wave projections, bounces, break detection, and forward stall detection. These types of strategies can get very complex. My approach is so complex that it is far to hard to program the whole thing (or at the very least not worth the effort when I can do it so much better visually). It works very well but how well depends on who is trading it. Generally discretionary traders remain so because their systems are too complex to fully program. I try to automate as many of these segements as I can to ease the work load and make it so I do not miss things.

The other thing is some programmers use a lot of indenting, commenting and white space. This adds up fast. With the more complex strategies it also becomes more important to do this.

John.
I also used to think my strategies were too complex to program since they were based entirely on price action. But in the end I learned that anything your brain can think of can be automated. If you can see it so can the program. If you don't program your ideas you won't know if they are profitable or not. I used to have 9 signals that I followed in my discretionary trading it turned out only 4 were profitable after backtesting!!!

The only time I feel a discretionary trader can win is during a very volatile market. During Fed meetings, after earnings, breaking news, etc. The rest of the time I have realized I can't out perform my systems.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 26 Feb 2010

Hi Flipflopper.
The only time I feel a discretionary trader can win is during a very volatile market. During Fed meetings, after earnings, breaking news, etc. The rest of the time I have realized I can't out perform my systems.
I think you nailed the difference. I am very short term. But maybe there is another point. Maybe there is a "discretionary trader" and a "systems discrectionary trader". One without rules and one with rules (system implies rules). The only difference is the input for answering the rules for the "systems discrectionary trader" is 70% eyes rather than computer code. If I see a break do I need a computer to tell me? Hardly. I ensure I follow my rules by a checklist (lately I have been memorizing it and will soon start stop watch timing of the speed of that memory work, but in the past I used a database voice step through before every single trade to ensure I follow my rules). I actually have a set of 5 to 10 rules for each market situation (there is overlap and some basic underlying concepts). Some of the input is the eyes reading computer code indicators and deciding the rules have been met. All that matters is that the trader has certain rules that work most of the time and they have some sort of method to ensure that they are followed most of the time (database voice read step through, high speed memory recall or rigid computer EL code - does it matter?). Human input is more flexible (would Tiger Woods program his golf game?) In actual fact I do a lot better job at inputting than EL code since often the EL code can't figure out something new yet I can see through it quickly (after all, I would have to program the fix and who knows how many fixes there are). In the end I use a blend of all three rule answering input methods.

Anyway, I think this is off topic (I beat Bruce to it :D - I think he will add an agreement.). If you want to discuss strategies and rule answering input methods, PM me. However, I have found discussing strategy is often a waste of time since each finds what works for them (it can be no other way).

John

User avatar
Bruce DeVault
Posts: 438
Joined: 19 Jan 2010
Location: Washington DC
Been thanked: 2 times
Contact:

Postby Bruce DeVault » 04 Mar 2010

Clearly there is a sluggishness problem with the editor. That there's a deficiency in the design that's causing specific headaches and it would be great if this could be addressed in coming builds.

bowlesj3
Posts: 2180
Joined: 21 Jul 2007
Has thanked: 227 times
Been thanked: 429 times

Postby bowlesj3 » 04 Mar 2010

Agreed.


Return to “MultiCharts”