MC Portfolio Back-Testing and Optimization Feedback Request

Questions about MultiCharts and user contributed studies.
User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

MC Portfolio Back-Testing and Optimization Feedback Request

Postby Marina Pashkova » 22 Aug 2007

Dear User,

After many additional improvements and enhancements MultiCharts has
become one of the fastest and most flexible tools for strategy
back-testing and optimization.

MultiCharts platform offers a large number of functions, allowing for
implementation of almost any conceivable strategies. By taking advantage
of multi-core CPU technology our platform’s advanced architecture offers
a dramatic performance advantage over most competing products.

We are currently developing portfolio back-testing and optimization
functions for our platform, and would like to ask you to help us realize
the full potential of portfolio modeling. Please let us know which
portfolio strategies you would like to implement. A short description
and/or an example of code, with comments, would be sufficient.

Please post your ideas on our forum. For more information
about MultiCharts visit http://tssupport.com/products/multicharts/

Thank you in advance,
TS Support Team

Bubba

Re: MC Portfolio Back-Testing and Optimization Feedback Requ

Postby Bubba » 26 Aug 2007

On Aug 22 Marina wrote >

Dear User,

After many additional improvements and enhancements MultiCharts has
become one of the fastest and most flexible tools for strategy
back-testing and optimization.

TS Support Team


************

I have a question in this area. Just about two full years ago a former member of your staff relayed that the back testing, portfolio optimization, and genetic optimization were proceeding along well and that they would soon be implemented in MC.

Two of the websites pointed out to me as showing some of what would be available in MC are

www.traderssoft.com/sys/
and
www.tsresearch.com/software/

If I am not mistaken,up to about a month or so ago both of these sites had clickable icons leading to the MC page/program.

Are they still your partners or part of your company?

Anyway,I do not understand why there has been problems reported with the Portfolio and Genetic parts of MC when these were available two years ago as add-ons to TS by some branch of your company?

The Genetic Optimizer relates -
"Unlimited number of optimized parameters (up to 100 in the current version);"

www.traderssoft.com/sys/go/
or
www.tsresearch.com/software/genetic_opt ... optimizer/

And the Portfolio Management or PortfolioAnalyzer relays

" In contrast to the contemporary equity management programs RealTime PortfolioAnalyzer allows:

To manage a portfolio of assets and strategies from a single account in the real-time.
To use any algorithms of equity management not limited by a small number of standard methods. The only existing limitation is trader’s imagination.
To analyze not the results of particular transactions (which are nothing else but realizations of a random value and reflect a trader’s desire “to be right”) but of equity growth, which really make sense as they reflect alterations in trader’s actual wealth.
To export all necessary portfolio parameters into the TS in the real-time in order to manage current risks.
To calculate equity growth statistics for different periods in percent and adjusted to current prices.
To calculate the portfolio drawdowns and runups statistics in percent and adjusted to current prices.
To test and analyze any equity management strategies on the portfolio level based on historical data.
In addition, RealTime PortfolioAnalyzer allows:

To combine different assets with different timeframes and different strategies.
To introduce portfolio total equity and components alterations, equity growth histograms for various periods, separate components’ share in the portfolio indices etc. using graphs.
Finally, RealTime PortfolioAnalyzer

www.traderssoft.com/sys/rtpa/
or
www.tsresearch.com/software/portfolio/p ... _analyzer/

My question is why,after more than two years After release of these products for TS,there should still be any problems at all with the MC release?

This ex-employee also mentioned that a far superior product to TS RadarScreen was being developed too at the time-is this due out in a future release of MC,or is one of the above products a substitute?

Thanks

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Postby glam_100 » 27 Aug 2007

I suggest the TSSUpport team take a look at wealth-lab as well as RINA for ideas.

Wealth-lab is a great product, all they lack is performance. It is so slow that renders intraday testing completely impossible, not to mention portfolio backtesting.

RINA has everything only trouble is it is not fully integrated into TS.

I think if Tssupport can create something like an integrated TS + RINA it will be the greatest product.

Guest

Postby Guest » 27 Aug 2007

Would be very powerful to have user defineable and multiple/compound fitness functions for optimization criteria.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 27 Aug 2007

Thank you very much for your suggestions. We appreciate your input and are going to include them into a list of options we are considering.

neurotrader
Posts: 86
Joined: 29 Dec 2005
Location: CANADA
Contact:

Postby neurotrader » 27 Aug 2007

One thing that I have asked awhile back and would love to see is a "System Equity" analysis that one can plot under the chart similar to TS.

After the optimization the System Equity analysis would complement.

Excellent tool for a visual as your system takes it triggers.

:P

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Re: MC Portfolio Back-Testing and Optimization Feedback Requ

Postby Marina Pashkova » 28 Aug 2007

On Aug 22 Marina wrote >

Dear User,

After many additional improvements and enhancements MultiCharts has
become one of the fastest and most flexible tools for strategy
back-testing and optimization.

TS Support Team


************

I have a question in this area. Just about two full years ago a former member of your staff relayed that the back testing, portfolio optimization, and genetic optimization were proceeding along well and that they would soon be implemented in MC.

Two of the websites pointed out to me as showing some of what would be available in MC are

www.traderssoft.com/sys/
and
www.tsresearch.com/software/

If I am not mistaken,up to about a month or so ago both of these sites had clickable icons leading to the MC page/program.

Are they still your partners or part of your company?



The above-mentioned are our partners.


Anyway,I do not understand why there has been problems reported with the Portfolio and Genetic parts of MC when these were available two years ago as add-ons to TS by some branch of your company?

************************

My question is why,after more than two years After release of these products for TS,there should still be any problems at all with the MC release?

This ex-employee also mentioned that a far superior product to TS RadarScreen was being developed too at the time-is this due out in a future release of MC,or is one of the above products a substitute?

Thanks


Once again, the above mentioned are our partners and the products referred to have not been developed by TS Support.

Apart from that, the letter opening this thread is not for fixing the problems but for getting the users' perspective on what porfolio back-testing and optimization features would be useful. With this goal in mind we asked our customers to share their ideas on what strategies they use/would like to use. This multitude of ideas would enable us to make back-testing and optimization really efficient in meeting our portfolio trading customers' needs.

drwar
Posts: 218
Joined: 31 Jul 2005

Re: MC Portfolio Back-Testing and Optimization Feedback Requ

Postby drwar » 28 Aug 2007

[quote]Dear User,

After many additional improvements and enhancements MultiCharts has
become one of the fastest and most flexible tools for strategy
back-testing and optimization.

MultiCharts platform offers a large number of functions, allowing for
implementation of almost any conceivable strategies. By taking advantage
of multi-core CPU technology our platform’s advanced architecture offers
a dramatic performance advantage over most competing products.

We are currently developing portfolio back-testing and optimization
functions for our platform, and would like to ask you to help us realize
the full potential of portfolio modeling. Please let us know which
portfolio strategies you would like to implement. A short description
and/or an example of code, with comments, would be sufficient.

Please post your ideas on our forum. For more information
about MultiCharts visit http://tssupport.com/products/multicharts/

Thank you in advance,
TS Support Team[/quote]

Quite frankly I don't understand why you are taking the resources to do this step while their are so many other related issues outstanding.

1) Full TS compatible strategy instruction set
2) Intrabar order generation
3) Alternate brokers that are reliable

to mention just a few.

J~

bunkerbuster
Posts: 10
Joined: 07 Feb 2006

Re: MC Portfolio Back-Testing and Optimization Feedback Requ

Postby bunkerbuster » 28 Aug 2007

I think that like other user suggest you can find a lot of help from RINA portfolio evaluator, money manager and portfolio stream, on Meyers analitycs and on Grail Suite Optimizator.
Sure RINA is a mess if you look the speed of optimization/work and the database seems not very efficient (I did test it in past).
But is plenty of metrics and I saw latest build permit also Monte Carlo analisys.

By my point of view Is a MUST for a trader platform today to have GA optimization, backtesting capabilities, portfolio optimization and portfolio performance report. I will think about a real WALK FORWARD ANALISYS that is the MUST to know if your strategy is curve fitting the market or not.
But seems lot of strategy trader not understand yet this.

But there's a point... drWar is correct. We need a real "perfect" engine that use intrabar calculation for entry and stop execution at the tick, we need to be sure that the strategy results are true..... if not all that you have in cascade is useless.

So, I'm happy MC is alwasy working on user request but the suer must understand what is the priority and the time to develop it.

Thank you
BunkerBuster

Nick
Posts: 490
Joined: 04 Aug 2006
Has thanked: 4 times
Been thanked: 24 times

Postby Nick » 29 Aug 2007

First of I am a discretionary trader but now and then mess with things like NeuroShell daytrader and Biocomp profit.

Are tick filtering and holidays supported yet? Pre requisites for strategy testing. Reliable data is important and manually cleaning it is laborious.

I've always liked the idea of adaptive algorithms which means the genetic optimizer would need to run from charts to update bar to bar. I love the idea of Bicomp's Dakota http://www.biocompsystems.com/products/Dakota/

I'd also like to see easy ways to do pattern recognition in the data and to do statistical analysis, EL is not ideal for this sadly.

I'd like to see support for other data streams (e.g. full order book/DOM) to research strategies that are not solely based on price.

My leaning is towards using the software to construct and test new tools rather than construct and test systems. For me, there is more value in general research than curve fitting a bunch of conventional indicators to my data. Anyway who wants great back tests but average actual results. :D Ok I am just being provocative here. :D

Cheers,
Nick.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 29 Aug 2007

I suggest the TSSUpport team take a look at wealth-lab as well as RINA for ideas.

Wealth-lab is a great product, all they lack is performance. It is so slow that renders intraday testing completely impossible, not to mention portfolio backtesting.

RINA has everything only trouble is it is not fully integrated into TS.

I think if Tssupport can create something like an integrated TS + RINA it will be the greatest product.


Hello,

What is it exactly that you like about RINA?

glam_100
Posts: 157
Joined: 14 Jun 2006
Has thanked: 2 times
Been thanked: 4 times

Postby glam_100 » 29 Aug 2007

I suggest the TSSUpport team take a look at wealth-lab as well as RINA for ideas.

Wealth-lab is a great product, all they lack is performance. It is so slow that renders intraday testing completely impossible, not to mention portfolio backtesting.

RINA has everything only trouble is it is not fully integrated into TS.

I think if Tssupport can create something like an integrated TS + RINA it will be the greatest product.


Hello,

What is it exactly that you like about RINA?


1) The ability to create/manage portfolio group under a hierachy tree
2) Support multiple currency (not just a fixed currency conversion, but rather they support a text file with daily exchange rate conversion)
3) Correlation matrix with the option to select daily/weekly/monthly equity for calculation as well as using price for calculation (ie. able to see different correlation)
4) Combined performance report with same performance matrix as single system performance report
5) Monte Carlo sim with choice to select daily equity or trade as individual draw. Also *very* important it supports user selected # of trails because without this Monte Carlo sim is meaningless. Many softwares runs a monte carlo sim across your whole sample and don't let you specify the # trail (whether smaller than number of sample or bigger than # of sample) and that is a wrong implementation. Also it allows for different draw scheme like sample replacement or without sample replacement. It doesn't allow an option to maintain serial correlation maybe that's something you would like to add to MC. Wealth-lab monte carlo lab has the maintain serial correlation feature and is something worth doing.
6) Various money management schemes
7) Out of sample optimization and walk forward analysis might be something worth adding. Many users are crazy about it.

Nice to have:

8) Portfolio optimization. Adding genetic portfolio optimization might be nice.

That's what I can think of out of my head. I think the biggest strength of RINA is it's reporting format. It's very logical and includes almost every reports you want to see.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 29 Aug 2007

Thank you for the very valuable suggestion. That's something we are definitely going to think about. Not all of the features you are describing will be implemented in the next MC 2.2 release. But we will try to implement them in later releases.

rsheftel
Posts: 17
Joined: 10 Mar 2006

Postby rsheftel » 11 Sep 2007

I would suggest that when you do portfolio optimization, you include the ability to run through a set of symbols for all the data streams. So that for a given run, the setup would define the symbol for data1, symbol for data2, etc. This is very important for any system that uses the supporting signal data to make decisions. A simple example would be trading an equity against an ETF that it is a part of. So during the portfolio backtesting you would want to run the combinations:

data1, data2
MSFT, TechnologyETF
AAPL, TechnologyETF
GM, AutoETF
...

This is a stylized example but should get the point across.

I have not seen anything on the market (including RINA) which is capable of this. It is also very important for looking at spread-trading.


My next suggestion would be to build in exposure management. So that you could control that across the entire portfolio limit the number of open trades to X. Or create a subset so that you cannot be long more that Y shares of all AUTO sector stocks.

The market for portfolio testing and monitoring is wide open as most of the tools I have seen on the market are good at a limited subset.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 12 Sep 2007

Hello,

Could you use some exact figures to illustrate your idea? We need more details to fully understand what the feature you are talking about should be like.

Thank you.

rsheftel
Posts: 17
Joined: 10 Mar 2006

Postby rsheftel » 12 Sep 2007

Let me give a better example, sorry it was unclear.

I have the following strategy idea: If an index (NDX, SPX, etc) closes higher over the prior 5 days, and the stock is also higher, then buy. (simple example)

The data would be:

data1 - the stock (example: MSFT)
data2 - the index (example: SPX)

The code would be

if close of data2 > close[5] of data2 then
if close of data1 > close[5] of data1 then
buy next bar at market.

So in the portfolio backtest I want to look at the results for the combination of each market and index. So I would want to run through the following:

data1 data2
MSFT SPX
MSFT NDX
AAPL SPX
AAPL NDX
GOOG SPX
GOOG NDX

....


Another example (more real):

We do a lot of trading of stocks versus their residuals from a multi-factor regresssion. We perform all of the regression work and calculate and store the residuals. So here we will be doing the decision making based on the residual time series (in data2), and doing the trade on the underlying stock (in data1). So when running a portfolio backtest, at each cycle I need to insert a new symbol into data1 and data2.

data1 data2
AAPL Residual of AAPL (this is calculated outside MC and stored in a database/ascii)
GOOG Residual of GOOG

I think this is very common in more advanced places as you require often more than one data stream to make a decision.

Check out www.quanthouse.com they have the ability to do this.

In many ways for us, without this ability the portfolio would be of limited value only to strategies where only data1 is used as an input into the decision making. righ now in TS we use BTSeqMgr (3rd party) and ADE to sync and create this process, but I would much rather do with you.


Last comment, for anything having to do with portfolio backtesting, or the live portfolio watcher (like Radarscreen), please allow the symbol list to be imported from a file (csv, excel, etc). It would be a real pain to have to type it in by hand into a dialog box.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 14 Sep 2007

Hello,

Since you contacted us directly I we won't be posting the answer to your suggestion here on forum.

Thank you.

traderstuff
Posts: 68
Joined: 24 Jul 2005

Postby traderstuff » 30 Sep 2007

Marina,

Since these were posted on the public users forum,why can't you post the answers here?

There are probably others besides me who would apprerciate reading the official response.

Thanks


Hello,

Since you contacted us directly I we won't be posting the answer to your suggestion here on forum.

Thank you.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 08 Oct 2007

Marina,

Since these were posted on the public users forum,why can't you post the answers here?

There are probably others besides me who would apprerciate reading the official response.

Thanks


Hello,

Since you contacted us directly I we won't be posting the answer to your suggestion here on forum.

Thank you.


Hello,

I apologize for the delay in getting back to you.

In a nutshell, the general idea is as follows:

the portfolio concept suggested by Ryan, includes not only primary instruments but also those that are purely informational. Previously, we were going to consider all the instruments in a portfolio as traded/tradeable. However, we decided to accommodate Ryan's concept and created an engine which can recognise those purely informational instruments and treat them accordingly.

If you need further details please feel free to ask.

traderstuff
Posts: 68
Joined: 24 Jul 2005

Postby traderstuff » 09 Oct 2007

Marina,
Hello,

I apologize for the delay in getting back to you.

In a nutshell, the general idea is as follows:

the portfolio concept suggested by Ryan, includes not only primary instruments but also those that are purely informational. Previously, we were going to consider all the instruments in a portfolio as traded/tradeable. However, we decided to accommodate Ryan's concept and created an engine which can recognise those purely informational instruments and treat them accordingly.

If you need further details please feel free to ask.

****

Thank you for the update Marina, it is appreciated.

Fabrice Daniel
Posts: 71
Joined: 25 Aug 2007
Been thanked: 2 times

Postby Fabrice Daniel » 10 Oct 2007

Hi,

Here are the most important feature for a Portfolio Backtest Engine :

- Display Stats and Equity Curve when running N trading systems on N instruments. So not only backtest an "instrument portfolio" but a "Trading System Portfolio" too. All the other tools provides the first feature (AB, WealthLab, ...) but rare are the tools providing the second one.

So for example I will run a global backtest on "MyTrendFollowingSystem", "MyRangeSystem" and "MyReversalSystem" on a portfolio including all the Forex Symbols + All the Nasdaq 100 and Dow 30 components + 10 commodities Futures. So I want to get a global Equity Curve and Global Statistics.


- After having run a backtest on "X systems / Y Instruments" I would like to get not only the global Results (like the current "mono" system/instrument backtest engine do) but a Breakdown / multidimensional result :

* Performance / System
* Performance / Instrument Category (FX, Futures, Equities)
* Performance / Instrument
* Performance / Day Of Week
* Performance / Month
* Performance / Year

Please take a look at Lars Kestner Book "Quantitative Trading Strategies" to see how a results Breakdown could be :

www.amazon.com/Quantitative-Trading-Str ... 327&sr=1-3

- Multi-level results presentation: Very detailed statistics are not very efficient to use when you run a lot of test, look at a lot of results, it could be tedious. It's a mandatory feature when you have to zoom on a good trading system but when you look at results on a portfolio, a synthetic view is better at first (with the capability to zoom on the detailed standard backtest report)

* So into the Porfolio Report the Breakdown View could be a simple set of tables showing only the most important performance statistics per Market, System, Instrument, ... (Equity, %Win/Loss, Expectancy, Max Drawdown, Sharpe Ratio, ...)

- Adding Filters or Position Sizing tuning after calculation : This will avoid to modify a script and run a new calculation

* Capability to exclude a list of days (all the mondays, option expiry days, annoucement days like non farm payroll, etc ...)

* Excluding the monday from backtest could be setting up directly into MC but a list of specific days could be simply define importing a file where 1 line = 1 date



I hope this could help you
www.addictfx.biz

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 10 Oct 2007

Hi Fabrice,

Thanks a lot for your suggestions. I've passed them on to our software analysts. Once they have analyzed the possibility of implementing the features you are referring to I will get back to you with the results.

Thanks again.


Return to “MultiCharts”