Custom Criteria – creating your own optimization criteria

Studies that have been contributed to the community by other users. If you’ve got something useful to share, that’s great!
User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Custom Criteria – creating your own optimization criteria

Postby JoshM » 29 Jun 2011

I’d like to start a discussion about using Custom Criteria, partly because there’s not a lot of information available about these and also because I think these are a great feature which deserve some more attention.

Please note that this is still a ‘work-in-progress’ and I’m no expert on this – just trying to learn more about Custom Criteria. So if you have any ideas or thoughts about Custom Criteria, I encourage you to add these to this thread.

If you have any ideas for custom optimization criteria, also add them to this thread, and perhaps we can code them.

General information
MultiCharts allows you to optimize on 18 different optimization criteria, such as percentage profitable, profit factor, maximum intraday drawdown and win/loss ratio. These criteria are already quite suited for optimization, but if want to use something more exotic, you could write your own Custom Criteria.

Custom Criteria can be found through the following:
Right click the chart -> Format Signals -> Optimize -> choose Genetic and press Ok -> Go to the tab Algorithm-specific Properties, and choose Custom Criteria and click ‘Edit…’.

Image
customCriteriaImage1.PNG
customCriteriaImage1.PNG (81.49 KiB) Viewed 4522 times


Custom Criteria aren’t written in EasyLanguage but use Microsoft Jscript. See MSDN for more information about this, and here you can find a very accessible tutorial. More information about the Math class and it's method can be found here.

Some handy things to know about Jscript and Custom Criteria:

    Jscript is case sensitive; the variable ‘TotalTrades’ is something different than ‘totalTrades’;

    If you made an error, you’ll either receive an error message with the line number at which the error occurred or the Custom Criteria shows “-“ in the Optimization Report;

    Code can be commented out with double slashes, for example:
    // this line is commented out

    To comment out large parts of code, you can use /* */, for example:
    /* this
    whole
    section
    is commented out */
    Note: The /* ... */ can not be nested. For example, this will give an syntax error:
    /* commented
    out
    /* another comment /*
    section */

List of available reserved words
The following is a list of available reserved words which can be used in Custom Criteria (besides creating your own Jscript variables):

Code: Select all

The reserved words supported in Multicharts optimization:
StrategyPerformance.NetProfit
StrategyPerformance.GrossProfit
StrategyPerformance.GrossLoss
StrategyPerformance.TotalTrades
StrategyPerformance.PercentProfitable
StrategyPerformance.WinningTrades
StrategyPerformance.LosingTrades
StrategyPerformance.AvgTrade
StrategyPerformance.AvgWinningTrade
StrategyPerformance.AvgLosingTrade
StrategyPerformance.WinLossRatio
StrategyPerformance.MaxConsecWinners
StrategyPerformance.MaxConsecLosers
StrategyPerformance.AvgBarsInWinningTrades
StrategyPerformance.AvgBarsInLosingTrades
StrategyPerformance.MaxStrategyDrawDown
StrategyPerformance.ProfitFactor
StrategyPerformance.ReturnOnAccount

The reserved words supported in Portfolio optimization:
* NetProfit
* GrossProfit
* GrossLoss
* TotalTrades
* PercentProfitable
* WinningTrades
* LosingTrades
* MaxStrategyDrawDown

Source: the default Custom Criteria.

Accessing trade properties if pyramiding is turned on:

Code: Select all

Numeric OpenEntriesCount- number of open entries
Numeric OpenEntryDate(Numeric entry_index=0) - date of entry order execution. entry_index - entry number.
Numeric OpenEntryTime(Numeric entry_index=0) - time of entry order execution. entry_index - entry number.
Numeric OpenEntryPrice(Numeric entry_index=0) - price of entry order execution. entry_index - entry number.
Numeric OpenEntryContracts(Numeric entry_index=0) - number of entry order contracts. entry_index - entry number.
Numeric OpenEntryProfit(Numeric entry_index=0) - current entry profit. entry_index - entry number.
Numeric OpenEntryMaxProfit(Numeric entry_index=0) - maximum potential entry profit which was achieved. entry_index - entry number.
Numeric OpenEntryMinProfit(Numeric entry_index=0) - minimum potential entry profit which was achieved. entry_index - entry number.
Numeric OpenEntryProfitPerContract(Numeric entry_index=0) - current entry profit per contract. entry_index - entry number.
Numeric OpenEntryMaxProfitPerContract(Numeric entry_index=0) - maximum potential entry profit per contract which was achieved. entry_index - entry number.
Numeric OpenEntryMinProfitPerContract(Numeric entry_index=0) - minimum potential entry profit per contract which was achieved. entry_index - entry number.
Numeric OpenEntryComission(Numeric entry_index=0) - value of entry comission. entry_index - entry number.

Source: viewtopic.php?f=1&t=7532&p=41592#p34407


Examples
Taken from the default Custom Criteria:

The following function maximizes the ratio of Net Profit to Max Drawdown. The goal of this function is to find the trading rules that have a high net profit, but a small total drawdown over the back-tested period.

Code: Select all

if (StrategyPerformance.MaxStrategyDrawDown != 0) {
   return StrategyPerformance.NetProfit / (-StrategyPerformance.MaxStrategyDrawDown);
}


The following function maximizes the number of Winning trades minus the number of Losing Trades.

Code: Select all

return StrategyPerformance.WinningTrades - StrategyPerformance.LosingTrades;


The following function maximizes the ratio of gross profit to gross loss. The smaller the gross loss is and the higher the gross profit is the higher this ratio will be.

Code: Select all

if (StrategyPerformance.GrossLoss != 0) {
   return StrategyPerformance.GrossProfit / StrategyPerformance.GrossLoss;
}


The following function maximizes the percent of profitable trades.

Code: Select all

if (StrategyPerformance.TotalTrades != 0) {
   return StrategyPerformance.WinningTrades / StrategyPerformance.TotalTrades;
}


If you find this thread helpful, in any way, please show your support by hitting the "Thanks" button. Thanks! :)
Last edited by JoshM on 01 Jul 2011, edited 4 times in total.

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Custom Criteria: CPC Index by Sunny Harris

The CPC Index is an idea from Sunny Harris, author of the book TS Made Easy! Using EasyLanguage to Build Profits with the World's Most Popular Trading Software published by Wiley Trading.
Sunny uses this single number as the index by which to measure all trading systems. "A system with a CPC Index < 1.2 is not robust enough for me to trade." A system with a CPC Index >= 1.2 is robust enough to trade. This is not an indicator; it is a strategy that must be added to your other strategies to determine profitability.

Source: http://www.moneymentor.com/

The formula for the CPC Index is:

Code: Select all

(PercentProfitable) * (AverageWin/AverageLoss) * (ProfitFactor)

Source: viewtopic.php?f=1&t=8822&p=41591#p41552

The following Custom Criteria makes it possible to optimize on the CPC Index (see new version further in this post):

Code: Select all

// MultiCharts Custom Criteria: Sunny Harris' CPC Index

if (StrategyPerformance.AvgWinningTrade == 0 || StrategyPerformance.AvgLosingTrade == 0) {
    return 0;           // Prevents division by zero
}

var percentProfitable = StrategyPerformance.PercentProfitable / 100;

return (percentProfitable *
    (StrategyPerformance.AvgWinningTrade / StrategyPerformance.AvgLosingTrade) *
    StrategyPerformance.ProfitFactor );


A big thanks for Khalaad for pointing out the CPC Index, providing the formula and resources. :)

-------

*** Edit: 30-6-2011 ***

Further working with the CPC Index led me to believe the earlier posted version is somewhat counter-intuitive. The MultiCharts calculation of the Profit Factor for strategies with a Net Loss is different than the calculation when the strategy is profitable, which is:

Code: Select all

Profit Factor = Gross Profit / abs(Gross Loss)


Source: http://www.tradingmarkets.com/.site/sto ... -42291.cfm, see also Pardo’s book and George Pruitt’s Building Winning Trading Systems with TS book which both use absolute values for the Profit Factor calculation. Also see this blog post, which calculates the CPC Index using only absolute values.

The same goes for the Ratio Avg Win / Avg Loss – if the strategy is a net loser, MC calculates ‘avg win / avg loss’, but with a strategy is a net winner, this ratio is calculated as ‘avg win / abs(avg loss)’. This behaviour occurs in the Performance Report, but not in the Optimization Report, which makes the interpretation somewhat counter-intuitive.

So, I’ve changed the code below to make the CPC Index easier to interpret by using absolute values.

Code: Select all

// MultiCharts Custom Criteria: Sunny Harris' CPC Index
if (StrategyPerformance.AvgWinningTrade == 0 || StrategyPerformance.AvgLosingTrade == 0) {
    return 0;           // Prevents division by zero
}

var percentProfitable   = StrategyPerformance.PercentProfitable / 100;
var ratioWinLoss        = StrategyPerformance.AvgWinningTrade / Math.abs(StrategyPerformance.AvgLosingTrade);
var profitFactor        = StrategyPerformance.GrossProfit / Math.abs(StrategyPerformance.GrossLoss);

return (percentProfitable * ratioWinLoss * profitFactor);

See screenshot below.
Image
OptimizeOnCPCIndex.PNG
OptimizeOnCPCIndex.PNG (75.38 KiB) Viewed 4466 times


Interested readers might want to check out this presentation (pdf file) from Sunny Harris using MultiCharts (!). See page 44 and 86 and beyond for more about the CPC Index.
Last edited by JoshM on 30 Jun 2011, edited 2 times in total.

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Custom Criteria: Pessimistic Return on Margin (PROM)

Pessimistic Return on Margin (PROM) is a performance metric developed by Robert Pardo in his book The Evaluation and Optimization of Trading Strategies, 2nd edition, published by Wiley.

PROM is the yield on margin that is adjusted in a way that pessimistically assumes that a trading strategy will win less and lose more in real-time trading than it did in back testing.

The function also gives a penalty to strategies with few trades by using the square root of the number of winning trades and losing trades. This makes it a robust performance metric.

PROM is calculated as follows:

Code: Select all

PROM = {[Average Win x (Number of Wins - √(Number of wins))] – [ ABS(Average Loss) x (Number of Losses + √(Number of Losses))] / Margin


Pardo uses annualized profits and losses for the calculation of the PROM, but since dates of the trades in a strategy aren’t available as reserved word in Custom Criteria (correct me if I’m wrong!), I’ve opted to calculate the PROM without annualizing the results. Besides that I've also replaced margin with account size, since using the minimum margin won't allow for drawdowns and artificially inflates the PROM.

The following Custom Criteria makes it possible to optimize on PROM:

Code: Select all

// MultiCharts Custom Criteria: Pessimistic Return on Margin (PROM) by Robert Pardo

// Change the value below to your account size
var AccountSize = 10000;

/* The code below can be left untouched  */

// check for errors / wrong values
if (AccountSize < 1 || StrategyPerformance.WinningTrades == 0 ||
    StrategyPerformance.LosingTrades == 0) {
    return 0;
}

var sqrtWins = Math.sqrt(StrategyPerformance.WinningTrades);
var sqrtLosses = Math.sqrt(StrategyPerformance.LosingTrades);

return ( ((StrategyPerformance.AvgWinningTrade * (StrategyPerformance.WinningTrades - sqrtWins) ) -
    ( -StrategyPerformance.AvgLosingTrade * (StrategyPerformance.LosingTrades + sqrtLosses) )) /
    AccountSize) * 100;

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Writing calculations in a Custom Criteria to a file

Since the Custom Criteria window in MultiCharts doesn’t have a Output Window like the PowerLanguage Editor has, you can choose to write the results of calculations to a text file, and doing some ‘debugging’ of your code. See the example below.

Code: Select all

// MultiCharts Custom Criteria: Printing results to a file

// Enter the file location to write to below
var myFileLocation = "c:\\Temp\\JScriptTests\\CustomCriteria_test.txt";

// Function to write data to a file
function WriteToFile(dataToWrite) {
    var fso, tf;

    fso = new ActiveXObject("Scripting.FileSystemObject");    // create new write object

    tf = fso.OpenTextFile(myFileLocation, 8, true);         // open text file, if it doesn't exits: make it, else append to it

    // Write a new line with the data
    tf.WriteLine(dataToWrite.toString());

    tf.Close();                                           // Close file
}

/* Usage example: printing the NetProfit of each strategy to the file */

// Create a string which can be written to the text file
var stratOutput = "Net profit: " + StrategyPerformance.NetProfit.toString();        // note the .toString() : this convert the numeric value to the string

// Call our 'WriteToFile' method to export your data
WriteToFile(stratOutput);

This gives the following output in the C:\Temp\JScriptTests\CustomCriteria_test.txt file:
Net profit: -515
Net profit: -1090
Net profit: -685
Net profit: -895
Net profit: -1345
Net profit: -975
Net profit: -2460
Net profit: -2695
Net profit: -1250
Net profit: 130
Net profit: -1255
Net profit: -635
Net profit: -2555
Net profit: 70
Net profit: -2025
Net profit: -2380
Net profit: 410
Net profit: -1060
Net profit: 45
Net profit: -1645
Net profit: -1835
Net profit: -3055
Net profit: -1345
Net profit: 25
......

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Custom Criteria: Required Capital

This version of Required Capital, provided by Pardo in his book (see post above for reference), is calculated as followed:

Code: Select all

Required Capital = (Maximum drawdown * Safety Factor) / Capital Stop Loss


Where…
Safety Factor = ‘safety buffer’ which enables continuing trading when the strategy reaches its “maximum” drawdown. Since the ‘maximum’ drawdown in real-time trading can be greater than the back tested maximum drawdown, Pardo suggest using a Safety Factor of 3. Aggressive traders can opt for a lower Safety Factor, while conservative traders might choose a higher factor.

Capital Stop Loss = the percentage loss, for example 40%, that the trader is prepared to withstand during real-time trading of his strategy. If this stop loss is achieved in real-time trading (i.e. the loss on capital allocated to the strategy equals or is greater than 40%), the strategy would be terminated.

The following Custom Criteria enables optimizing on this definition of Required Capital:

Code: Select all

// MultiCharts Custom Criteria: Required Capital by Robert Pardo

// Change the values below to suit your preferences
var SafetyFactor = 3.00;
var CapitalStopLoss = 0.40;        // 0.40 = 40%

/* The code below can be left untouched */

// Check for wrong values
if(CapitalStopLoss < 0.01 || SafetyFactor < 0.01 ||
    StrategyPerformance.MaxStrategyDrawDown == 0)
{
    return 0;
}

return (-StrategyPerformance.MaxStrategyDrawDown * SafetyFactor) / CapitalStopLoss;

You probably don’t want to use this Custom Criteria as the sole filter to determine the indicator settings, but it can give you an idea what kind of capital you will need for your strategy given your risk preferences.

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Custom Criteria: Pessimistic Return On Required Capital (PRORC)

Just got a nice idea.. why don’t we raise the bar even further, and show some extra though love to our strategies?

Let’s use the pessimistic assumption of the PROM, which artificially raises the number of losing trades while at the same time lowers the number of winning trades, and combine these with the Required Capital, which gives the amount of capital to “safely” trade this strategy given our risk preferences. Using the PROM function to determine the results also gives an penalty to strategies that have few trades, which lowers the chances that these turn up as best results.

The benefit of this approach would be that it reduces the subjectivity of the strategist – if we calculate the pessimistic return on our own defined account size, we run the risk of wanting to trade a strategy with risk capital that might too little given the performance of the strategy.

So, let’s use the performance of the strategy itself to determine the minimum account size needed to trade it, taking into account the risk preferences of the trader and give the strategy also opportunity to recover from a draw down, and calculate the pessimistic return based on this account size.

I don’t know what a good name for this function would be, but just call it ‘PRORC’ (Pessimistic Return On Required Capital’) for now. The formula for this ‘PRORC’ would be:

Code: Select all

{[Average Win x (Number of Wins - √(Number of wins))] – [ ABS(Average Loss) x (Number of Losses + √(Number of Losses))]
/
(Maximum drawdown * Safety Factor) / Capital Stop Loss

(see posts above for definitions of these words).

Example:
Avg winning trade: 23.22
Avg losing trade: 14.93
Number of wins: 367
Number of losses: 569
Maximum strategy drawdown: $1400
Safety Factor: 3
Capital Stop Loss: 50%

The Strategy Performance report of this strategy shows a net profit of 25, which is a Return on Initial Capital of 0.25% (10.000 account). One might argue that this strategy, without optimization, can hold some (emphasis on ‘some’) merit. Let’s look at the PRORC for this strategy: -9.21%; quite more pessimistic than the Return on Initial Capital of +0.25%, and more pessimistic than the PROM of -7.74%. Let’s code it!

The following Custom Criteria allows optimization on the PRORC:

Code: Select all

// MultiCharts Custom Optimization Criteria: PRORC [Pessimistic Return On Required Capital]

// Change the values below to your risk perferences
var SafetyFactor = 3.00;            // Multification factor of the MDD to give a buffer
var CapitalStopLoss = 0.40;         // Percentage loss after which the strategy is stopped. 0.40 = 40%

/* The code below can be left untouched */

// Check for wrong/missing values
if(CapitalStopLoss < 0.01 || SafetyFactor < 0.01 ||
    StrategyPerformance.MaxStrategyDrawDown == 0 ||
    StrategyPerformance.WinningTrades == 0 ||
    StrategyPerformance.LosingTrades == 0)
{
    return 0;
}

// Calculate the pessimistic result
var sqrtWins = Math.sqrt(StrategyPerformance.WinningTrades);
var sqrtLosses = Math.sqrt(StrategyPerformance.LosingTrades);
var pessReturn = (StrategyPerformance.AvgWinningTrade * (StrategyPerformance.WinningTrades - sqrtWins) ) -
    ( -StrategyPerformance.AvgLosingTrade * (StrategyPerformance.LosingTrades + sqrtLosses) );

// Calculate the Required Capital for this strategy
var reqCapital = (-StrategyPerformance.MaxStrategyDrawDown * SafetyFactor) / CapitalStopLoss;

return (pessReturn / reqCapital) * 100;

khalaad
Posts: 314
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 63 times
Been thanked: 57 times

Re: Custom Criteria – creating your own optimization criteri

Postby khalaad » 29 Jun 2011

Josh,

Simply brilliant!

Khalid

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 29 Jun 2011

Thanks Khalid! :)

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 30 Jun 2011

Note: Readers who've already used the CPC Index Custom Criteria might want to check out the new version in that post. See post viewtopic.php?f=5&t=8838#p41609

Custom Criteria: Profitability Rule

The Profitability Rule is a concept developed by Michael Harris that calculates the profitability that is needed to achieve a given profit factor. The Profitability is determined as follows:

Code: Select all

 w = Pf /(Pf + Rwl )


Where..
W = winning percentage (i.e. profitability)
Pf = Profit factor (gross profit / gross loss)
Rwl = Ratio Average winning trade / Average losing trade

It is an equation that determines the minimum value of the win rate w that is required to achieve a profit factor PF in the presence of an average win to average loss ratio R. For example, for R = 2 and PF = 2, the minimum win rate w is 2/(2+2) = 0.5 or 50%. Thus, if a trading system wins double the amount that it loses on the average, for a profit factor equal to 2 the win rate must be at least 50%.

Source: http://www.priceactionlab.com/Blog/2011 ... ve-answer/ (this page is also worth a read: http://www.priceactionlab.com/Blog/2011 ... off-ratio/)

Optimizing using the Profitability Rule Custom Criteria has some advantages. First, it tells you what percentage profitability is needed to achieve your goals – and how much your current strategy achieves or falls shorts of this goal. A second advantage is that the Profitability Rule selects the optimized results with a high ratio average winning trade / average losing trade and a low winning percentage, since the higher the Rwl ratio, the lower the required probability is, and the easier the needed Profitability is achieved. One might argue that this leads to selecting more robust strategies since a strategy with a high winning percentage tends to suffer more from a rough patch than a strategy with a low winning percentage (in other words, a strategy with a win percentage of 80% needs to experience continuing favourable conditions to keep this good performance up).

In Harris’ book Profitability and Systematic Trading (published by Wiley) he expands the Profitability Rule to include α:

Code: Select all

W = Pf / (Pf + (α * Rwl))


Where α is “a factor that accounts for commission, slippage, and other random effects that impact trade execution and bottom line-performance” (p. 55). He suggest using a value of 0.5 – 0.7 for intraday trading strategies, and a value in the range of 0.7 – 0.9 for other short time frames.

Since MultiCharts already incorporates slippage and commission perhaps the trader can use a higher α value (i.e. a smaller commission and slippage penalty) than the ones advised by Harris. In the Custom Criteria code below I’ve included nonetheless this α penalty so that the trader be though against his strategy if he wishes so. Setting this value to one (1) turns it off.

Code: Select all

// MultiCharts Custom Optimization Criteria: Profitability Rule

// Change these values to your preferences
var TargetedProfitFactor = 2.00;
var CommissionPenalty = 0.60;       // Enter a one (1) here to disable commission penalties

// * The code below can be left untouched  *
var WinLossRatio = StrategyPerformance.AvgWinningTrade / Math.abs(StrategyPerformance.AvgLosingTrade);
var RequiredProfitability = TargetedProfitFactor / (TargetedProfitFactor + (CommissionPenalty * WinLossRatio));

return StrategyPerformance.PercentProfitable - (RequiredProfitability * 100);


Reading the output values
The resulting values in the Optimization Report show the amount of percentage points that the profitability of the strategy differences from the needed profitability to achieve the given Profit Factor.

For example, if the trader targets a Profit Factor of 2.00, and the optimized strategy returns a ratio average winning trade / average losing trade of 0.822, the needed percentage winning trades would be: 2.00 / (2.00 + 0.822) = 70.86%. If the backtested strategy ‘only’ has a percentage winning trades of 50%, the Custom Criteria column returns -20.86, indicating that the strategy falls 20 percentage points short of the needed profitability to achieve a Profit Factor of 2. Positive values indicate that the profitability of the strategy is higher than the one required to reach the wished Profit Factor.

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 01 Jul 2011

In the previous posts some Custom Criteria were created to optimize on. But what if we want to optimize on the CPC and PROM? Or the Profitability Rule combined with our own Custom Criteria?

Wrapping it up – Combining Custom Criteria and exporting for further analysis

The code for exporting the Custom Criteria (see below) has the following features:

  • Allows usage of all the Custom Criteria and performance metrics you can think of. Note that you still need to specify one Custom Criteria that MultiCharts should use to run the Genetic Optimization on. Currently it’s not (yet) possible to optimize on multiple Custom Criteria on the same time, but you can optimize on one and export the values of the others.
  • The code exports the data in txt file format, for further analysis in other software like Excel. The separator between values is by default “ ; “ but this can be changed in the code. The number of decimals you’d like to export can also be changed.
  • You can use this code as an example if you’d like to export your own Performance Metrics. You can practically export everything you can calculated with the available reserved words (listed in the first post of this thread).

Known limitations
The current version of the code has the following limitations:

  • No export of parameter settings. Since these are not available as reserved word (to my knowledge), you’ll need to copy/paste the columns with your parameter settings if you’d also like to incorporate these in your further analysis.
  • The exported file needs to be deleted, moved, or renamed after the optimization. That way the results of different optimization runs won’t get mingled up in the file.

Usage
  1. Copy the code below into the ‘Custom Criteria Edit’ in MultiCharts (Optimize -> Genetic -> Algorithm-specific Properties).
  2. Choose your objective function you’d like to optimize on, and change the variable ‘myObjectiveFunction’ to reflect this choice.
  3. Enter the file location to which the file should be written.
  4. Change the settings in the code to your preferences/situation. Most, if not all, of these variables only need to be changed once to your situation (such as AccountSize).
  5. Optimize your strategy.
  6. Navigate to your text file and open these in your software (for example, Excel).
    1. For importing into Excel, use Data -> Transform Text into Columns in Excel.
    2. Note: Depending on your locale/region settings, you might need to change the settings for thousands to “,” and decimals to “.”. See this tutorial if you run into problems.
  7. Use Excel’s Sort function to organize and analyze your data (Tip: Colour the background of the best results, for example the best 10 percent, and look for clusters when sorting on various ways). With the Sort function you can for example sort on the CPC Index, then on PercentageProfitability and then on NetProfit.
  8. If needed, copy the columns with your indicator settings into your Excel file.
  9. If done with your analysis: save your file under a different name and delete the original output file. This is needed to prevent that the results of a new optimization gets mingled up with those of an earlier one. Alternatively you may change the file location to which the file gets written with each optimization, though that’s a more cumbersome process.

Screens
The output in the txt file:
Image

Imported in Excel with some basic graphs created:
Image

Code
Note: the attached PDF file shows the same code but with coloured words and proper formatting to make it more easily readable than the forums code formatting.

Code: Select all

// MultiCharts - Raising the Trading Standard                                           http://www.multicharts.com
//  Custom Optimization Criteria: Wrapping it up; Combining Custom Criteria and exporting for further analysis.
//  Source: http://www.multicharts.com/discussion/viewtopic.php?f=5&t=8838

/* Choose the objective function, the performance metric on which you want to perform the Genetic Optimization, below.
    Give the variable 'myObjectiveFunction' a number corresponding with your choice:
    1: Sunny Harris' CPC Index;
    2: Robert Pardo's Pessimistic Return on Margin (PROM);
    3: Robert Pardo's Required Capital;
    4: Pessimistic Return On Required Capital (PRORC);
    5: Michael Harris' Probability Rule.

    For example, if you'd like to optimize on the CPC Index, give variable 'myObjectiveFunction' a value of 1.
*/
var myObjectiveFunction = 1;

/* Settings start here; change these to your preferences */
    /*  Change the variable 'myFileLocation' to the location to where you want to write your file.
        Note: Before running a new optimization, you'll need to remove, rename or delete this file.
        Else it will get overwritten and/or will the header of the text file be unsuitable for importing in Excel.
    */
var myFileLocation = "c:\\Temp\\CustomCriteria_Output.txt";
var AccountSize = 100000;
var SafetyFactor = 3.00;            // The SafetyFactor times the MaxDrawDown gives a buffer for the Required Capital.
var CapitalStopLoss = 0.40;         // The percentage loss after which you'll stop trading the strategy (0.40 = 40%).
var TargetedProfitFactor = 2.00;    // The Profit Factor the trader want to achieve.
var CommissionPenalty = 0.60;       // Penalty used to calculate if the Profit Factor is achieved; enter a one (1) here to disable this.

/*
                This is the end of the settings
                        The code below can be left untouched
*/

// Strategy performance variables
var avgWinTrade         = StrategyPerformance.AvgWinningTrade;
var avgLoseTrade        = StrategyPerformance.AvgLosingTrade;
var percentProfitable   = StrategyPerformance.PercentProfitable;
var profitFactor        = StrategyPerformance.GrossProfit / Math.abs(StrategyPerformance.GrossLoss);
var winLossRatio        = StrategyPerformance.AvgWinningTrade / Math.abs(StrategyPerformance.AvgLosingTrade);
var numWinningTrades    = StrategyPerformance.WinningTrades;
var numLosingTrades     = StrategyPerformance.LosingTrades;
var maxDrawdown         = StrategyPerformance.MaxStrategyDrawDown;
var grossProfit         = StrategyPerformance.GrossProfit;
var grossLoss           = StrategyPerformance.GrossLoss;
var netProfit           = StrategyPerformance.NetProfit;
var numOfTrades         = StrategyPerformance.TotalTrades;
var avgTrade            = StrategyPerformance.AvgTrade;
var maxConsWins         = StrategyPerformance.MaxConsecWinners;
var maxConsLoss         = StrategyPerformance.MaxConsecLosers;
var avgBarWinner        = StrategyPerformance.AvgBarsInWinningTrades;
var avgBarLoser         = StrategyPerformance.AvgBarsInLosingTrades;
var returnOnAcc         = StrategyPerformance.ReturnOnAccount;
var returnOnAccSize     = (netProfit / AccountSize) * 100;              // Percentage profit on the AccountSize provided by the user
var maxDDPerc           = (maxDrawdown / AccountSize) * 100;            // The MaxDrawDown in percentage

// Objective functions variables
var cpcIndex, prom, reqCapital, prorc, reqProfitability, headerString, outputString, sep;

var sqrtWins            = Math.sqrt(numWinningTrades);
var sqrtLosses          = Math.sqrt(numLosingTrades);
var pessResult          = (avgWinTrade * (numWinningTrades - sqrtWins)) - (Math.abs(avgLoseTrade) * (numLosingTrades + sqrtLosses));

sep = ";";          // The seperator between values exported in the text file; change this if you have trouble importing the data with your Locale/Region settings
var decimals = 4;       // Number of decimals to export
var curDecimals = 2;    // Number of decimals for currency values

/* The calculation of the Custom Criteria starts below */

// Sunny Harris' CPC Index
if (avgWinTrade == 0 || avgLoseTrade == 0) {
    cpcIndex = 0;
}
else {
    cpcIndex = (percentProfitable / 100) * winLossRatio * profitFactor;
}


// Robert Pardo's Pessimistic Return on Margin (PROM)
if (AccountSize < 1 || numWinningTrades == 0 || numLosingTrades == 0) {
    prom = 0;
}
else {
    prom = (pessResult / AccountSize) * 100;
}


// Robert Pardo's Required Capital
if (CapitalStopLoss < 0.01 || SafetyFactor < 0.01 || maxDrawdown == 0) {
    reqCapital = 0;
}
else {
    reqCapital = (Math.abs(maxDrawdown) * SafetyFactor) / CapitalStopLoss;
}


// Pessimistic Return On Required Capital (PRORC); inspired on Pardo's concepts
if (prom == 0 || reqCapital == 0) {
    prorc = 0;
}
else {
    prorc = (pessResult / reqCapital) * 100;
}

// Michael Harris' Probability Rule
if (TargetedProfitFactor < 0.01 || CommissionPenalty < 0 || CommissionPenalty > 1) {
    reqProfitability = 0;
}
else {
    reqProfitability = percentProfitable - (TargetedProfitFactor / (TargetedProfitFactor + (CommissionPenalty * winLossRatio)) * 100);
}

// Function to print the data to the file
function WriteToFile(stringToWrite) {
    // Create a new FileSystemObject
    fsObj = new ActiveXObject("Scripting.FileSystemObject");

    // If the file doesn't exits, create it, write the header {and the first line}
    if (!fsObj.FileExists(myFileLocation)) {
       
        // Create the header string
        headerString =
            "CPC Index"             + sep +
            "PROM"                  + sep +
            "RequiredCapital"       + sep +
            "PRORC"                 + sep +
            "ProbabilityRule"       + sep +
            "GrossProfit"           + sep +
            "GrossLoss"             + sep +
            "NetProfit"             + sep +
            "ReturnOnAccount"       + sep +
            "ReturnOnAccountSize"   + sep +
            "PessimisticProfit"     + sep +
            "TotalTrades"           + sep +
            "PercentProfitable"     + sep +
            "WinningTrades"         + sep +
            "LosingTrades"          + sep +
            "AvgTrade"              + sep +
            "AvgWinningTrade"       + sep +
            "AvgLosingTrade"        + sep +
            "WinLossRatio"          + sep +
            "ProfitFactor"          + sep +
            "MaxStrategyDrawDown"   + sep +
            "MaxDrawDownPercentage" + sep +
            "MaxConsecWinners"      + sep +
            "MaxConsecLosers"       + sep +
            "AvgBarsInWinningTrades"+ sep +
            "AvgBarsInLosingTrades"
            ;

        // Create the file
        fsObj.CreateTextFile(myFileLocation);

        // Open stream to write to it
        os = fsObj.GetFile(myFileLocation);
        os = os.OpenAsTextStream(2, 0);

        // Write data to it
        os.WriteLine(headerString);
        //os.WriteLine(stringToWrite);

        // Close the file
        os.Close();
    }
    else {
        os = fsObj.GetFile(myFileLocation);
        os = os.OpenAsTextStream(8, 0);

        os.WriteLine(stringToWrite);

        os.Close();
    }
}

// Create the string with strategy data
outputString =
    cpcIndex.toFixed(decimals)              + sep +
    prom.toFixed(decimals)                  + sep +
    reqCapital.toFixed(curDecimals)         + sep +
    prorc.toFixed(decimals)                 + sep +
    reqProfitability.toFixed(decimals)      + sep +
    grossProfit.toFixed(curDecimals)        + sep +
    grossLoss.toFixed(curDecimals)          + sep +
    netProfit.toFixed(curDecimals)          + sep +
    returnOnAcc.toFixed(decimals)           + sep +
    returnOnAccSize.toFixed(decimals)       + sep +
    pessResult.toFixed(curDecimals)         + sep +
    numOfTrades.toFixed(0)                  + sep +
    percentProfitable.toFixed(decimals)     + sep +
    numWinningTrades.toFixed(0)             + sep +
    numLosingTrades.toFixed(0)              + sep +
    avgTrade.toFixed(decimals)              + sep +
    avgWinTrade.toFixed(decimals)           + sep +
    avgLoseTrade.toFixed(decimals)          + sep +
    winLossRatio.toFixed(decimals)          + sep +
    profitFactor.toFixed(decimals)          + sep +
    maxDrawdown.toFixed(curDecimals)        + sep +
    maxDDPerc.toFixed(decimals)             + sep +
    maxConsWins.toFixed(0)                  + sep +
    maxConsLoss.toFixed(0)                  + sep +
    avgBarWinner.toFixed(decimals)          + sep +
    avgBarLoser.toFixed(decimals)
    ;

WriteToFile(outputString);

// Return the choosen objective function
if (myObjectiveFunction == 1) {
    return cpcIndex;
}
else if (myObjectiveFunction == 2) {
    return prom;
}
else if (myObjectiveFunction == 3) {
    return reqCapital;
}
else if (myObjectiveFunction == 4) {
    return prorc;
}
else if (myObjectiveFunction == 5) {
    return reqProfitability;
}
else {      // makes sure that at least something gets returned
    return netProfit;
}
Attachments
ExportingOptimizationData.pdf
(18.66 KiB) Downloaded 442 times
FurtherAnalysisInExcel.PNG
FurtherAnalysisInExcel.PNG (58.29 KiB) Viewed 5443 times
OutputTextFile.PNG
OutputTextFile.PNG (98.91 KiB) Viewed 4460 times

khalaad
Posts: 314
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 63 times
Been thanked: 57 times

Re: Custom Criteria – creating your own optimization criteri

Postby khalaad » 19 Jul 2011

Josh,

You rekindle the urge to run optimisations!

Khalid

shadrock55
Posts: 47
Joined: 24 Feb 2011
Location: Atlanta, GA
Has thanked: 5 times
Been thanked: 8 times

Re: Custom Criteria – creating your own optimization criteri

Postby shadrock55 » 11 Aug 2011

Thanks Josh,

A couple of questions.

1. Instead of optimizing on one parameter, do you have a version that uses some results as filters or have you come up with a scored weighting system? Rather than exporting to Excel, there should be a way to get the best system without an export.

2. Here are a couple of links that you may find interesting about optimization from Dr. Mike Bryant over at Breakout Futures. You probably can't work it into the custom fitness box though. You should be able to the text files on his site if you are interested.

Equity Curve Correlation
http://www.breakoutfutures.com/Newslett ... er1104.htm

Optimize to Perfect Profit similar to Pardo (improvement on previous)
http://www.breakoutfutures.com/Newslett ... er0605.htm

Downloads:
http://www.breakoutfutures.com/PreDownload.htm

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 15 Aug 2011

Thanks for the interesting links Shadrock55. :)
shadrock55 wrote:1. Instead of optimizing on one parameter, do you have a version that uses some results as filters or have you come up with a scored weighting system? Rather than exporting to Excel, there should be a way to get the best system without an export.

I don't think filtering can be done (depending on how you want to filter), since MultiCharts calculate each strategy variant only once and there's no 'higher-level' code in Custom Criteria than can be used to save the results of the various optimisations and then, before the optimization concludes, filter these results.

Alternatively, you could use an own index. For example, if you want to optimize on Expectancy and MaxDrawDown, you could do something like 'Expectancy * MaxDrawDown'. Or if you want to give Expectancy of weight of 70%: 'myIndex = (Expectancy * 0.70) + (MaxDrawDown * 0.30)'.

It is possible to add some rough manual filters in the Custom Criteria code by, for example, giving any value that fails to meet your standard a value of -1 so they turn up along side the highest rated optimizations. I don't know though if that's the kind of 'filtering' you're looking for.

If you could use/want a code example, let me know what you want to achieve and I'll look what I can do.

Regards,
Josh

shadrock55
Posts: 47
Joined: 24 Feb 2011
Location: Atlanta, GA
Has thanked: 5 times
Been thanked: 8 times

Re: Custom Criteria – creating your own optimization criteri

Postby shadrock55 » 15 Aug 2011

The rough filtering is what I've been doing, and it seems like an application here could avoid the necessity for outputting. The main reason why I want to do this is for walk forward testing. To use the built in WF test, you need MC to select the top set of variables for you.

For example, I'm currently using the code below (which is not optimal so I'm working on one with the data you provided). I filter profit factors above 2 because its likely a unsustainable peak and I limit to less than 10 consecutive loser because the strategy is not consistent if you have that many.

Ideally, you could filter on PROM, Required Capital, PRORC and Probability Rule and then return CPC index, and MC would use the top CPC index to Walk Forward. The scored weighting system would also work, but I could see it taking a lot of testing to make sure everything is weighted properly.

I can write the code. I was just wondering if you'd already solved the WFA "problem", or if you were doing WF by hand from the outputs.

Code: Select all

// Criteria Filters will Return Zero For items I want to excluded from walk forward testing results.

if (StrategyPerformance.AvgWinningTrade == 0 ||
    StrategyPerformance.AvgLosingTrade == 0  ||
    StrategyPerformance.ProfitFactor > 2     ||
    StrategyPerformance.ProfitFactor < 1     ||
    StrategyPerformance.MaxConsecLosers <= 10)

{
    return 0;           // Prevents division by zero
}

var percentProfitable   = StrategyPerformance.PercentProfitable / 100;
var ratioWinLoss        = StrategyPerformance.AvgWinningTrade / Math.abs(StrategyPerformance.AvgLosingTrade);
var profitFactor        = StrategyPerformance.GrossProfit / Math.abs(StrategyPerformance.GrossLoss);

return (percentProfitable * ratioWinLoss * profitFactor);


oliveagle
Posts: 20
Joined: 07 Jul 2011
Has thanked: 4 times
Been thanked: 4 times

Re: Custom Criteria – creating your own optimization criteri

Postby oliveagle » 23 Aug 2011

very useful. thx a lot.

khalaad
Posts: 314
Joined: 07 Jan 2007
Location: Lahore, Pakistan
Has thanked: 63 times
Been thanked: 57 times

Re: Custom Criteria – creating your own optimization criteri

Postby khalaad » 09 Oct 2011

JoshM has created a Project Management entry here:

https://www.multicharts.com/pm/viewissue ... _no=MC-648

Thank you, Josh.

Please vote for it.

Thank you.

Khalid

SAL
Posts: 2
Joined: 17 May 2011
Has thanked: 1 time
Been thanked: 2 times

Re: Custom Criteria – creating your own optimization criteri

Postby SAL » 12 Mar 2012

Hi Josh,

Great work on this thread, really helpful. However, I seem to get an error when trying to apply any of the codes you have provided. The error I get is "Microsoft JScript execution error: Description: Object doesnt support this action. Line: 1" This is an example of the error. The PROMRC you posted looks really interesting and it gives me the same error but Line: 21 instead. I tried all of them and they all give me errors. Except the ones in the beginning of the thread which are the example ones already available in the Custom Criteria. Those work fine. But the CPC Index and the other interesting ones dont. Am I doing something wrong? Is there something wrong with my java?

Any suggestions will be highly appreciated.

SAL

User avatar
JoshM
Posts: 2080
Joined: 20 May 2011
Location: The Netherlands
Has thanked: 1525 times
Been thanked: 1483 times
Contact:

Re: Custom Criteria – creating your own optimization criteri

Postby JoshM » 14 Mar 2012

SAL wrote:I tried all of them and they all give me errors. Except the ones in the beginning of the thread which are the example ones already available in the Custom Criteria. Those work fine. But the CPC Index and the other interesting ones dont. Am I doing something wrong?

I've retried them and they work fine here (MultiCharts Version 7.4 Release (Build 4906)). Have you perhaps not uncommented a section of irrelevant code, or something went wrong with copy-pasting it into the Edit Custom Criteria box?

SAL
Posts: 2
Joined: 17 May 2011
Has thanked: 1 time
Been thanked: 2 times

Re: Custom Criteria – creating your own optimization criteri

Postby SAL » 14 Mar 2012

Hey Josh,

I just figured it out. Apparently I was using the Portfolio Backtester, which does not support some of these reserved words. Multicharts supports a few more reserved words than the portfolio backtester, a list can be found here:

http://www.multicharts.com/trading-software/index.php/Performing_Optimization#Custom_Criteria

SAL

Dru
Posts: 107
Joined: 28 Aug 2007
Has thanked: 4 times
Been thanked: 170 times

Re: Custom Criteria – creating your own optimization criteri

Postby Dru » 20 Mar 2012

For information: No limits for custom criteria optimizations. Sharpe Ratio? It's simplest!
If you can calculate your own optimization criteria value then you can optimize your stratagy on it.
See on:
1) https://www.multicharts.com/trading-software/index.php/SetCustomFitnessValue
2) the new signal in MC 8.0 beta 2 for optmize strategy by Sharp Ratio

skan
Posts: 36
Joined: 15 Jul 2010

Re: Custom Criteria – creating your own optimization criteri

Postby skan » 05 Apr 2012

Hello

How would you use setcustomfitnessvalue to optimize using the Sortino ratio?
(without exporting data to excel)

User avatar
Henry MultiСharts
Posts: 8448
Joined: 25 Aug 2011
Has thanked: 1207 times
Been thanked: 2705 times

Re: Custom Criteria – creating your own optimization criteri

Postby Henry MultiСharts » 13 Apr 2012

skan wrote:Hello

How would you use setcustomfitnessvalue to optimize using the Sortino ratio?
(without exporting data to excel)

Hello Skan,

Unfortunately that is not possible to optimize using the Sortino ratio with setcustomfitnessvalue at the moment.

quantarb
Posts: 51
Joined: 14 Apr 2012
Has thanked: 9 times
Been thanked: 32 times

Re: Custom Criteria – creating your own optimization criteri

Postby quantarb » 15 Nov 2012

Hello guys, I found this thread to be really helpful. I’m currently using Sunny Harris CPC index as a custom fitness value for optimizing my strategies in Multicharts. However, sometimes I get trading parameters with a good CPC index but with a very low number of trades. This concerns me because there are not enough trades to determine whether or not the system is robust.

My solution was to multiply the CPX index with a Logistic Function.

http://en.wikipedia.org/wiki/Logistic_function

The formula is L = 1/ [(1+ exp(p)]

For p, I used p = ( totaltrades – m ^0.8 )*-1, and m is your number of target number of trades you want the strategy to have in order for it be considered robust. In my example, m would be 30 trades. I got the 0.8 by playing with the function in excel.
What is nice about this function is the value is between 0 and 1. I think this is a better alternative using constraints.

You could create a constraint to filter out strategy parameters with less than 30 trades, but what if you get a really good strategy parameter but it only has 29 trades. Is 30 trades that much more robust than 29 trades? The logistic function will slightly penalize the trading parameter with 29 trades and dramatically penalize a trading parameter with 12 trades.

You can also use it to level off criteria after they have reached a certain threshold. Someone earlier in the thread mentioned they don’t like strategies with profit factor greater than 2.0. If you combine the logistic function with profit factor, your custom criteria will stop improving after your profit factor has reached your target threshold. Below is my code for the CPC index with the logistic function.
In this case, I set m to 30 trades.


Code: Select all

// MultiCharts Custom Criteria: Sunny Harris' CPC Index
if (StrategyPerformance.AvgWinningTrade == 0 || StrategyPerformance.AvgLosingTrade == 0) {
    return 0;           // Prevents division by zero
}

var netProfit = StrategyPerformance.NetProfit;
var percentProfitable   = StrategyPerformance.PercentProfitable / 100;
var ratioWinLoss        = StrategyPerformance.AvgWinningTrade / Math.abs(StrategyPerformance.AvgLosingTrade);
var profitFactor        = StrategyPerformance.GrossProfit / Math.abs(StrategyPerformance.GrossLoss);
var m = 30;
var p = ( StrategyPerformance.TotalTrades - m^0.8 )*-1;
var L = 1/(1+Math.exp(p));


return (netProfit * percentProfitable * ratioWinLoss * profitFactor*L);


/*

The reserved words supported in Multicharts optimization:
* NetProfit
* GrossProfit
* GrossLoss
* TotalTrades
* PercentProfitable
* WinningTrades
* LosingTrades
* AvgTrade
* AvgWinningTrade
* AvgLosingTrade
* WinLossRatio
* MaxConsecWinners
* MaxConsecLosers
* AvgBarsInWinningTrades
* AvgBarsInLosingTrades
* MaxStrategyDrawDown
* ProfitFactor
* ReturnOnAccount
* CustomFitnessValue

*/

/*

The reserved words supported in Portfolio optimization:
* NetProfit
* GrossProfit
* GrossLoss
* TotalTrades
* PercentProfitable
* WinningTrades
* LosingTrades
* MaxStrategyDrawDown

*/


Return to “User Contributed Studies and Indicator Library”