Hi everyone, I'm interested to hear how people test their trading programs for correctness (does it enter and exit when it is supposed to).
Background:
I test my own software by saving a short series of ticks and running the trigger code through unit tests. However, using MC .NET, I've found the main source of errors is from making wrong assumptions about the MC .NET API behavior such as SignalObject member values, etc.
So I want to do an integration test where I run the MC .NET Backtester on a quote file and make sure I get the entries and exits that I previously verified by eye. I would run the Backtester programatically and specify a quote file, then analyze some output file.
Question:
Is there a way to run the MC .NET Backtester programatically?
Also I would be interested to hear about how others are doing this kind of testing.
Thanks!
Testing algorithm correctness
- jwebster503
- Posts: 24
- Joined: 13 Mar 2014
- Has thanked: 9 times
- Been thanked: 14 times
Re: Testing algorithm correctness
Most of our code lives outside MultiCharts as a DLL -- this allows for unit testing specific scenarios.
We have the ability to save and replay streams of ticks, which allows us to do regression tests -- we generate logs of events, such as when a position changes, what the price was, and what time the change happened. We then compare these to known files which we've manually verified.
Also, as you mention, much of our struggle has been with the interaction with MultiCharts -- Realtime Ticks are delivered while historical data is processing, so we need to queue them until historical data catches up. It feels like MultiCharts will call StartCalc and StopCalc on a whim -- I think I understand in general when it will happen, but the fact that StartCalc gets called multiple times during a data "run" means that we have to handle much of the caching of resources ourselves instead of letting the calculation run do it, etc.
I'd also be very interested in running Portfolio Backtester programmatically and getting programmatic access to generated reports.
Good luck,
Jeff
We have the ability to save and replay streams of ticks, which allows us to do regression tests -- we generate logs of events, such as when a position changes, what the price was, and what time the change happened. We then compare these to known files which we've manually verified.
Also, as you mention, much of our struggle has been with the interaction with MultiCharts -- Realtime Ticks are delivered while historical data is processing, so we need to queue them until historical data catches up. It feels like MultiCharts will call StartCalc and StopCalc on a whim -- I think I understand in general when it will happen, but the fact that StartCalc gets called multiple times during a data "run" means that we have to handle much of the caching of resources ourselves instead of letting the calculation run do it, etc.
I'd also be very interested in running Portfolio Backtester programmatically and getting programmatic access to generated reports.
Good luck,
Jeff
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Testing algorithm correctness
Hello hoimlet,
Please specify the exact functionality you want to automate.
Please specify the exact functionality you want to automate.
Re: Testing algorithm correctness
Portfolio Trader:
Run a saved portfolio from the command line, and,
automatically save the generated reports to the file system.
This way, we can automate our strategy testing. (Meaning, test if it gives the correct trades on a given day; that the algorithm is working correctly.)
Run a saved portfolio from the command line, and,
automatically save the generated reports to the file system.
This way, we can automate our strategy testing. (Meaning, test if it gives the correct trades on a given day; that the algorithm is working correctly.)
- Henry MultiСharts
- Posts: 9165
- Joined: 25 Aug 2011
- Has thanked: 1264 times
- Been thanked: 2957 times
Re: Testing algorithm correctness
Hello hoimlet,
Thank you for your feedback. Unfortunately there is no such functionality at the moment.
You may want to submit a feature request to the Project Management of our web site so other users can vote for it: https://www.multicharts.com/pm/
Thank you for your feedback. Unfortunately there is no such functionality at the moment.
You may want to submit a feature request to the Project Management of our web site so other users can vote for it: https://www.multicharts.com/pm/