AutoTrade-partially filled order issue?

Questions about MultiCharts and user contributed studies.
SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

AutoTrade-partially filled order issue?

Postby SUPER » 28 Nov 2008

I have tested Auto Trade mode for few months now with IB and have come across a serious issue with partially filled orders in Async Mode and I am hoping that TS team will come out with a workable solution soon.

Bar resolution: 5 minute
AutoTrade Setting: Async Mode
Timeout= 0 seconds
OrderType= Limit
Strategy Mode=IOG (intrabar ordergeneration)
Strategy Type:Stop Reverse

Scenario 1:
Let us say we are long one contract and system generates a short signal and accordingly an order is sent to IB to sell two contracts, if only one contract gets filled the remaining order is not renewed and also conversion to market order does not take place thus creating a position mismatch between chart and broker. The reason for this I believe is that MC's auto engine treats partial orders as filled.


Scenario 2:

Timeout = 10 seconds

Let us say 5 seconds are left for completion of a bar and an order is filled at chart level but not at broker as yet, the timeout event is triggered and it will wait for 10 seconds to convert the order into market order. Now since the timeout event of 10 seconds can not happen within this bar as only 5 seconds are let to complete this bar the conversion of order into market will not happen thus creating a position mismatch between chart and broker.


I will appreciate if other users have anything more to add to this or any other issues they have come across with autotrade.

Regards
Super

brendanh
Posts: 157
Joined: 07 Apr 2007

Postby brendanh » 08 Dec 2008

Although I have not suffered these probems, it is probably because I've been lucky rather than because they don't exist.

If there isn't code that specifically handles these situations then they are bugs to be fixed. Can someone from TSS confirm?

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Re: AutoTrade-partially filled order issue?

Postby Marina Pashkova » 09 Dec 2008

I have tested Auto Trade mode for few months now with IB and have come across a serious issue with partially filled orders in Async Mode and I am hoping that TS team will come out with a workable solution soon.

Bar resolution: 5 minute
AutoTrade Setting: Async Mode
Timeout= 0 seconds
OrderType= Limit
Strategy Mode=IOG (intrabar ordergeneration)
Strategy Type:Stop Reverse

Scenario 1:
Let us say we are long one contract and system generates a short signal and accordingly an order is sent to IB to sell two contracts, if only one contract gets filled the remaining order is not renewed and also conversion to market order does not take place thus creating a position mismatch between chart and broker. The reason for this I believe is that MC's auto engine treats partial orders as filled.


You are right: MC considers partially filled orders as filled. A substitute market order for the remaining unfilled part is not issued. Hence a subsequent position mismatch between the broker and MC. As a workaround you can either

1. Use All or None Execution options in Stategy Properties -> Auto Trading -> Interactive Brokers -> Settings -> IB Settings. This option will not be available for all symbols, though.

2. Use the synchronous auto trading mode. Although it will not change the partial order filling behavior, it will help to avoid the position mismatch.

Scenario 2:

Timeout = 10 seconds

Let us say 5 seconds are left for completion of a bar and an order is filled at chart level but not at broker as yet, the timeout event is triggered and it will wait for 10 seconds to convert the order into market order. Now since the timeout event of 10 seconds can not happen within this bar as only 5 seconds are let to complete this bar the conversion of order into market will not happen thus creating a position mismatch between chart and broker.


This is a classical position mismatch that can result when the asynchronous auto trading mode is used. Either using a shorter timeout or switching to the synchronous auto trading mode will help.

Regards.

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 09 Dec 2008

Marina,

Many thanks for your feedback, I have few suggestion which may solve this problems, It would be nice if other members can also come up with possible solutions so that the Autotrade can work in most efficient manner.

Scenario1 - IB provides information of unfilled order status and that information can be used in the Autotrade logic to overcome this problem.

Scenario2 - I think the conversion should not be dependent on bar event, it should run independent to the bar status logic. What I mean is that once the count down starts it should issue conversion market order at the time out and should not look whether it is within the same bar or next bar, only then it would have true synchronisation with chart and broker.

Regards

brendanh
Posts: 157
Joined: 07 Apr 2007

Postby brendanh » 09 Dec 2008

1. Use All or None Execution options in Stategy Properties -> Auto Trading -> Interactive Brokers -> Settings -> IB Settings. This option will not be available for all symbols, though.

What is All or None Execution? Like most of the IB broker settings, this isn't documented in the help system.

2. Use the synchronous auto trading mode.

This isn't possible if you want to use limit order conversion to ensure they are filled every time. Likewise sync mode can't be used by people who want their exit signals to be filled when starting autotrading in a chart and RW position.

I agree with Super, the limit order conversion timer should be bar-independent.

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 18 Dec 2008

Hi SUPER,

Below, please find my comments regarding your two scenarios.



Scenario1 - IB provides information of unfilled order status and that information can be used in the Autotrade logic to overcome this problem.


Here's what we are going to do. If, for example, you want to reverse a position, but only one half of the reversal has been completed using a 'price' order, MC will know that only a part of the order has been filled and a market order will be issued for the remaining part - to guarantee order filling.

Scenario2 - I think the conversion should not be dependent on bar event, it should run independent to the bar status logic. What I mean is that once the count down starts it should issue conversion market order at the time out and should not look whether it is within the same bar or next bar, only then it would have true synchronisation with chart and broker.


In the scenario that you are describing, the time remaining for the conversion to be completed will be determined dynamically. For example, you have a 10 second timeout and there are 5 seconds left for the bar completion. In this case, the 10-second value will be disregarded and the timeout will equal the time remaining till the bar completion (5 seconds in your example).

Best regards.

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 18 Dec 2008

Marina,

Many thanks for your update it sounds like a workable solution, look forward to New Feature soon.

Regards
Super

2haerim
Posts: 502
Joined: 01 Sep 2006
Been thanked: 2 times

Postby 2haerim » 22 Feb 2009

Can anybody explain the meaning of All or None option in IB Autotrading settings?

Thanks

brodnicki steven
Posts: 407
Joined: 01 Jan 2008
Been thanked: 3 times

Postby brodnicki steven » 22 Feb 2009

Can anybody explain the meaning of All or None option in IB Autotrading settings?

Thanks


* I think it refers to a trade execution, ONLY if it can be completely filled at the price specified, no Partial fills are allowed. If it can't completely fill, then the order is skipped (ignored).

2haerim
Posts: 502
Joined: 01 Sep 2006
Been thanked: 2 times

Postby 2haerim » 22 Feb 2009

Code: Select all

Input: Qty(10);

if LastBarOnChart then begin

if MarketPosition = 0 then
      buy Qty Contracts next bar at market
else if MarketPosition = 1 then
   sell next bar at market;

end;   // LastBarOnChart


Let's assume Async Autotrade mode.

[1] All 10 buy were filled at one time at the same bar. => Ideal and ok.
[1-1] only 3 buy were filled initially and a bit later remaining 7 were filled at the same bar. => I think this is ok, too.
[2] 3 of the initial 10 buy were filled at the current bar, but the remaining 7 were filled across next multiple bars.
[3] None of the initial 10 buy were at the current bar, but all filled at one of next coming bars. Fill can be one time full fill or multiple partial fills within that bar.
[4] None of 10 buy were filled at all for pretty long time.
[5] Will the reactions on these scenarios be identical in time-based or tick-based chart?


[3] and [4] are not much probability, but who knows? There are all kinds of brokers and anything can happen in real world.

From more experiment, in case [4], MC would wait for a predefined time to ignore the current order and issue a new one. Don't know how long it waits.


I would like to know what will happen in each of these scenarios. Once I figure it out thoroughly, I would be able to proceed to more complicated code like:

Code: Select all

inputs: PL(0.1), QTY(1);
vars: MP(0), EP(0), MyBN(0);

If LastBarOnChart then begin
  MyBN = currentbar + MaxBarsBack;
  MP = MarketPosition;
  EP = EntryPrice;
  print(d:7:0,t:5:0," ","BA",MP:3:0, mod(MyBN,2):3:0,EP:7:2);
end;

If MP = 0 then
  IF mod(MyBN,2) = 0 then
    Buy ("BE") QTY Contracts Next Bar at Market
  Else
    Sellshort ("SE") QTY Contracts Next Bar at Market
Else If MP = 1 then
begin
  Sell ("SP") QTY Contracts Next Bar EP + PL Limit;
  Sell ("SL") QTY Contracts Next Bar EP - PL Stop;
end
Else If MP = -1 then
begin
  BuyToCover ("BP") QTY Contracts Next Bar EP - PL Limit;
  BuyToCover ("BL") QTY Contracts Next Bar EP + PL Stop;
end;


Partial fills or Delayed fills for this case would be more complicated to understand how MC reacts unless we know the first simple case.


Anybody have experiences on partial fill please share them here.


I hope TSS can list up all kinds of exceptional cases and describel how MC reacts on them.

[/code]

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 23 Feb 2009

[1-1] partial orders will not get filled as MC is not able to handle partial fills at present time. So only 3 initial fill will be valid.
[2] Unless your code can detect that only 3 contracts were filled and resubmit order for remaining unfilled order ( I don't know how this can be done in MC).
[3] this can only happen if the buy condition are still valid for next bar, but if the order is filled at chart level then there is no way you can keep order valid for next bar at brokers end.
[4] as long as the buy condition are valid the order will remain open at brokers end
[6] the order handling capability will work identical on tick and time based charts.

User avatar
TJ
Posts: 7078
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 985 times
Been thanked: 2022 times

Postby TJ » 23 Feb 2009

why do people use async mode?

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 24 Feb 2009

why do people use async mode?


Because it allows me an option of "unfilled order replacment", when I am using Limit orders.

User avatar
TJ
Posts: 7078
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 985 times
Been thanked: 2022 times

Postby TJ » 24 Feb 2009

why do people use async mode?

Because it allows me an option of "unfilled order replacment", when I am using Limit orders.

IC, thx.

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 24 Apr 2009

Hi SUPER,

Below, please find my comments regarding your two scenarios.



Scenario1 - IB provides information of unfilled order status and that information can be used in the Autotrade logic to overcome this problem.


Here's what we are going to do. If, for example, you want to reverse a position, but only one half of the reversal has been completed using a 'price' order, MC will know that only a part of the order has been filled and a market order will be issued for the remaining part - to guarantee order filling.

Scenario2 - I think the conversion should not be dependent on bar event, it should run independent to the bar status logic. What I mean is that once the count down starts it should issue conversion market order at the time out and should not look whether it is within the same bar or next bar, only then it would have true synchronisation with chart and broker.


In the scenario that you are describing, the time remaining for the conversion to be completed will be determined dynamically. For example, you have a 10 second timeout and there are 5 seconds left for the bar completion. In this case, the 10-second value will be disregarded and the timeout will equal the time remaining till the bar completion (5 seconds in your example).

Best regards.


Marina,

Many thanks for incorporating solution for partial fill into beta 6.

Please do not forget to include solution to scenario2, "dynamic timeout for conversion " in your next upgrade.

May I point out that at present we are not able to select timeout > 0 using IOG mode for conversion of unfilled order, it would be useful and consistent if this feature was available for IOG system too.

Regards
Super

User avatar
Marina Pashkova
Posts: 2758
Joined: 27 Jul 2007

Postby Marina Pashkova » 24 Apr 2009

Hi Super,

Replacement for partially filled orders was a very good suggestion. I hope you will now be able to take advantage of this feature in the latest beta.

As for Scenario 2, it now works the following way: limit/stop orders are replaced by market orders either after the timeout has expired or on closing the bar - whichever happens first.

As for the IOG mode, how would you like to see it work in Scenario 2?

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 25 Apr 2009

Marina,

Very happy to hear that the Scenario-2 situation has been fixed too-Thank you very much.

With regard to IOG, it should work exactly same as solution you have provided for Scenario-2. " limit/stop orders are replaced by market orders either after the timeout has expired or on closing the bar - whichever happens first". This feature would allow sufficient time for IOG order to be filled at Limit price.

Regards
Super

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Postby SUPER » 26 May 2009

Marina,

Very happy to hear that the Scenario-2 situation has been fixed too-Thank you very much.

With regard to IOG, it should work exactly same as solution you have provided for Scenario-2. " limit/stop orders are replaced by market orders either after the timeout has expired or on closing the bar - whichever happens first". This feature would allow sufficient time for IOG order to be filled at Limit price.

Regards
Super


Marina,

Any news on Conversion of Stop/Limit orders in to market after timeout event for IOG orders.

Will appreciate your feedback.

Regards
Super

SUPER
Posts: 629
Joined: 03 Mar 2007
Has thanked: 101 times
Been thanked: 81 times

Re:

Postby SUPER » 25 Oct 2010

Any news on Conversion of Stop/Limit orders in to market after timeout event for IOG orders please?


SUPER wrote:
Marina,

Very happy to hear that the Scenario-2 situation has been fixed too-Thank you very much.

With regard to IOG, it should work exactly same as solution you have provided for Scenario-2. " limit/stop orders are replaced by market orders either after the timeout has expired or on closing the bar - whichever happens first". This feature would allow sufficient time for IOG order to be filled at Limit price.

Regards
Super


Marina,

Any news on Conversion of Stop/Limit orders in to market after timeout event for IOG orders.

Will appreciate your feedback.

Regards
Super


Return to “MultiCharts”