When Interactive Brokers hangs, MC leak slowly eats up all memory

Questions about MultiCharts and user contributed studies.
jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 02 Jan 2020

I hit this again over the holiday. I'm using IB TWS 978.1g (latest) and MC 5.0.19396.400

When you finally get enough resources to shut down MC, you can see it slowly, slowly freeing all of the objects and then finally exiting. It is the tsServer.exe process that holds most of the memory.

The rate of growth is not huge - if I catch the error dialog (sorry there weren't enough resources to grab a screen shot) in time, I can re-authenticate TWS and all is well. It is when it is unattended like over a New Year's holiday that it builds up enough to catastrophically impact the machine.

Anyone else seen this? It has happened to me perhaps 5 times in 6 months and takes from 1-2 hours to try to "stop gracefully" so that in-memory data is flushed to the db.

User avatar
syswizard
Posts: 295
Joined: 15 Dec 2012
Has thanked: 16 times
Been thanked: 28 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby syswizard » 02 Jan 2020

No question this is a serious problem. Any chance you can start running on the new version of MC ?
Version: MultiCharts64 OpenBeta Version 14.0 (Build 19285)
Taking that action will more likely result in this being resolved in the upcoming edition of MC rather than the current version.
Another idea is reverting back to an older release of IB Gateway. I am on release 978.1b. Thus, release 978 has experienced 5 mini-updates since my release. That is not a good sign. Finally, instead of using TWS, you should probably be using the IB Gateway. It's basically a stripped-down version of TWS...and therefore has a much lower memory footprint. I have found it to be rock-solid as long as it's Windows execution priority is above that of MC.

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 15 Jan 2020

Has anyone else seen this?

I admit I worry about installing a beta without knowing there are some changes that might improve it. Gateway also does the restart now and my guess is that it is the restart that causes this. I did upgrade to the 978.1h version and it still happened. How far away might MC 14 production release be?

This other topic indicates increasing a timeout for autotrading. Could that affect charting as well?
viewtopic.php?t=52355

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 22 Jan 2020

Hello, jek ,

If MultiCharts (Portfolio, QuoteManager) collects real-time data during several hours or days, then call for the “Flush Cashed Data to Database” option (main MultiCharts window -> File -> Flush Cashed Data to Database, or QuoteManager -> Edit -> Flush Cashed Data to Database) 1–2 times a day. It should free up tsServer.exe process faster when you close MC.

As for memory leak, we would ask you to provide additional information for analysis. When the issue is reproduced, please send us the report as described below before you close the application error window and restart MC.
Please take full sized screenshots demonstrating the issue (error dialog) and send us directly to our support email address. Also collect and send logs and dumps using Feedback application. You can access it through Windows Start Menu -> MultiCharts folder -> Feedback. In the Feedback window please select:
Attach a screenshot;
Attach Log files…;
Attach minidumps…;
Also enable the following options if you are ready to share your user data with us so that we could reproduce the issue on our end and investigate it:
Snapshot of the current MC setup;
Open workspaces…;
Data for active instruments…;
Include tick data;
Active indicators…;
Active signals…;
Then check “Upload collected data to help desk” and click Send.

If the Feedback is not accessible, please make dumps using MC Dumper:
https://www.multicharts.com/trading-sof ... lect_Dumps

After this is done, please close MultiCharts, make sure that all processes with MultiCharts in description are finished in Task Manager -> Details tab.

After which steps do you get this error?
When did you notice the issue for the first time? Did you have any changes in the platform or your system before the issue appeared?

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 23 Jan 2020

Flush Cached Data to Database, or QuoteManager
Thanks for the tip.

I have never been able to get any of these dump methods to work.

When failures like this happen, the logs are too big and won't upload and transient information is gone after a reboot which is likely necessary.

It has happened perhaps 5 times in 4 months.

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 27 Jan 2020

jek,

You can use share your working environment with us so that we could test it on our end.

To do it please open the workspace where the issue is typically reproduced.
Go to Help -> Feedback.
In the description field please specify the steps to reproduce the issue.
Please select the options:
Snapshot of the current MC setup;
Open workspaces…;
Data for active instruments…;
Include tick data;
Active indicators…(if possible);
Active signals…(if possible);
Then check “Upload collected data to help desk” and click Send.

Or please come to our live chat MON-FRI from 7:00 AM till 12:00 PM EST to demonstrate the issue via remote connection. Live Chat is accessible from our web site http://www.multicharts.com/ (at the top of the page). We’ll do our best to help you.

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 21 Feb 2020

When it happened over night, I tried the "send minidump" from the error dialog panel. In addition this is the dialog that comes up noting that it has gotten confused about the IB TWS process going away. The tsserver.exe process rises slowly to > 4GB.
The TWS version in 978.1h
The MC version is MultiCharts64 Version 12.0 Release (Build 19396)
The OS version is Windows 8 version 6.3 (Build 9600)

On restart, it asked me and I sent some trace files just now.
Screen Shot 2020-02-21 at 06.35.49.png
(38.39 KiB) Not downloaded yet

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 27 Feb 2020

Hello jek,

What specifically do you mean by “Interactive Brokers hangs”? Have you tried to report this issue to IB support? Have you tried to restart IB application to resolve the issue?
Which steps do you perform in MC before the issue appears?

User avatar
TJ
Posts: 7740
Joined: 29 Aug 2006
Location: Global Citizen
Has thanked: 1033 times
Been thanked: 2221 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby TJ » 27 Feb 2020

When it happened over night, I tried the "send minidump" from the error dialog panel. In addition this is the dialog that comes up noting that it has gotten confused about the IB TWS process going away. The tsserver.exe process rises slowly to > 4GB.
The TWS version in 978.1h
The MC version is MultiCharts64 Version 12.0 Release (Build 19396)
The OS version is Windows 8 version 6.3 (Build 9600)

On restart, it asked me and I sent some trace files just now.

Screen Shot 2020-02-21 at 06.35.49.png
Unless you need the latest features, I would not download the latest TWS.

For trading with real money, you should use the "Stable" version, which is Version 972.1x.

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 27 Feb 2020

What specifically do you mean by “Interactive Brokers hangs”?
Once every few weeks, it fails to re-authenticate around the nightly reset and exits.
Have you tried to report this issue to IB support?
Yes, as have others.
Have you tried to restart IB application to resolve the issue?
Yes. As I mentioned, if I see the dialog within a few hours or even up to a day, I can re-authenticate IB and MC will catch up and behave just fine. If I don't it eventually fills up memory.
Which steps do you perform in MC before the issue appears?
Nothing. MC is just displaying some charts. The problem is that I leave it alone so long that it fills up memory.

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 28 Feb 2020

jek,

What does IB TWS show when it fails to re-authenticate? Please take a screenshot and send us.

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 28 Feb 2020

jek,

What does IB TWS show when it fails to re-authenticate? Please take a screenshot and send us.
Ok. It says something like "Logon failed - Soft token=0 ..."

See this screenshot: https://groups.io/g/twsapi/message/43092

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 06 Mar 2020

Jek,

We received the dumps that you sent from the error dialog panel and forwarded to our dev team. We continue to analyze this issue. We will let you know when there is any result.

User avatar
Svetlana MultiCharts
Posts: 645
Joined: 19 Oct 2017
Has thanked: 3 times
Been thanked: 163 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby Svetlana MultiCharts » 21 Apr 2020

Hello Jek,

Our dev team continues to investigate the issue.
Meanwhile we would recommend reporting the error that you see in IB TWS to IB support. Probably they can fix it on their end.
Also it will be helpful if you send us your MultiCharts workspaces with which you usually face the issue to our support email address.
To reproduce the issue you just leave your workspaces working, and when the error appears in IB TWS, you face the memory leakage in MultiCharts, correct? If you perform other steps please let us know.

jek
Posts: 181
Joined: 24 Dec 2006
Has thanked: 1 time
Been thanked: 2 times

Re: When Interactive Brokers hangs, MC leak slowly eats up all memory

Postby jek » 21 Apr 2020

To reproduce the issue you just leave your workspaces working, and when the error appears in IB TWS, you face the memory leakage in MultiCharts, correct? If you perform other steps please let us know.
Correct, that was all.


Return to “MultiCharts”