1334824 - On startup, or getting imap mail clicking Inbox, not responding with high memory (OOM?). Reordering (12 dovecot) accounts helps.

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[dpln_mozilla]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161208153507

Steps to reproduce:

do not know, last action I did was delete some folders, however in another account, do not believe easy reproducible, but the log message might point to an issue


Actual results:

thunderbird got unusable for me on every start it immediatly goes to not responding.
starting it up with no network was ok, so I disabled sync on startup of the account that was mentioned in the starus bar upon freezing. now thunderbird starts aup alright, as soon as I click on the inbox of this account (imap), not responding again. 
I do not know why this happens, the last thing I did, was delete some folders, however in another mail account.


While doing this, one thread is fully working and memory increases continuosly. 
Same thing in Windows save mode and with disabled add ons.

I tried logging imap but there was nothing in the log file at the time of freeze, so I set log level to all:5, 

the log file gets quickyl filled with:
...
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.274000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.275000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
2017-01-28 17:15:54.275000 UTC - 4712[1811140]: nsComponentManager: CreateInstanceByContractID(@mozilla.org/array;1) succeeded
... and it goes on like this forever ...




Expected results:

mails should be displayed instead of not responding forever (killed TB after 10 min)

Comment 1

[Wayne Mery (:wsmwk)]

Start *Windows'* safe mode with networking enabled
- win10 https://support.microsoft.com/en-us/help/12376/windows-10-start-your-pc-in-safe-mode
Still In Windows safe mode, start thunderbird in safe mode
- https://support.mozilla.org/kb/safe-mode-thunderbird

Does problem go away?

Comment 2

[dpln_mozilla]

I already did as I wrote above, but maybe I made this not clear enough. 
So I did it now again, safe mode windows, then safe mode thunderbird, and it still happens.  
I let it run longer that usually to see if it recovers, however after TB memory usage  reached 1GB (from well below 200MB), I killed it all and gave up.

Comment 3

[Wayne Mery (:wsmwk)]

Given that this is imap, open your profile folder in Windows https://support.mozilla.org/en-US/kb/profiles-tb, shutdown Thunderbird, navigate to imapmail directory and then account directory, move Inbox and Inbox.msf files to somewhere else, start thuderbird.

Comment 4

[dpln_mozilla]

there are 5 similar name folders, the difference being -1 -2 etc added to the servername, 
after reading on the web, I then identified the proper folder with prefs.js content, 
then moved inbox and inbox.msf out of thunderbird profile, 
now restarted thunderbird, this unfortunately did no really help, did freeze again.

next, I will do a full backup of the profile, then fully delete the account and try to  add it again , this will take a while until all mails are back downloaded

Comment 5

[Wayne Mery (:wsmwk)]

What is your current status? Sorted out?

Comment 6

[dpln_mozilla]

well, for me kind of yes, but I am not happy about the solution, as It should not happen.
What helped: After backup of profile, I deleted and recreated manually the mail accounts and redownloaded all mails. 

What else I did: I restored the backup on another PC and "played" around with the preference files, 
trying to change the accounts directly there, create a new entry, etc. basically just understanding how it works. 
Then, with an new restore, I found the following, it turns out, just bei rearranging the order of accounts (mail.accountmanager.accounts), sometimes TB works, changing the order back to original order, it fails again. How can it be that the order is relevant for such a bug?

Now I am wondering, if the recreation above worked because of the recreation or because of the new order after recreation. But I really do not want to test this on my productive profile and also, this not on top of my todo list at the moment.

Comment 7

[Judy]

I am not using Imap but having the same problem, on two computers.  One is Win10 the other is Win7, both become "not responding" as soon as I open it and click on anything (including the help tab, to get the version number)

Comment 8

[Wayne Mery (:wsmwk)]

(In reply to Judy from comment #7)
> I am not using Imap but having the same problem, on two computers.  One is
> Win10 the other is Win7, both become "not responding" as soon as I open it
> and click on anything (including the help tab, to get the version number)

Judy, Please try the beta at http://www.mozilla.org/en-US/thunderbird/channel/

Comment 9

[Wayne Mery (:wsmwk)]

> there are 5 similar name folders, the difference being -1 -2 etc added to the servername, 
> .... then moved inbox and inbox.msf out of thunderbird profile, ... did not help

(In reply to neiaberau from comment #6)
> ...
> What else I did: I restored the backup on another PC and "played" around
> with the preference files, 
> trying to change the accounts directly there, create a new entry, etc.
> basically just understanding how it works. 
> Then, with an new restore, I found the following, it turns out, just by 
> rearranging the order of accounts (mail.accountmanager.accounts), sometimes
> TB works, changing the order back to original order, it fails again. How can
> it be that the order is relevant for such a bug?

what do you think - bad .msf file?
It seems to often be causes of high memory.
Does Thunderplunger addon still work these days?

Comment 10

[Judy]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #8)
> (In reply to Judy from comment #7)
> > I am not using Imap but having the same problem, on two computers.  One is
> > Win10 the other is Win7, both become "not responding" as soon as I open it
> > and click on anything (including the help tab, to get the version number)
> 
> Judy, Please try the beta at
> http://www.mozilla.org/en-US/thunderbird/channel/

Tried, problem remains

Comment 11

[Matt]

Something that has worked for me with Firefox,  and seen reports of it working for imap issues is to clear the cache.

Three bar icon on the toolbar > Options > Advanced > Network and disk space.

Comment 12

[Wayne Mery (:wsmwk)]

(In reply to Matt from comment #11)
> Something that has worked for me with Firefox,  and seen reports of it
> working for imap issues is to clear the cache.
> 
> Three bar icon on the toolbar > Options > Advanced > Network and disk space.

Judy, can you try that please?

It's an excellent point. I had someone else mention that a couple weeks ago.

Comment 13

[Judy]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #12)
> (In reply to Matt from comment #11)
> > Something that has worked for me with Firefox,  and seen reports of it
> > working for imap issues is to clear the cache.
> > 
> > Three bar icon on the toolbar > Options > Advanced > Network and disk space.
> 
> Judy, can you try that please?
> 
> It's an excellent point. I had someone else mention that a couple weeks ago.

I am presuming you are saying to clear the cache in Firefox?  I did do that, didn't help.  If there is a way to clear the cache in Thunderbird, I can't do that, as it becomes unresponsive as soon as I open it, so I can't do anything inside the program

Comment 14

[Wayne Mery (:wsmwk)]

>  If there is a way to clear the cache in Thunderbird, I can't do that, as it becomes unresponsive as soon as I open it, so I can't do anything inside the program

my instructions were for Thunderbird. But if you can't get to the options menu in Thunderbird then 
- In Thunderbird Help | troubleshooting | Profile Folder | click "open folder" 
- DON'T do anything in this folder
- shut down thunderbird
- go back to that folder windows, and change "roaming" to "local"
- you should see many cache directories and a few files listed - delete everything in the list
- start thunderbird

Comment 15

[Wayne Mery (:wsmwk)]

And if you can't get to troubleshooting menu ... with Thunderbird shut down ...

- Use instructions at https://www.microsoft.com/security/portal/mmpc/shared/variables.aspx to get to 
%localappdata%\Thunderbird\Profiles
- navigate to your thunderbird profile, which is probably in the form of xxxx.default 
- delete everything in the list
- start thunderbird

Comment 16

[Wayne Mery (:wsmwk)]

neiaberau, 
what happens if you clear cache


> all:5
bad idea :)

Comment 17

[dpln_mozilla]

- clear cache does not help

funny thing I noticed, but cannot explain:
- when started with no network, all is ok, 
  until I turn on wifi and click get messages, so it seems network related

However
- when, while tb is not responding, I turn off network (set wifi to airplane mode in win10), 
  then tb stays on not responding, despite no networking

Comment 18

[Wayne Mery (:wsmwk)]

Backup Inbox and Inbox.msf file from your Thunderbird profile.
https://support.mozilla.org/t5/Install-Migrate-and-Update/Profiles-Where-Thunderbird-stores-your-messages-and-other-user/ta-p/14096

What is size of Inbox?   right+click folder and pick properties
What happens if you repair the folder?  right+click folder > properties > repair

Comment 19

[dpln_mozilla]

The inbox of the account in question is 5MB, however there are 12 mail accounts on this profile. 

side note, as not related to the problem:
Generally I keep Inboxes small, as I had already twice data lost due to thunderbird getting confused with mbox content and deciding to sync thhis with mail server. The whole folder Imap for all of the 12 mail acccounts is 3.41 GB on Disk, however on the mail servers it is only 1.9 GB.
Looking at the profile I see a some duplicated folders within some of the mail accounts with numbers at the end. The timestamps differ, so it seems to happen randomly. Thunderplunger did only recognize one orphaned mail account, but not the orphaned folders. 
end note.

repairing folder I cannot do directly, since as soon as I click on the folder TB freezes. 

So I started up thunderbird with networking disabled, then clicked on the folder to repair.
that immediattely deletes the whole inbox, and nothing more as no network was available, 
when I reenabled networking, TB froze again. as written above, while hanging, disabling network again, does not help. there is either no timeout for this networking operation, or it is not network related. 

so back to disable syncing on this account and no clicking on it.

Comment 20

[Wayne Mery (:wsmwk)]

In the original order, was the problem account also the default account?
How many messages in the Inbox of this problem account? 

> The whole folder Imap for all of the 12 mail acccounts is 3.41 GB on Disk, however on the mail servers it is only 1.9 GB.
Could be caused by folders redownloading.
Are any of these gmail accounts?

Comment 21

[Matt]

Could this have something to do with the max connections?  Like the issue where having more than 1 is a problem on Charter IMAP?

Comment 22

[dpln_mozilla]

By default account you mean the first account in the list?
No, it was the second account, when I move it to the end it works again. 
When I move it back to 2nd place, it fails again, did this several times to verify. 

size:
by the looks of the content, you are right, at some point in time tb seemed to have redownloaded the folders, leaving the old folders in place
no they are not gmail accounts, centos 6 with dovecot, as far as I know.

Comment 23

[Wayne Mery (:wsmwk)]

neiaberau,

Does the problem also happen with version 52?
http://www.mozilla.org/thunderbird/all.html

Comment 24

[dpln_mozilla]

I am mainly useing another mail client at the moment, however I restored the profile (where I redordered the accounts), updated TB to 52.6 (32bit), update to current status, TB was working, this is on Win 10 Home 64bit

then reorderd the accounts (to what was a critical order the last time where it all started), restarted TB
  => TB again not responding and using one cpu thread fully and slowly increasing memory

Comment 25

[Wayne Mery (:wsmwk)]

(In reply to neiaberau from comment #4)
> there are 5 similar name folders, the difference being -1 -2 etc added to
> the servername, 

This is similar to the issues that Gene and Jorg are working on, and probably a duplicate. Reordering of accounts probably changed the timing of events affecting the problem account.

Comment 26

[Jorg K (CEST = GMT+2)]

I'm afraid I can't help here. I've seen "-1" being added to folder names, like Inbox, and there's a bug for that. I haven't seen it for servers, but if you create various accounts on the same server that's expected (I think).

Other than that, what exactly is the question?

Comment 27

[gene smith]

I have 16-17 accounts running (well mostly just for testing) and am constantly abusing them and never see a "hang". I've also sometimes seen imapmail/ directory names with -X (X = 1,2...) on the end but they seem rather innocuous.
I don't see this bug as a duplicate of anything I'm working on and it doesn't seem like the charter imap bug referred to above. 
I would say, if the reporter is still interested, an imap log would be the good step in seeing what is going on.
see https://wiki.mozilla.org/MailNews:Logging
Run these command until tb hangs, then shutdown/kill tb and attach the resulting imap.log file:
set MOZ_LOG=IMAP:5,timestamp
set MOZ_LOG_FILE=%USERPROFILE%\Desktop\imap.log
"%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"

Comment 28

[dpln_mozilla]

as I already wrote in the initial mail: 
  > I tried logging imap but there was nothing in the log file at the time of freeze, so I set log level to all:5, 

nevertheless, I tried again today, it is still freezing. see attached log with imap and timestamp logging
as soon es a find out how to attach, I will attach the log

I switched to claws mail 4 months ago ....

Comment 29

[dpln_mozilla]

Attachment: imap log, server and account names anonymized

log with imap set

Comment 30

[gene smith]

Were there problems with the password or logging in? The log just stops suddenly after tb ran for a couple minutes. Your note says you killed it. What was it doing when you killed it?  Otherwise, the log looks fairly normal.

Comment 31

[dpln_mozilla]

- no issue with password
- I killed it as it was not responding and "eating" more and more memory as written in the initial post

Comment 32

[Wayne Mery (:wsmwk)]

> reordering accounts helps

Perhaps in bug 1253235 the net effect of recreating the profile was a reordering of accounts which resolved that user's issue (in that case all pop accounts)

How many imap accounts do you have?

(have you saved your current failing profile for future forensics?  it could be helpful)

Comment 33

[Wayne Mery (:wsmwk)]

neiaberau,

Can you give us an update to comment 32?

Comment 34

[Wayne Mery (:wsmwk)]

neiaberau seems to be gone and we don't have a testcase, so => unfortunately incomplete. But it is an interest issue and we'd be interested in more information, including having a look at the profile

(In reply to neiaberau from comment #29)

log with imap set

Gene wrote there's nothing interesting here, so we are stuck until there is more information. To restate a previous comment, memory issues often involve corrupt .msf files.