802217 - TB16 Redownloads messages/Failure to store messages locally - Constantly bringing folders "up to date" / horrible IMAP performance.

Status: RESOLVED DUPLICATE of bug 803843

(MailNews Core :: Networking: IMAP, defect)

Description

[Erik M Jacobs]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0
Build ID: 20121011153627

Steps to reproduce:

Upgraded to Thunderbird 16 with existing IMAP accounts.


Actual results:

It appears that Thunderbird 16 does something different with IMAP. I am seeing "Bringing Folder ... Up To Date" for many folders in the activity manager. As soon as this finishes, a few moments later, it seems to start again.

Navigating between messages in the folder seems to take a long time. It can take many moments for a message to appear in the preview pane once selected, even after making sure that mail is sync'd/downloaded for local use by using "File" -> "Offline" -> "Download/Sync Now"

Even after clicking a message and having it come up in the preview pane, clicking off the message and clicking back onto the same message results in a "Loading Message" -> "Downloading Message" notification, as if the message was never downloaded from the server in the first place.


Expected results:

Nothing should have changed.

Comment 1

[Wayne Mery (:wsmwk)]

Does this also happen when started in Thunderbird safe mode?  (I expect it would, but we should check before going much further)

Please install glodaquilla addon, and determine that messages are in fact stored locally.  (you'll need to add the column)

Please obtain imap+autosync log per https://wiki.mozilla.org/MailNews:Logging ... might be good to do it with just a small folder
use   imap:5,ImapAutoSync:5

I don't think there are any bugs reported in this area that started in version 16

Comment 2

[Erik M Jacobs]

1) Yes, this also happens in safe mode.

2) I have installed the glodaquilla addon, but cannot figure out how to make it work.  Trying to open the options screen for Glodaquilla in Thunderbird 16 doesn't work. I click the "options" button and no window appears.  But I can't click anywhere else in Thunderbird at that time. If I hit CTRL-W then the AddOns menu disappears and control is restored, but I see no way to modify the Glodaquilla options.

3) I don't have any small folders. The log appears to have a lot of "unsafe" information (email contents, email addresses, etc). Is there a way to cleanse it?

I now have several users who are all complaining about issues after upgrading to Thunderbird 16.

Comment 3

[Erik M Jacobs]

Note that every single time I open Thunderbird, it appears that Thunderbird tries to re-download every message from every folder.

Thunderbird 15 does *not* do this.

Comment 4

[Wayne Mery (:wsmwk)]

a short imap log may help us.

Irving may know if this relates to any other issues we have with tragic message redownloading

Comment 5

[Erik M Jacobs]

How can I obtain a "short" IMAP log?

The parser script referenced in the logging wiki gives me errors:
[thoraxe@t510dora Downloads]$ python nspr_imap_log_parser.py imap.log 
Traceback (most recent call last):
  File "nspr_imap_log_parser.py", line 266, in <module>
    parser.parse(f)
  File "nspr_imap_log_parser.py", line 129, in parse
    timestamp = int(line[1:idx])
ValueError: invalid literal for int() with base 10: ''

I suppose I could manually parse the log.  Can you provide any insight on what you're looking for?

Comment 6

[Erik M Jacobs]

Attachment: first 287 lines of imap logging

Here are the first 287 lines of the imap log generated by the requested log settings.

Comment 7

[Erik M Jacobs]

It appears that, despite the settings to keep messages stored offline, messages are not stored offline:

The body of this message has not been downloaded from the server for reading offline. To read this message, you must reconnect to the network, choose Offline from the File menu and then uncheck Work Offline. In the future, you can select which messages or folders to read offline. To do this, choose Offline from the file menu and then select Synchronize. You can adjust the Disk Space preference to prevent the downloading of large messages.

Note that the settings for this account are set to store/sync for offline:
Keep Messages for this account on this computer (checked)
All folders checked for "Download"
Synchronize all messages locally regardless of age
Don't delete any messages

Comment 8

[Wayne Mery (:wsmwk)]

Jay, can you havea look at the imap log? (though it may not reveal what's needed if this is a backend problem)

Comment 9

[Erik M Jacobs]

Can any of you think of a reason from the IMAP server side why Thunderbird would think that it needs to redownload the messages every time?

It appears this issue may be related to the zarafa-gateway IMAP server, but I'm not 100% sure.

Comment 10

[Erik M Jacobs]

This problem does not occur against zarafa-gateway with TB15. I just re-installed Thunderbird 15 on Win7x64 and everything worked perfectly. Messages didn't seem to be redownloaded, the activity manager wasn't indicating anything going on, the preview pane pulled up messages snappily.

Comment 11

[Nils]

Please see https://bugzilla.mozilla.org/show_bug.cgi?id=803843 ... I am also seeing this behaviour, with the IMAP files growing extremely large.

Comment 12

[Mark Banner (:standard8)]

Adding a dependency, although we're not sure if these are related yet.

Comment 13

[rkdixit]

I'm facing exactly the same issues as mentioned above. Emails with attachments will not open for hours. This is frustrating when I have to respond to customers immediately. Thunderbird 16 does not satisfy the basic requirement of email clients i.e. opening emails. Will someone escalate the severity of this issue and fix it please.

Comment 14

[Nils]

Please look at bug #803843 which is listed in the depends and is fixed. If you compact the folders this is happening in, there is a chance that this will help until the fix is in the build. If compacting all the folders does not help (also check if your free disk space drastically increases when you compact, then its an indication for that bug) then this is a different issue and probably needs more info to be tracked down.

Comment 15

[Mark Banner (:standard8)]

Without extra info, I'm going to assume this is a duplicate of bug 803843. 

If you still see issues in 16.0.2 which is being released later today, then please read bug 803843 comment 33 and onwards for information on what to do.

Comment 16

[Guillaume Melquiond]

I am not sure the two bugs are directly related. I was suffering from bug 803843 too, and TB 16.0.2 fixed it indeed. However, the new version still keeps downloading the same IMAP messages over and over (though that no longer causes the local folders to grow).

Comment 17

[Nils]

Reopening bug #802217 as bug #806760 as it is a separate issue from bug #803843 as per https://bugzilla.mozilla.org/show_bug.cgi?id=803843#c33

This is happening (at least) for 16.0.2 as downloaded from Ubuntu.

Comment 18

[Erik M Jacobs]

This is not a duplicate. I am not experiencing growing IMAP filesizes with TB16 or TB16.0.2 on Win7x64.  I am still experiencing constant redownloading and reindexing of various folders, and slow performance.

Comment 19

[Tomas Gustavsson]

For me this is also a critical problem. I have been forced to ditch Thunderbird for now. It always user 100% of all CPU cores downloading everything over and over again. I want to use Thunderbird, but right now it is not possible.

This is _NOT_ a dupliace of 803843. My mailfiles never exceeds 181MB, no matter how many times it downloads everything.

Comment 20

[Tomas Gustavsson]

Ok just found 806760, sorry.