Closed Bug 816327 Opened 12 years ago Closed 3 years ago

Constantly "Downloading M of N in All Mail" or Inbox messages from Gmail IMAP account, slows system from redownloading message headers

Categories

(MailNews Core :: Networking: IMAP, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: bdevine, Unassigned)

References

(Blocks 4 open bugs, )

Details

(Keywords: perf, regression, Whiteboard: [gs][regression:TB18 bug 721316])

User Story

https://support.mozilla.org/en-US/questions/1040391
https://support.mozilla.org/en-US/questions/1079013
https://support.mozilla.org/en-US/questions/1117013
https://support.mozilla.org/en-US/questions/1203956
https://support.mozilla.org/en-US/questions/1205939
https://support.mozilla.org/en-US/questions/1217321 (5/16/2018 claims beta resolved issue)
https://support.mozilla.org/en-US/questions/1242826
https://support.mozilla.org/en-US/questions/1243371
https://www.reddit.com/r/Thunderbird/comments/9259gh/whats_happening_very_slowdownloading_x_of_xxxxxx/

Attachments

(2 files)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20100101 Firefox/16.0
Build ID: 20121024073032

Steps to reproduce:

Nothing


Actual results:

Almost all of the time, the lower left hand bar will cycle through each mailbox I have.  For exmaple, it will say "1 of 5233 messages in All Mail"  and it will cycles all the way to 5233, and then start over.  This goes on all day and has been going on for several weeks.  It really seems to be slowing down my system.

I'm using Earlybird, version 190a2
I'm using Gmail through IMAP
Summary: constantly cycling e-mail → Constantly reiterating (redownloading?) messages from IMAP account
Component: Untriaged → Networking: IMAP
Product: Thunderbird → MailNews Core
Blocks: tb-gmailWIP
Summary: Constantly reiterating (redownloading?) messages from IMAP account → Constantly reiterating (redownloading?) messages from Gmail IMAP account
Hey Brian, are you facing the same problem in TB-17?
(In reply to Atul Jangra [:atuljangra] from comment #1)
> Hey Brian, are you facing the same problem in TB-17?


I'm seeing this. Sending log to Atul :)
Flags: needinfo?(bdevine)
Atul has indicated that it should be downloading only headers, and that this this is to be expected.

However, 
a) a few users are reporting performance issues during this period
b) there is *significant* delay (caused by gmail?) between when status bar says "1 of N" messages and when it actually completes - 10 - 40+ seconds for example

un?/related? bug 815730

starting bug query https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=nowordssubstr;list_id=5485012;short_desc=download;field0-1-0=short_desc;field0-0-0=short_desc;bug_severity=major;bug_severity=normal;resolution=---;resolution=FIXED;resolution=INVALID;resolution=DUPLICATE;resolution=WORKSFORME;chfieldto=Now;query_format=advanced;chfieldfrom=5m;value0-1-0=attach%20junk%20newsg;short_desc_type=allwordssubstr;longdesc=downloading;type0-0-0=nowords;value0-0-0=numbers%20pop%20rename;product=MailNews%20Core;product=Thunderbird;longdesc_type=allwordssubstr
Keywords: perf
Whiteboard: [gs]
(In reply to Wayne Mery (:wsmwk) from comment #3)
> However, 
> a) a few users are reporting performance issues during this period
> b) there is *significant* delay (caused by gmail?) between when status bar
> says "1 of N" messages and when it actually completes - 10 - 40+ seconds for
> example

The delay I'm seeing while status bar says "x of y messages" for my 50 or so larger messages on my gmail account is definitely a lot longer than just downloading headers, so I strongly suspect that TB unnecessarily re-downloads entire messages during that period, although those messages have not changed in any way.
also reported at http://forums.mozillazine.org/viewtopic.php?f=39&t=2624333&sid=78966c80e4bd34d8f771201cdf1c2315&p=12636257#p12636257

possibly https://getsatisfaction.com/mozilla_messaging/topics/thunderbird_is_slow (pending more info)

(we should also remember ask users if their profile/thunderbird account is on a network drive)
Severity: normal → major
Summary: Constantly reiterating (redownloading?) messages from Gmail IMAP account → Constantly reiterating (redownloading?) messages from Gmail IMAP account, slows system
I looked at the imap protocol log. I think that we are not re-downloading the messages. It's just the re-iterating of the headers. I will look into the situation developing on the get-satisfaction. Currently my exams are going on :(
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?
Summary: Constantly reiterating (redownloading?) messages from Gmail IMAP account, slows system → Constantly "Downloading M of N in All Mail" or Inbox messages from Gmail IMAP account, slows system
Thanks Atul.

possible regression of bug 721316?  (which landed 2012-10-10)
Flags: needinfo?
Keywords: regression
Whiteboard: [gs] → [gs][regression:TB17?]
Blocks: 837620
On speedy Dell at work I've seen gmail "downloading" a few times and indeed it makes navigation and viewing messages slower.

Since I set imap server directory to [Gmail] in my advanced account settings it might now be downloading more. Can't say that's definite yet.
Wayne, can you send me an IMAP protocol log for this? I've tried to create the same situation on my TB, it shows "downloading", but that is just the headers, which we surely want to do.

(In reply to Wayne Mery (:wsmwk) from comment #9)
> On speedy Dell at work I've seen gmail "downloading" a few times and indeed
> it makes navigation and viewing messages slower.
> 
> Since I set imap server directory to [Gmail] in my advanced account settings
> it might now be downloading more. Can't say that's definite yet.
After seeing many instances of redownloading the messages, I think it can be  possible regression of Bug 721316. I've requested imap protocol log from reporters, I'd wait for logs to show up.
We really need to fix this issue.

(In reply to Wayne Mery (:wsmwk) from comment #8)
> Thanks Atul.
> 
> possible regression of bug 721316?  (which landed 2012-10-10)
I don't think it should be normal to download over 5000 items continuously even if it's just headers? Mine seems to never stop doing this. using tbird 17.0.3
Oh okay, Can you attach imap protocol here? Or maybe send it to me on my mail, if you'r not comfortable to attach here. That would be a lot helpful :-) Thanks
I will post an imap log if someone can instruct how to do this
Attached image screen shot of syncing
screen shot shows not only downloading, but also gloda indexing activity. And, during this period which was pretty long, thunderbird cpu average ~13%, which is a lot of activity on this fast PC

Also, Atul has my logs from private email.
I think autosync may be the culprit here. Those having this problem, please try disabling autosync.

If you don't know how to do this, open the Tools menu, then select 'Options...' Click the 'Advanced' button on the top row, make sure the 'General' tab is selected and then click the 'Config Editor...' button. Search for 'autosync', it should find something like 'mail.server.default.autosync_offline_stores'. Change the value here to false and restart thunderbird to make sure any running queues are cleared.

Here is a snippit from https://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsAutoSyncManager.h

24 /* Auto-Sync
25  *
26  * Background:
27  *  it works only with offline imap folders. "autosync_offline_stores" pref
28  *  enables/disables auto-sync mechanism. Note that setting "autosync_offline_stores"
29  *  to false, or setting folder to not-offline doesn't stop synchronization
30  *  process for already queued folders.
(In reply to Wayne Mery (:wsmwk) from comment #8)
> possible regression of bug 721316?  (which landed 2012-10-10)

I have done some more testing and I am afraid that this is true. More specifically, I think we need to tweak autosync to handle the changes from bug 721316.

Here is my hypothesis that I would like for people to help me prove. First you will need to download and install my gmailbuttons addon v0.6.0b6. https://addons.mozilla.org/en-US/thunderbird/addon/gmailbuttons/versions/0.6.0b6 

I have added a feature that lets you see the offline storage location that was added as part of bug 72131. One you have installed the addon, select a gmail folder and then right-click in column headers of the message list pane. There should be an column called "Offline Storage Location". Check it to enable it. You will have to do this for each folder I believe. 

Now, here is my hypothesis based on what I am seeing. The offline storage location will not match the currently selected folder on some messages. This was one of the main objectives for bug 72131 - done to conserve disk space. If you sort by Offline Storage Location, you should be able to easily count the number of message where the offline storage location does not match the current folder - even if you have 1000s of messages. You can do this by doing a shift-click to select all of the messages where the offline storage location does not match the current folder and then look at the status bar at the bottom right of the window to see how many messages are selected. I think you will find that the number of messages selected will equal the N in "Downloading M of N in <folder>".

Apparently autosync is not aware of the offline storage location and is trying to redownload those messages all of the time.


So I think we can spread the following info/workaround on getsatisfaction, etc. until we get this fixed:

If you have Thunderbird version >= 17 and you are using Gmail IMAP, then in the advanced config editor, set "autosync_offline_stores" to false for each of your gmail accounts.
(In reply to David Lechner (:dlech) from comment #18)
> ...
> Now, here is my hypothesis based on what I am seeing. The offline storage
> location will not match the currently selected folder on some messages. This
> was one of the main objectives for bug 72131 - done to conserve disk space.
> If you sort by Offline Storage Location, you should be able to easily count
> the number of message where the offline storage location does not match the
> current folder - even if you have 1000s of messages. You can do this by
> doing a shift-click to select all of the messages where the offline storage
> location does not match the current folder and then look at the status bar
> at the bottom right of the window to see how many messages are selected. I
> think you will find that the number of messages selected will equal the N in
> "Downloading M of N in <folder>".
> 
> Apparently autosync is not aware of the offline storage location and is
> trying to redownload those messages all of the time.

Bingo.  David, using your procedure I see this is true for the two gmail accounts on my laptop mentioned in my private email


I still need to check my work PC regarding attachment 724495 [details]
Blocks: 721316
Whiteboard: [gs][regression:TB17?] → [gs][regression:TB17]
Thanks David for the awesome analysis. Great work :-)
(In reply to Wayne Mery (:wsmwk) from comment #19)
> I still need to check my work PC regarding attachment 724495 [details]

situation different on PC at work...
it also has [gmail] in imap server directory but ...

it occasionally syncs the *entire* All Mail folder (and others) even though very few All Mail messages show Offline Storage Location other than All Mail. but...

sometimes it sync only as described in comment 19, downloading per David's theory. Note - in this case it includes messages where offline location is blank.  When I click on said messages, the location changes to All Mail.
(In reply to David Lechner (:dlech) from comment #18)
>..
> If you have Thunderbird version >= 17 and you are using Gmail IMAP, then in
> the advanced config editor, set "autosync_offline_stores" to false for each
> of your gmail accounts.

David, I see only mail.server.default.autosync_offline_stores
Flags: needinfo?(david)
Default values aren't written to prefs, so you have to add a pref for 
mail.server.<serverXX>.autosync_offline_stores
Flags: needinfo?(david)
(In reply to Magnus Melin from comment #23)
> Default values aren't written to prefs, so you have to add a pref for 
> mail.server.<serverXX>.autosync_offline_stores

Thanks. I didn't know this. 

(In reply to Wayne Mery (:wsmwk) from comment #22)
> David, I see only mail.server.default.autosync_offline_stores

Now that I understand this better, I am putting this down just to be clear. If you follow Magnus's suggestion, you can change the preference for accounts individually. Changing mail.server.default.autosync_offline_stores preference itself, it will affect all of your accounts.
I did the account on two computers and it does seem to help
if I turn off autosync, what will I miss out on? will I actually need to sync up anytime I wish to check for new mail?
Here is a short description of autosync - although is seems a bit dated.
http://kb.mozillazine.org/Offline_folders#AutoSync

Basically, with autosync on, all folders are checked for new mail during idle time. With autosync off, only the inbox is checked for new mail. To check other folders, you have to select them manually. So, if you have some filters setup on the server to deliver messages to folders other than the inbox, you will miss the new mail notifications. If everything is coming to your inbox, I don't think you will notice much of a difference.
Depends on: 854162
hmm, I got a big DL of All Mail today on work PC even with mail.server.server15.autosync_offline_stores = false
(In reply to Wayne Mery (:wsmwk) from comment #28)
> hmm, I got a big DL of All Mail today on work PC even with
> mail.server.server15.autosync_offline_stores = false

I think that per bug 854161, autosync is not the only area where there is a problem.

The following strings are specific to autosync, so if you see these, then it is probably an autosync issue. If you see a message in the activity manager or status bar other than these, please post it so that we can figure out where it is coming from.

> # LOCALIZATION NOTE (autosyncProcessProgress): Do not translate the word "%1$S", "%2$S" and "%3$S" below.
> autosyncProcessProgress=Downloading %1$S of %2$S in %3$S
> # LOCALIZATION NOTE (autosyncProcessDisplayText): %S will be replaced by the folder name
> autosyncProcessDisplayText=Bringing folder %S up to date
> # LOCALIZATION NOTE (autosyncEventDisplayText): %S will be replaced by the account name
> autosyncEventDisplayText=%S is up to date
> # LOCALIZATION NOTE (autosyncEventStatusText): %S will be replaced by total number of downloaded messages
> autosyncEventStatusText=Total number of messages downloaded: %S
> autosyncEventStatusTextNoMsgs=No messages downloaded
> # LOCALIZATION NOTE (autosyncContextDisplayText): %S will be replaced by the account name
> autosyncContextDisplayText=Synchronizing: %S
Blocks: 833661
(tagged several many more getsatisfaction topics with bug 816327)
I have made https://getsatisfaction.com/mozilla_messaging/topics/downloading_11000_of_11357_in_all_mail canonical for this bug. 

And now that bug 854162 is fixed I have posted there to request users test TB 21 beta
(In reply to Wayne Mery (:wsmwk) from comment #32)
> I have made
> https://getsatisfaction.com/mozilla_messaging/topics/
> downloading_11000_of_11357_in_all_mail canonical for this bug. 
> 
> And now that bug 854162 is fixed I have posted there to request users test
> TB 21 beta

results are mixed. Some people are helpd.

Some are not. eg. https://getsatisfaction.com/mozilla_messaging/topics/tb_17_constantly_downloading_all_folders


I just had a redownload today.
Flags: needinfo?(david)
I have not had time to really did deeply into this any more lately. I did notice one thing that has made a big difference in the performance of Thunderbird. For a long time, I had my Gmail "All Mail" folder set to not keep offline copies of messages (Tools -> Account Settings, Synchronization and Storage, Advanced... (button), uncheck "Download" for "All Mail" folder.

About 1 month ago, I changed this back ("Download" checked for "All Mail" folder) and have noticed periodic "freeze ups" especially when I am composing new messages. 

Perhaps disabling offline storage of "All Mail" could be a work around for people with this issue.
Flags: needinfo?(david)
Attached image syncing
after a few months of hiatus, I'm seeing this again. But not for "All Mail" folder afaict.  And it's not constant. Seemingly random, in small bits. And sometimes it's stuck with no CPU.
See Also: → 562977
See Also: → 708342
boomhauer, can you consistently reproduce this using a current version?


(In reply to Atul Jangra [:atuljangra] from comment #1)
> Hey Brian, are you facing the same problem in TB-17?
Brian seems to be gone
Flags: needinfo?(boomhauer)
Flags: needinfo?(bdevine)
Updated the bug URL to a support issue in 38.2

https://support.mozilla.org/en-US/questions/1079013

While Googling the issue I did run across a suggestion that disabling show all mail in the Google interface fixed the issue for mac mail.
https://discussions.apple.com/thread/4368870?start=15&tstart=0  While it is a workaround, it might be worth trying.

I assume the OP on that thread means disabling from IMAP in the gmail settings here https://mail.google.com/mail/#settings/labels
Just to inform, this still happens in TB 38.2.0 for Fedora fc22.

The workaround (unsubscribe from all mail) did not work for me.

The problem occurs with maildir or standard berkeley format.

BTW: corporate gmail.
Indeed I have this same issue on TB 38.2.0 running on a Windows 8.1 64 bits, only with Lightning 4.0.2.1 (default installation).

I already turned 'autosync' to 'false' and it seems to stop the 'downloading' stress, but as mentioned, I lost actualizations as mention in comment #27 (https://bugzilla.mozilla.org/show_bug.cgi?id=816327#c27).

BTW: My profile is over 38GB and I have 5 IMAP accounts and 3 POP3 accounts (if it matters).
Also happens on TB 45.2.0.
Same problem on TB 45.3.0, ubuntu packaged version, with lightning installed. Somehow a right click on All Mail -> Folder Properties -> Synchronization -> "Select this folder for offline use" keeps getting rechecked after turning it off, as well as in "Subscribe -> Folder List".
I am also facing this on TB 45.3.0 + Ubuntu
Also happens running TB 45.4.0 on Win 7, 64bit and Win 10, 64bit.
Also happens in 52.0.1, Arch Linux.
(In reply to zhangjunnan1224 from comment #45)
> Also happens in 52.0.1, Arch Linux.

Turning off auto-sync seems to resolve the issue
See Also: → 1379592
As a lot of people using a lot of different versions of TB since around v.16, I stumbled on this problem, as I was setting up a new computer and trying to sync all my email from my provider's IMAP server to the local space.

I was not able to do it, as I encountered:
- very long message per message mailbox downloading time
- messages download for a given mailbox pausing or stopping with no reason
- messages downloads switching from one mailbox to another without rime or reason (or let's say, depending on the folders I was trying to access in the browser window)
- messages download completing to something like 90% and then starting back again from zero, to no end
- impossibility to follow what TB is doing, as the Activity Manager show indications which are obviously not consistent with what is downloaded and what the status line at the bottom of the browser window shows

I finally found a workaround, asking TB to go offline, with prior sync.
And then, TB was able to download all my folders, at a reasonable speed, one after the other.

So my opinion is that these never ending complaints about IMAP slow, incomplete and restarting downloads is not a matter about transferring the messages, but a matter of how TB decides what to download and in which order to download it when syncing in background.
I write this, not hoping that this will be fixed as TB developers seem to turn a blind eye to this for more than 10 years, but to provide a workaround to people who will end up on this thread while googling for information.
My apologies, as I thought that I was very clever finding the workaround, but the main problem stays.

Even if the "work offline" trick helps to download all the folders, 3 of my biggest folders (1000 to 2500 emails) still fail to download completely.
While downloading headers, it constantly fails after a number of emails, e.g. 194 out of 938, and then restarting.
Is the cause:
- connection failure with the IMAP server?
- corrupted 194th or 195th email which makes the download process crash?
I can't guess -- I don't even know how to go to mail n. 194 in the displayed list.

Deleting the .msf file, one folder at a time, and then restarting TB and accessing the folder seems to bring some success. But nothing really in a reliable and repeatable manner.

Anyway, the main subject of this bug's discussion thread looks like still alive and ready to give more users some hard time.
Blocks: 1213021
User Story: (updated)

datapoint - The last time this happened, but in a big way, was the horrible version 16 regression Bug 806760 - TB16 Redownloads messages / Constantly bringing folders "up to date" / horrible IMAP performance - which of course was fixed in version 17.0 and version 16 was canned.

User Story: (updated)
OS: Windows 7 → All
Hardware: x86_64 → All
Summary: Constantly "Downloading M of N in All Mail" or Inbox messages from Gmail IMAP account, slows system → Constantly "Downloading M of N in All Mail" or Inbox messages from Gmail IMAP account, slows system from redownloading message headers
Whiteboard: [gs][regression:TB17] → [gs][regression:TB18 bug 721316]

I've seen situations where you get folder(s) re-downloaded on startup that shouldn't need updating. However, I was abusing the system for test purposes. You end up with multiple storage files file-1, file-2, file-3 etc each time a new download update occurs. I believe tb thinks the folder has been deleted and a new one if present so it re-fetches headers and/or message bodies. I've only seen this when folder names contain unusual character such as slash.

Severity: major → normal

Now this I think is odd.

The activity manager advised for probably 15 minutes that it was bringing the all mail folder up to date and downloading 1 or 131,000 odd messages. Harddisk activity could be heard grinding away

Then it decided it was up to date and reported one message downloaded. This was despite the all mail folder having some 161,000 messages per the status bar.

There is something wrong in processing of GMail IMAP.

Blocks: 1098069

Still occasionally happens to me, randomly. It also forces global reindex of the file - so there is a double performance hit.

Flags: needinfo?(boomhauer)

I eventually solved my problem, reported in my 2 messages 2 years ago.
I found out that the constant re-downloading of the 3 affected folders was due to something wrong (some invalid character?) in one of the emails the folder contained.
Doing some try and fail experiments (based on the index of the last processed email before TB started re-downloading, which only gives an approximation), I was able to identify the "bad" email, and after deleting it server side using my mail provider's webmail, ... I was done, no more re-downloading ever since.
Unfortunately, I was unable to identify what was wrong with the 3 or 4 emails that obviously triggered the problem; to me they seemed perfectly fine, without any special characters or whatever -- actually, one of them was a booking hotel confirmation such as I had many others in the folder.

This is happening to me on TB 78.3.1 (on linux) with all my gmail accounts on two different computers. Does anyone have an idea what I could do to help get to the base of this problem? Whenever I start TB, I am told taht messages I have deleted can be purged from disk, saving 1 TB of data. Then every time I click on any of my gmail Inboxes, it starts Downloading message 1 of some ten thousand and I cannot open any emails until it is finished. This is maddening.

Stan, Do you allow "purged from disk" to occur? What is the exact message you see here. I'm thinking this is actually to "compact" folders that have messages that have been deleted but not sure.

Are you storing gmail message for offline use or are you just letting tb keep the headers locally. You can tell this by right-click on Inbox and select Properties and then Synchronization. Is "Select this folder for offline use" checked or not?

If you could record an IMAP:5 log it may help show why you are seeing the re-downloads. The instructions for recording the log are here: https://wiki.mozilla.org/MailNews:Logging. You can attach the log above. Just be sure that while recording the log that the problem actually occurs and explain in the comment exactly what you were doing while recording the log.

Hi gene smith,
Thanks for your response and sorry for taking so long. After seeing your response, I played around with the synchronization settings and clicked on "Keep messages on all folders on this computer", which filled up my hard drive over night. Luckily it could be undone by de-activating that option, so I keep messages on selected folders only. However, TB re-starts downloading all messages again episodically. Now I clicked on "Sent mail" of a different gmail account and it is "Downloading message 9345 of 345" with numbers rising. Something is seriously wrong. Often, when I want to view an email, it starts "Downloading message..." and take forever without showing anything. It's infuriating.

"Select this folder for offline use" is ticked for each of these folders.
I ran a log, which became very large in no time, so I cannot post it here. I have a a few different imap accounts, but see the below "HasNoChildren" pattern repeatedly only for gmail:

2020-11-28 10:17:33.945302 UTC - [(null) 2449859: IMAP]: D/IMAP SetConnectionStatus(0x0)
2020-11-28 10:17:33.945299 UTC - [(null) 2449859: IMAP]: D/IMAP SetConnectionStatus(0x0)
2020-11-28 10:17:33.945310 UTC - [(null) 2449859: IMAP]: D/IMAP ReadNextLine [rv=0x0 stream=0x7f3f2824c940 nb=51 needmore=0]
2020-11-28 10:17:33.945313 UTC - [(null) 2449859: IMAP]: D/IMAP ReadNextLine [rv=0x0 stream=0x7f3f22eed7c0 nb=56 needmore=0]
2020-11-28 10:17:33.945317 UTC - [(null) 2449859: IMAP]: I/IMAP 0x7f3f21ecb000:imap.gmail.com:A:CreateNewLineFromSocket: * XLIST (\HasNoChildren \Sent) "/" "[Gmail]/Sent Mail"

Is there something particular I should search for in the log file?

The "has no children" just means the folder, e.g., Sent, has no sub-folders under it. Not a problem.

Did it really fill up your drive completely? If so, you must have a lot of email or a just a small drive.
Anyhow, just unchecking the setting usually doesn't get rid of the messages downloaded. It just stops more messages from being saved. If you want to get rid of the message that have been downloaded after disabling local storage you have to either "repair" the folder (right click, click repair button) or, with tb shutdown, go into the profile under ImapMail and delete the appropriate mbox files (files with just a name and no .msf extension). Then on tb restart, the folders with missing mbox files, if configured for local storage, will be downloaded when accessed. You could also delete both files, e.g., Sent and Sent.msf and tb will bring in clean data for both headers (the .msf) and message bodies (only if configured for local storage) to the mbox file.

If repairing or deleting doesn't help another more drastic solution might be just to clean out all the files under your problem gmail account and let tb bring everything back from gmail server cleanly. Here's what I would do:

For the account(s) you want to fully repair, under server setting uncheck "Get new mail at startup" and "Check mail every X min". Also, see note below regarding global search.
For the problem account, go to "Account Settings" then "Synchronization and Storage"
Uncheck "Keep message for in all folders ... on this computer"
Click "Advanced" button and in the account you are trying to fix, choose specifically which folder to keep messages locally.
Note: All Mail is usually very large since it contains "all mail". Definitely a candidate to not store locally. You might also consider just not subscribing to "All Mail" so it is not accessed by tb at all
Click OK for the "advanced" locally stored messages selection
Shutdown tb
With file mgr or cmd line, go to your profile like ~/.thunderbird/akfafas.default/ImapMail/<account name>
Delete everything under <account name>
Note: This should all come back when you restart tb, but you can back this up before deleting if you are paranoid.
Restart tb and select/click Inbox on the account. The other folder will appears.
Only headers will download except for folders you have chosen to store locally.
Once all is downloaded (of course may take some time) re-enable the setting turned off in step 1.

An even more drastic fix would be to just set up a new profile (start tb with -p option I think) and let account mgr help you create the accounts again from scratch. But if doing this you want to go into the "advanced" setting of the wizard so the tb doesn't use the default setting of storing all messages locally.

Note: I have 2 gmail accounts I keep active for test purposes mostly and never see a problem.
Note also that global search is enabled by default, so when you download new stuff to tb, background indexing may occur for some time making tb use a lot of cpu. You may also see full messages downloaded but not saved to mbox files. So you might consider turning it off as part of step 1 above under General preferences Indexing uncheck "Enable global search and indexing".

Wow, thanks a lot for the detailed guide! I really appreciate your time. I will try what you said and report back. Yes, I figured that "All mail" is not what one might want to sync. :-) It's weird, after running over night, I got a "disk almost full" warning, so I removed the "All mail" setting and reverted to my original settings and when I next checked, the TB profile was of a normal size again (mind you, 20 GB). But I hadn't checked its size when I got the "disk almost full" message.
Anyway, will try to clean up one account at a time as you suggested and re-download and see if this helps.

Right-click on folders for which TB kept re-downloading messages, then Properties -> Repair Folder really did the trick! I did not need to delete the mailboxes and re-download after all. A very big THANK YOU again!

Ok, good to know that fixed it. I'll go ahead and close this as FIXED since you did fix the the problem. Usually we close this type of issue as INVALID but I do think it is a valid issue but determining the exact cause without direct access to your tb setup is very difficult.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED

Moving to WFM (FIXED will mess up uplift queries and such, so we only use it when there's a patch).

Resolution: FIXED → WORKSFORME
You need to log in before you can comment on or make changes to this bug.