False "Some folders ... cannot be compacted because there is not enough free disk space."

RESOLVED FIXED in Thunderbird 49.0

Status

MailNews Core
Backend
RESOLVED FIXED
2 years ago
a year ago

People

(Reporter: mike, Assigned: aceman)

Tracking

(Blocks: 1 bug, {regression})

Thunderbird 49.0
All
Windows
regression
Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird46 wontfix, thunderbird47 fixed, thunderbird48 fixed, thunderbird49 fixed, thunderbird_esr38+, thunderbird_esr4547+ fixed)

Details

(Whiteboard: [regression:TB38])

Attachments

(2 attachments, 2 obsolete attachments)

(Reporter)

Description

2 years ago
Created attachment 8622024 [details]
tb_38.0.1_compact_issue.PNG

User Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150525141253

Steps to reproduce:

Choose File -> Compact Folders

Version 38.0.1


Actual results:

Ran for a while, then complained that for some folders there was not enough disk space to compact, where it worked fine just now in 37.  On upgrade to 38.0.1, this issue happens.


Expected results:

There are 750GB of space on local drive, 2TB on server:  Should have completed compacting all folders with raising an issue about disk space
(Reporter)

Comment 1

2 years ago
IMAP folders - NOT synchronized locally.

Archive = by month and year

Comment 2

2 years ago
Did you mean upgrade from 31 (which was the previous major release)? And are you talking about some earlier bug report?
Keywords: regression
(Reporter)

Comment 3

2 years ago
Thunderbird auto-upgraded when I selected Help -> About to 38.0.1, and I thought it was on some version of 37, but i could be wrong.  I had seen another issue like this on an older release, when I searched the bugs database.
I've been seeing this for some time when doing manual compacts.  I rarely do compacts so I don't know when it started. 

Precisely "Some folders (e.g. 'bugday') cannot be compacted because there is not enough free disk space. Please delete some files and try again."   "Please delete some files and try again." was added by bug 854791 2015-02-17.
Blocks: 854791
Status: UNCONFIRMED → NEW
Component: Untriaged → Backend
Ever confirmed: true
Product: Thunderbird → MailNews Core
Summary: 38.0.1 - it's back - compact complains there is not enough disk space to compact, when there are hundreds of GB of disk space available → False "Some folders ... cannot be compacted because there is not enough free disk space."
Whiteboard: [regression:TB38]
(Assignee)

Comment 5

2 years ago
Wayne, do you get it also when trying to compact a single folder? What is the platform?

I think there must be something going on as we also have a similar bug 1143231 and bug 1160842.

It is suspicious that GetDiskSpaceAvailable() at least for Windows, can return a size of 0 while still reporting success (NS_OK). But that is a branch taken when all determination of free disk space failed.
http://mxr.mozilla.org/comm-central/source/mozilla/xpcom/io/nsLocalFileWin.cpp#2739

Wayne, I can make you a patch that somehow visualises whether GetDiskSpaceAvailable returned 0. Will you be able to test a try build with it?
Assignee: nobody → acelists
See Also: → bug 1160842, bug 1143231
(Assignee)

Comment 6

2 years ago
Maybe this (nsFolderCompactState::Compact) is another place where we need to use nsIFile->Clone() first due to the caching issues on Windows?
(Reporter)

Comment 7

2 years ago
No, if I right click on the so-called offending folder and select Compact, I do not get the issue.
I could try a try build.  :)
I do not see failure on individual foldlers, including the folder which gets named in the error message
(Reporter)

Comment 9

2 years ago
Right it only seems to happen when you select  from the File menu to compact folders.
(Assignee)

Comment 10

2 years ago
Created attachment 8622631 [details] [diff] [review]
debug experiment

Wayne, can you try making a try build with this and running the compact? Te debugging messages in the form "Compact: DiskFree - attempt X ..." should appear in the console (terminal, not Error console).
Flags: needinfo?(vseerror)
Windows Try build triggered: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=fe46d69e4dab
Blocks: 1160842
Duplicate of this bug: 1160842
(Reporter)

Comment 13

2 years ago
Any idea when this will be fixed?  Thanks.
(Assignee)

Comment 14

2 years ago
Wayne, are you running with the try build? Can you look for the messages in the console output?
So far unable to reproduce. I'll have to try some more. 
Here's another report https://support.mozilla.org/en-US/questions/1067922

mike, you might try it from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/archaeopteryx@coole-files.de-fe46d69e4dab/try-comm-central-win32/thunderbird-41.0a1.en-US.win32.installer.exe

start thunderbird at command prompt 
 thunderbird.exe -console
Flags: needinfo?(mike)
(Assignee)

Comment 16

2 years ago
(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #15)
> So far unable to reproduce. I'll have to try some more. 

Sorry, I forgot to note the patch could potentially affect the problem of this bug (positively).

So even if your compaction is successful, please see the console output and check the lines saying
Compact: DiskFree - attempt 1: xxx
Compact: DiskFree - attempt 2: yyy

If at attempt 1 xxx is 0 (zero) or similar but yyy at attempt 2 is the correct free space on disk, that is the event I am looking for. Please report such an event.
OK. I did another pass which had 3 with zero value - 1 near the beginning and two near the end, which are 
Compact: DiskFree - attempt 1: 54553001984
Compact: DiskFree - attempt 2: 54553001984, expected needed size = 0
Compact: DiskFree - attempt 1: 54553001984
Compact: DiskFree - attempt 2: 54553001984, expected needed size = 56678
Compact: DiskFree - attempt 1: 54552997888
Compact: DiskFree - attempt 2: 54552997888, expected needed size = 0
Compact: DiskFree - attempt 1: 54553001984
Compact: DiskFree - attempt 2: 54553001984, expected needed size = 94920
Compact: DiskFree - attempt 1: 54553001984
Compact: DiskFree - attempt 2: 54553001984, expected needed size = 213829

Question - why so many passes at  54553001984?
Flags: needinfo?(vseerror)
(Assignee)

Comment 18

2 years ago
(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #17)
> Compact: DiskFree - attempt 1: 54553001984
> Compact: DiskFree - attempt 2: 54553001984, expected needed size = 0

The 54553001984 is the free disk space in bytes (so ~54GB), it is no folder identifier.
So it could happen that compacting any of your folders actually did not free any space so the space indicator is at the same value. Except the one event of 54552997888.

But what I am looking for is a pattern like this (zero on attempt 1, not 2):
> Compact: DiskFree - attempt 1: 0
> Compact: DiskFree - attempt 2: 54553001984, expected needed size = 8375834
(Reporter)

Comment 19

2 years ago
I installed TB41: started from cmd, still get the error, here is output from console:

1434721176792   addons.xpi      WARN    Disabling foreign installed add-on {e2fda1a4-762b-4020-b5ad-a41df1933103} in app-global
Compact: DiskFree - attempt 1: 75446898688
Compact: DiskFree - attempt 2: 75446898688, expected needed size = 0
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 920881863
Flags: needinfo?(mike)
This was at the very end
Compact: DiskFree - attempt 1: 54152925184
Compact: DiskFree - attempt 2: 54152925184, expected needed size = 94968
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = -611763
Flags: needinfo?(acelists)
Compact on laptop in my wsm0 account.
And in this case I did get "cannot be compacted ... not enough disk space"

Compact: DiskFree - attempt 1: 53913874432
Compact: DiskFree - attempt 2: 53913874432, expected needed size = 183908
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0
Compact: DiskFree - attempt 1: 53913870336
Compact: DiskFree - attempt 2: 53913870336, expected needed size = 75681
Compact: DiskFree - attempt 1: 53913862144
Compact: DiskFree - attempt 2: 53913862144, expected needed size = 2789452
Compact: DiskFree - attempt 1: 53913845760
Compact: DiskFree - attempt 2: 53913845760, expected needed size = 4426695
Compact: DiskFree - attempt 1: 53913849856
Compact: DiskFree - attempt 2: 53913849856, expected needed size = 15646
Compact: DiskFree - attempt 1: 53913849856
Compact: DiskFree - attempt 2: 53913849856, expected needed size = 14782
Compact: DiskFree - attempt 1: 53913849856
Compact: DiskFree - attempt 2: 53913849856, expected needed size = 2823187
Compact: DiskFree - attempt 1: 53913837568
Compact: DiskFree - attempt 2: 53913837568, expected needed size = 455903
Compact: DiskFree - attempt 1: 53913309184
Compact: DiskFree - attempt 2: 53913309184, expected needed size = 8765901
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0
Compact: DiskFree - attempt 1: 53913309184
Compact: DiskFree - attempt 2: 53913309184, expected needed size = 7864330
Compact: DiskFree - attempt 1: 53913747456
Compact: DiskFree - attempt 2: 53913747456, expected needed size = 908094
Compact: DiskFree - attempt 1: 53913755648
Compact: DiskFree - attempt 2: 53913755648, expected needed size = 345122914
Compact: DiskFree - attempt 1: 53926096896
Compact: DiskFree - attempt 2: 53926096896, expected needed size = 16308585
Compact: DiskFree - attempt 1: 53926035456
Compact: DiskFree - attempt 2: 53926035456, expected needed size = 63481
Compact: DiskFree - attempt 1: 53926031360
Compact: DiskFree - attempt 2: 53926031360, expected needed size = 38417
Compact: DiskFree - attempt 1: 53926027264
Compact: DiskFree - attempt 2: 53926027264, expected needed size = 534449
Compact: DiskFree - attempt 1: 53926027264
Compact: DiskFree - attempt 2: 53926027264, expected needed size = 1313496
Compact: DiskFree - attempt 1: 53926031360
Compact: DiskFree - attempt 2: 53926031360, expected needed size = 253152
Compact: DiskFree - attempt 1: 53926027264
Compact: DiskFree - attempt 2: 53926027264, expected needed size = 613868
Compact: DiskFree - attempt 1: 53925949440
Compact: DiskFree - attempt 2: 53925949440, expected needed size = 283496
Compact: DiskFree - attempt 1: 53925945344
Compact: DiskFree - attempt 2: 53925945344, expected needed size = 10106449
Compact: DiskFree - attempt 1: 53925810176
Compact: DiskFree - attempt 2: 53925810176, expected needed size = 8840
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 3245249
After responding to the dialog, a few more...

ompact: DiskFree - attempt 1: 0
ompact: DiskFree - attempt 2: 0, expected needed size = 429324
ompact: DiskFree - attempt 1: 53924065280
ompact: DiskFree - attempt 2: 53924065280, expected needed size = 285749
ompact: DiskFree - attempt 1: 53924061184
ompact: DiskFree - attempt 2: 53924061184, expected needed size = 3925
ompact: DiskFree - attempt 1: 0
ompact: DiskFree - attempt 2: 0, expected needed size = 0

Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0
Compact: DiskFree - attempt 1: 53923991552
Compact: DiskFree - attempt 2: 53923991552, expected needed size = 841774
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0
Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0

Compact: DiskFree - attempt 1: 0
Compact: DiskFree - attempt 2: 0, expected needed size = 0
Compact: DiskFree - attempt 1: 53928534016
Compact: DiskFree - attempt 2: 53928534016, expected needed size = -8385995
(Assignee)

Comment 23

2 years ago
Thanks for the findings!

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #21)
> Compact: DiskFree - attempt 1: 53913874432
> Compact: DiskFree - attempt 2: 53913874432, expected needed size = 183908
> Compact: DiskFree - attempt 1: 0
> Compact: DiskFree - attempt 2: 0, expected needed size = 0
So this means GetDiskSpaceAvailable() lies to us and silently returns 0 even if that is not the true free space size. May be some race condition or that getting the space of a directory that is in use is blocked by the OS. But this shouldn't be the case as it is not every time.

We'd need a Windows expert to make the return value of GetDiskFreeSpaceExW() output to the log file.

And it means nsIFile->Clone() does not help here.

>Compact: DiskFree - attempt 1: 53928534016
>Compact: DiskFree - attempt 2: 53928534016, expected needed size = -8385995
I'm curious about the negative expected size here. That should not happen. I'll prepare more debugging output.
Flags: needinfo?(acelists)
(Assignee)

Comment 24

2 years ago
Created attachment 8625063 [details] [diff] [review]
debug experiment 2

This is further debugging out. Wayne, please make a try build and run with this version.
Attachment #8622631 - Attachment is obsolete: true
Flags: needinfo?(vseerror)
(Assignee)

Comment 25

2 years ago
Aryx, please pus to try server to get us a new Windows binary. Thanks.
Flags: needinfo?(archaeopteryx)
(Reporter)

Comment 26

2 years ago
Any progress?

Updated

2 years ago
tracking-thunderbird_esr38: --- → +
Pushed to Try, but possible that the builds will be busted because of the state of the tree: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=b35cd8e71fdc
Flags: needinfo?(archaeopteryx)
New Try push: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=11cfcc330820
(In reply to Aryx [:archaeopteryx][:aryx] from comment #28)
> New Try push:
> https://treeherder.mozilla.org/#/jobs?repo=try-comm-
> central&revision=11cfcc330820

It errored out again.  Boo Hiss
Flags: needinfo?(vseerror) → needinfo?(acelists)
(Assignee)

Comment 30

2 years ago
Does that failure also happen on trunk? It seems to be unrelated to the patch, it is in import component.
Flags: needinfo?(acelists)

Comment 31

2 years ago
Trunk seems "fine" atm.

Comment 32

2 years ago
I'm seeing this same issue, in both Thunderbird 38.0.1 and 38.1.0 on Windows 7, and not in previous versions with the same profile.

I'm willing to help debug if someone can tell me how to download one of these "try" builds you talk about. In my case, I'm using an IMAP server (MS Exchange), but I have "Keep messages for this account on this computer" unticked so there is little disk space used by Thunderbird, just the .msf files and zero byte folder-name files. I have gigabytes of free disk space.

I'm just adding the following paragraph here as keywords for other people searching for this issue. This is the same error text but in British English (different spelling of "disk"). I searched for the British English error message and couldn't find any hits, but hopefully this bug report will now show up in Internet searches:

"Some folders ... cannot be compacted because there is not enough free disc space".
(Assignee)

Comment 33

2 years ago
So we should push this to try server again when building gets un-busted.
Flags: needinfo?(aryx.bugmail)
New Try push: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=3b956fa1e19c
Flags: needinfo?(aryx.bugmail)
(Assignee)

Comment 35

2 years ago
It seems this build succeeded. Wayne, can you try it out?
Flags: needinfo?(vseerror)
emailed log to aceman
Flags: needinfo?(vseerror)

Comment 37

2 years ago
Any update? Because this issue still exists in 38.3.0

https://www.thunderbird-mail.de/thread/71804-komprimieren-nicht-mehr-möglich/?postID=395380
https://www.thunderbird-mail.de/thread/70648-fehlermeldung-komprimierung/?postID=388052#post388052
https://www.thunderbird-mail.de/thread/70648-fehlermeldung-komprimierung/?postID=387988#post387988

Comment 38

2 years ago
Since some time, I have this "Some folders (e.g. XXXX) cannot be compacted because there is not enough free disk space." error. However, I never had a single disk space problem... In my case, the message always began to appear immediately after renaming a IMAP folder (generally containing special characters like "&" or "é", however I'm not sure this is important). Monitoring disk accesses using Process Monitor, I've noticed that the error message appears when Thunderbird tries to access the empty file named as the folder (e.g. $profile/ImapMail/server/FolderName), whereas the corresponding .msf file does exist. If I just manually create the empty file named after the folder, the error immediately disappears.
(Assignee)

Comment 39

2 years ago
Why are those files empty? Don't they represent folders with messages? Are they not populated (synced from server) after renaming? Also, you run "compact all" and that hits also these empty folders (which maybe should be skipped as there is nothing to compact) ?

Comment 40

2 years ago
(In reply to :aceman from comment #39)
The files are empty because messages are only stored server-side ("Keep messages for this account on this computer" is unchecked, and folders are not marked to download for offline use). So the empty files are normal... the only thing the "Compact All" should do is to actually compact ("expunge") those folders on the server (removing messages marked for deletion).

It look like this bug is a regression, but I'm not sure about which version of Thunderbird it first appeared in. I was doing some testing (all with "Keep messages for this account on this computer" unchecked):

* creating a new (empty) folder and compacting all: works fine, no error message, .msf file is created at folder creation, the empty folder file is created when compacting (after all folders have been compacted on the server)
* creating a new folder, copying a message into it, and compacting all: triggers the bug! This stops the folder compacting operation and the empty folder file is never creat

Comment 41

2 years ago
(In reply to :aceman from comment #39, replacing comment #40 submitted by error)
The files are empty because messages are only stored server-side ("Keep messages for this account on this computer" is unchecked, and folders are not marked to download for offline use). So the empty files are normal... the only thing the "Compact All" should do is to actually compact ("expunge") those folders on the server (removing messages marked for deletion).

It look like this bug is a regression, but I'm not sure about which version of Thunderbird it first appeared in. I was doing some testing (all with "Keep messages for this account on this computer" unchecked):

* creating a new (empty) folder and compacting all: works fine, no error message, .msf file is created at folder creation, the empty folder file is created when compacting (after all folders have been compacted on the server)
* creating a new folder, copying a message into it, and compacting all: triggers the bug! This stops the folder compacting operation and the empty folder file is never created.
* creating a new folder, copying a message into it, selecting the folder for offline use and clicking "Download now", and compacting all: the .msf AND the folder files are created, no error when compacting all.

All tried with folders named "NewFolder1" or similar, so no special characters involved.
(Reporter)

Comment 42

2 years ago
This is correct and accurate as to how we work.:

The files are empty because messages are only stored server-side ("Keep messages for this account on this computer" is unchecked, and folders are not marked to download for offline use). So the empty files are normal... the only thing the "Compact All" should do is to actually compact ("expunge") those folders on the server (removing messages marked for deletion).

Comment 43

2 years ago
FWIW, I have seen this situation for a while as well. This is on various Windows 7 x64 implementations and with plenty of space available >100 GB in some cases.

Right now, the folder mentioned is 'Junk', which has all of 9 files in it, none with attachments, none larger than 62.2KB and more than 100 GB space available. After "Emptying Junk", though, the compact succeeds.

Progressing on to the next account (for a manual compact), the compact fails in the folder 'Sent' "...because another operation is in progress." This account is not being used. After clicking OK, the folder 'All Mail' "cannot be compacted because another operation...". Re-issue the command and it succeeds.

Given that Thunderbird is often used with multiple Accounts and it is an account specific tool, when it is "Done Compacting", it would be proper to have it say which account it is done compacting, e.g. Done Compacting Account_Name.

Comment 44

a year ago
I'd like to add a "me too" to this bug.  I am currently running Thunderbird 38.7.0 on Windows 7 x64.  The mail folder triggering the problem contains 35 messages and takes up 181 KB.  This is happening when I compact my GMail folder which contains the problem folder and between 35 and 40 other folders.  If I compact just the problem, it succeeds.

And, just to confirm it is a Windows problem, my 38.6.0 installation on Linux does not exhibit this behavior.
(Assignee)

Comment 45

a year ago
This happening on newly created folders for IMAP may be a bit of a clue, some size variables could be initialized thus producing bogus results in the expected-size-needed calculations (e.g. the negative needed size seen in some of the logs).

Wayne, have you also been trying this on IMAP?

I'll try to make a new try build with recent TB with patch "experiment 2" above where the output is more verbose and shows what TB thinks is the currend database size (the .msf file).
OS: Unspecified → Windows
Hardware: Unspecified → All
(Assignee)

Comment 46

a year ago
I've pushed the patch to try server to create new try builds:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=ab40c562fe25a036b145fe3c639db8d8ad2ecf0c
Flags: needinfo?(vseerror)
(Assignee)

Comment 47

a year ago
It seems the try builds are successful (at least for Windows). Wayne, Aryx, can you try it out?

Comment 48

a year ago
I ran the try build from comment 46 and am still seeing the problem.  Here's the console output:

Compact: attempting compact of folder C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.com\Cygwin
Compact: attempt 1: free=0
Compact: attempt 2: free=0, expected needed size = 224681 (current size=203362,expunged=0,realDB=21319,expDB=21319)
++DOCSHELL 09ACF400 == 12 [pid = 3528] [id = 14]
++DOMWINDOW == 23 (09AD0000) [pid = 3528] [serial = 38] [outer = 00000000]
[3528] WARNING: No inner window available!: file c:/builds/moz2_slave/tb-try-c-c
en-w32-d-00000000000/build/mozilla/dom/base/nsGlobalWindow.cpp, line 9776
++DOMWINDOW == 24 (0A02EC00) [pid = 3528] [serial = 39] [outer = 09AD0000]

Let me know if there's something else I can provide.
(Assignee)

Comment 49

a year ago
For IMAP folders, I think they must be marked "for offline use" otherwise the compaction (expunge) is done on the server so takes a completely different path and should not check the free disk space locally.
Can anybody confirm this?

Mike, you say your folders are "not synchronized locally". Does that mean they are NOT marked for offline use? Are you sure about all of them? Can you check that folder that is producing the error?

Also, can anybody see this on POP3?
Flags: needinfo?(mike)
(Assignee)

Comment 50

a year ago
(In reply to Paul Keusemann from comment #48)
> Compact: attempting compact of folder
> C:\Users\Thunderbird\Profiles\xxxxx.paulkeus\ImapMail\imap.googlemail.
> com\Cygwin
> Compact: attempt 1: free=0
> Compact: attempt 2: free=0, expected needed size = 224681 (current
> size=203362,expunged=0,realDB=21319,expDB=21319)

OK, this is again the free space of 0, but at least no negative needed size.

> Let me know if there's something else I can provide.

Yes, please observe if it is always the same folder having the problem and when it hits, please look in the filesystem whether C:\Users\Thunderbird\Profiles\xxxxx.paulkeus\ImapMail\imap.googlemail.com\foldername and C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.com\foldername.msf does exist.

Now I look at it, are you running Thunderbird under a separate Thunderbird user? Or is that your username? How comes the profile is directly in C:\Users\Thunderbird and not in e.g. C:\Users\yourname\Applicationa Data\Thunderbird ?
(Assignee)

Comment 51

a year ago
So looking at the Windows version of GetDiskSpaceAvailable at
http://mxr.mozilla.org/comm-central/source/mozilla/xpcom/io/nsLocalFileWin.cpp#2712, the function may return 0 also in case when GetDiskFreeSpaceExW failed (not only if it lies and returns 0). We may be sending it some incorrect arguments to that it fails. I can make a debugging patch for that function to see what is going on.

Aryx, would pushing a m-c patch to c-c try server work?
Flags: needinfo?(aryx.bugmail)

Comment 52

a year ago
(In reply to :aceman from comment #50)
> (In reply to Paul Keusemann from comment #48)
> > Compact: attempting compact of folder
> > C:\Users\Thunderbird\Profiles\xxxxx.paulkeus\ImapMail\imap.googlemail.
> > com\Cygwin
> > Compact: attempt 1: free=0
> > Compact: attempt 2: free=0, expected needed size = 224681 (current
> > size=203362,expunged=0,realDB=21319,expDB=21319)
> 
> OK, this is again the free space of 0, but at least no negative needed size.
> 
> > Let me know if there's something else I can provide.
> 
> Yes, please observe if it is always the same folder having the problem and

Yes, it is always the same folder: Cygwin.

> when it hits, please look in the filesystem whether
> C:\Users\Thunderbird\Profiles\xxxxx.paulkeus\ImapMail\imap.googlemail.
> com\foldername and
> C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.
> com\foldername.msf does exist.

The C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.com\Cygwin.msf exists but C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.com\Cygwin does not.

Now here's something else that interesting or maybe just annoying.  Since I ran the try build yesterday, I am having to re-enable remote content for all of my trusted senders.

> 
> Now I look at it, are you running Thunderbird under a separate Thunderbird
> user? Or is that your username? How comes the profile is directly in
> C:\Users\Thunderbird and not in e.g. C:\Users\yourname\Applicationa
> Data\Thunderbird ?

No.  Thunderbird is configured to store profiles in C:\Users\Thunderbird.
(Assignee)

Comment 53

a year ago
(In reply to Paul Keusemann from comment #52)
> The
> C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.
> com\Cygwin.msf exists but
> C:\Users\Thunderbird\Profiles\xxxx.paulkeus\ImapMail\imap.googlemail.
> com\Cygwin does not.

That is interesting and useful. I'll see what happens when we try to get the disk space for a non-existent path.
 
> Now here's something else that interesting or maybe just annoying.  Since I
> ran the try build yesterday, I am having to re-enable remote content for all
> of my trusted senders.

Yes, that trust whitelist has changed from senders (emails) to websites (URLs), see bug 1193200. That should be more secure (faking sender is trivial, faking URL is not).
(Assignee)

Comment 54

a year ago
I've made a new try build at https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=cb65548788e2be09d8ccd36ea52a9fd4603f8ed8
(Assignee)

Comment 55

a year ago
No, that one is buggy. I'll make a new one.
(In reply to :aceman from comment #51)
> Aryx, would pushing a m-c patch to c-c try server work?
Sorry for the late reply. Yes, it's possible: https://wiki.mozilla.org/ReleaseEngineering/ThunderbirdTryServer#Pushing_mozilla-central_patches
Flags: needinfo?(aryx.bugmail)

Comment 57

a year ago
I am also experiencing the (false) insufficient disk space when compacting error.

:aceman, on 1-April reported that the T-bird profiles were configured to be stored in a non-default location. While my profiles are stored in the AppData tree, my "Local Directory", under Account Settings / Server Settings is a _non-standard_ location. This is the structure that is being compacted.

Also, while there is only one user, there are multiple Accounts, but not all accounts have the problem.

Don't know if it's relevent, but thought the data point might be useful.

Comment 58

a year ago
(In reply to Jim Figlik from comment #57)
> I am also experiencing the (false) insufficient disk space when compacting
> error.
> 
> :aceman, on 1-April reported that the T-bird profiles were configured to be
> stored in a non-default location. While my profiles are stored in the
> AppData tree, my "Local Directory", under Account Settings / Server Settings
> is a _non-standard_ location. This is the structure that is being compacted.
> 
> Also, while there is only one user, there are multiple Accounts, but not all
> accounts have the problem.
> 
> Don't know if it's relevent, but thought the data point might be useful.

The above occurs with (stock) v38.7.2.

Another observation is while the original error came up with the usual folder, it is followed by a subsequent Alert: "The folder 'xxx' cannot be compacted because another operation is in progress. Please try again later."

This is an IMAP configuration, with no special IMAP folder configurations, non-google servers.
(In reply to :aceman from comment #47)
> It seems the try builds are successful (at least for Windows). Wayne, Aryx,
> can you try it out?

sorry for missing this. let me know when the next one is available. unless you need my feedback on the earlier one.
Flags: needinfo?(vseerror)
(Assignee)

Comment 60

a year ago
Looks like this build succeeded: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=ef2df21bb1684eac70a80fa61d0c3432614d7f48
Flags: needinfo?(vseerror)

Comment 61

a year ago
(In reply to :aceman from comment #60)
> Looks like this build succeeded:
> https://treeherder.mozilla.org/#/jobs?repo=try-comm-
> central&revision=ef2df21bb1684eac70a80fa61d0c3432614d7f48

I am no longer getting the compacting error with this build.

I think this:

Compact: attempting compact of folder C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.com\Debian

is probably the confirmation you were looking for.
(Assignee)

Comment 62

a year ago
(In reply to Paul Keusemann from comment #61)
> I think this:
> 
> Compact: attempting compact of folder
> C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.
> com\Debian
> 
> is probably the confirmation you were looking for.

Is there no further debug output? I'd like to know why it skipped the compact.

Comment 63

a year ago
(In reply to :aceman from comment #62)
> (In reply to Paul Keusemann from comment #61)
> > I think this:
> > 
> > Compact: attempting compact of folder
> > C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.
> > com\Debian
> > 
> > is probably the confirmation you were looking for.
> 
> Is there no further debug output? I'd like to know why it skipped the
> compact.

Copy error behind the keyboard, er mouse.  Sorry about that, here it is again:

Compact: attempting compact of folder C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.com\Debian
Compact not needed: folder backing file does not exist yet.

Let me know if you would like to see anything else from the console output.
(Assignee)

Comment 64

a year ago
(In reply to Paul Keusemann from comment #63)
> Compact: attempting compact of folder
> C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.
> com\Debian
> Compact not needed: folder backing file does not exist yet.
> 
> Let me know if you would like to see anything else from the console output.

Perfect, thanks. This time it is a Debian folder, previously you mentioned a Cygwin folder. Sure, both folders could have the same properties thus the problem (without the patch).

So can you just confirm this info about your folders:
- you used a Gmail account
- those folders are NOT set for offline use (in folder properties inside Thunderbird)
- those 2 folders are NOT empty (contain some mesages on the server). This is seen e.g. in your comment 52 with 203362 bytes of used size)
- those folders do have the corresponding folder.msf file created on your local disk and it seems used (has a size above 1KB).

Comment 65

a year ago
(In reply to :aceman from comment #64)
> (In reply to Paul Keusemann from comment #63)
> > Compact: attempting compact of folder
> > C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.
> > com\Debian
> > Compact not needed: folder backing file does not exist yet.
> > 
> > Let me know if you would like to see anything else from the console output.
> 
> Perfect, thanks. This time it is a Debian folder, previously you mentioned a
> Cygwin folder. Sure, both folders could have the same properties thus the
> problem (without the patch).
> 

I created an empty Cygwin file in the imap.googlemail.com directory to see if that resolved the compaction error.  It did, for Cygwin but the Debian mail folder caused the error.  I just removed the (still empty) Cygwin file from the imap.googlemail.com directory and re-ran compaction on my Gmail account and got the compaction error on the Cygwin mail folder.  I also discovered something else that might be interesting.  Right clicking on the Cygwin (or Debian) mail folder and selecting compact from the pop-up menu does not cause the compaction error.  I have to select Compact Folders from the File drop-down menu in order for the error to occur.

> So can you just confirm this info about your folders:
> - you used a Gmail account

Correct.

> - those folders are NOT set for offline use (in folder properties inside
> Thunderbird)

Correct.

> - those 2 folders are NOT empty (contain some mesages on the server). This
> is seen e.g. in your comment 52 with 203362 bytes of used size)

Correct.

> - those folders do have the corresponding folder.msf file created on your
> local disk and it seems used (has a size above 1KB).

Correct.

Comment 66

a year ago
(In reply to Paul Keusemann from comment #65)
> (In reply to :aceman from comment #64)
> > (In reply to Paul Keusemann from comment #63)
> > > Compact: attempting compact of folder
> > > C:\Users\Thunderbird\Profiles\bcpi6v01.paulkeus\ImapMail\imap.googlemail.
> > > com\Debian
> > > Compact not needed: folder backing file does not exist yet.
> > > 
> > > Let me know if you would like to see anything else from the console output.
> > 
> > Perfect, thanks. This time it is a Debian folder, previously you mentioned a
> > Cygwin folder. Sure, both folders could have the same properties thus the
> > problem (without the patch).
> > 
> 
> I created an empty Cygwin file in the imap.googlemail.com directory to see
> if that resolved the compaction error.  It did, for Cygwin but the Debian
> mail folder caused the error.  I just removed the (still empty) Cygwin file
> from the imap.googlemail.com directory and re-ran compaction on my Gmail
> account and got the compaction error on the Cygwin mail folder.  I also
> discovered something else that might be interesting.  Right clicking on the
> Cygwin (or Debian) mail folder and selecting compact from the pop-up menu
> does not cause the compaction error.  I have to select Compact Folders from
> the File drop-down menu in order for the error to occur.

I forgot to mention that I switched back to 38.7.2 to recreate the error.

> 
> > So can you just confirm this info about your folders:
> > - you used a Gmail account
> 
> Correct.
> 
> > - those folders are NOT set for offline use (in folder properties inside
> > Thunderbird)
> 
> Correct.
> 
> > - those 2 folders are NOT empty (contain some mesages on the server). This
> > is seen e.g. in your comment 52 with 203362 bytes of used size)
> 
> Correct.
> 
> > - those folders do have the corresponding folder.msf file created on your
> > local disk and it seems used (has a size above 1KB).
> 
> Correct.
(Assignee)

Comment 67

a year ago
(In reply to Paul Keusemann from comment #65)
> I also
> discovered something else that might be interesting.  Right clicking on the
> Cygwin (or Debian) mail folder and selecting compact from the pop-up menu
> does not cause the compaction error.  I have to select Compact Folders from
> the File drop-down menu in order for the error to occur.

Yes, those menuitems take a slightly different code paths. Also the compaction can be done on server (expunge msgs marked deleted on the server) or in the local file. The menuitems sometimes differ in which version they call. Also, they have a different criteria when to run compact on a folder (e.g. only if size of deleted messages is non-zero). And it is the local compaction that caused the error for you as there was no local file to compact.

I don't want to search why those items differ. It is not needed for the bug here.

But I at least add the condition to only compact on expunged bytes > 0 (size of deleted message) into the backend (not just frontend) so it is always evaluated.

Now let's wait for others to test the try build. We need to see if they had the same problem as you (missing storage file, but which is normal for online IMAP) or there is something else to cover.
status-thunderbird_esr45: --- → affected
tracking-thunderbird_esr45: --- → ?
Flags: needinfo?(vseerror)
Flags: needinfo?(vseerror)
I can't seem to reproduce
Flags: needinfo?(vseerror)

Comment 69

a year ago
I had it occur today, W7 x64 v45.0 when clicking from the File Menu. It occurs consistently on the 54th folder (of maybe 130 or so).

If I go to the folder itself, right-click on the err-ed folder, Compact appears to run without error-ing out. Ditto from the immediate parent folder.

I moved a number of messages from a log file to "Local Folders" and had the d'box pop up asking if I wanted to "compact now". I didn't think much of it at the time, but I do believe that this ran without error as well, although it is in a different account. My assumption is that "compact" is running across all accounts & "Local Folders" given that I moved messages from Account #5 into "Local Folders", (virtually) Account #6. I'm seeing the problem on "Account #1", folder #54.
(Assignee)

Comment 70

a year ago
Created attachment 8744656 [details] [diff] [review]
patch v3

So thanks to Paul and Wayne for testing the try build. It seems this patch covered their cases and allows compact to finish.
Attachment #8625063 - Attachment is obsolete: true
Attachment #8744656 - Flags: review?(rkent)
aceman, 
https://support.mozilla.org/en-US/questions/1117173 might be another vector for this issue
(Assignee)

Comment 72

a year ago
Citing the report:
"In the working profiles (other accounts) there also was two files named "Sent". One with the extension .msf and one with no extension. In the profile with the problem the one without extension was missing. I created it by changing name of another file named "Skickat" (which is Sent translated to Swedish...) and restarted Thunderbird. And then It works..."

The missing file backing the folder locally is the problem being fixed by the current patch. I only expected it to happen only on IMAP, but maybe it can happen on POP3 too. It looks like after the upgrade TB could decide to use a new folder as Sent (not the translated Sent). So until no message was saved to the new folder, ther could be no backing file for it.

Comment 73

a year ago
I also still have this problem on TB 45.0. I have no offline storage of mails, everything is on the IMAP server, and the error message appears for every folder where the (empty) folder file near the .msf file is missing. Creating those files solves the problem all the time.

Comment 74

a year ago
Precision (sorry for the double comment): not that any of those mailbox files do disappear, however they are not created if mails are not downloaded (e.g. new folder appeared on the IMAP server, or folder move/rename).
http://forums.mozillazine.org/viewtopic.php?f=39&t=2982131 
can be added to the list
https://support.mozilla.org/en-US/questions/1067922
https://support.mozilla.org/en-US/questions/1117173
https://support.mozilla.org/en-US/questions/1091792
status-thunderbird48: --- → affected
status-thunderbird49: --- → affected
Blocks: 1142468
Aceman, 
The patch should probably ride some of the train. But definitely would like to see this uplifted to beta (maybe in time for 45.2?) and eventually to 45 as implied by my tracking 45 request. I mention this because I suspect it may be more common than we think. I was with a local user today who never reported the issue but sees it on every compact, and was concerned that something was wrong with her machine.
(Assignee)

Comment 77

a year ago
I'd be happy to ride it, just get me a reviewer :)
Yes, the problem affects all users of IMAP that have at least one folder that is NOT for marked offline use since its creation (no msg ever hit the offline storage file). That may be a large percentage, or not. I don't have numbers.
Comment on attachment 8744656 [details] [diff] [review]
patch v3

Review of attachment 8744656 [details] [diff] [review]:
-----------------------------------------------------------------

Everything that I checked seems to make sense. At this point I think the best thing to do is to land it and give it some exposure. Maybe for 45.2? But not for 45.1.2
Attachment #8744656 - Flags: review?(rkent)
Attachment #8744656 - Flags: review+
Attachment #8744656 - Flags: approval-comm-esr45?
Attachment #8744656 - Flags: approval-comm-beta?
Attachment #8744656 - Flags: approval-comm-aurora?
(Assignee)

Comment 79

a year ago
Sure, 45.2 is fine, it needs some baking.
Status: NEW → ASSIGNED
Keywords: checkin-needed
(Assignee)

Comment 80

a year ago
https://hg.mozilla.org/comm-central/rev/6f156e84d899ff6057c883fae21b3b71970cd0c7
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
Flags: needinfo?(mike)
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 49.0

Comment 81

a year ago
Comment on attachment 8744656 [details] [diff] [review]
patch v3

Aurora (TB 48):
https://hg.mozilla.org/releases/comm-aurora/rev/c238dd759028
Attachment #8744656 - Flags: approval-comm-aurora? → approval-comm-aurora+

Updated

a year ago
status-thunderbird46: --- → wontfix
status-thunderbird47: --- → affected
status-thunderbird48: affected → fixed
status-thunderbird49: affected → fixed
Comment on attachment 8744656 [details] [diff] [review]
patch v3

http://hg.mozilla.org/releases/comm-beta/rev/0669065eeff4
Attachment #8744656 - Flags: approval-comm-beta? → approval-comm-beta+

Updated

a year ago
status-thunderbird47: affected → fixed
Comment on attachment 8744656 [details] [diff] [review]
patch v3

http://hg.mozilla.org/releases/comm-esr45/rev/7c0ef68ec30b
Attachment #8744656 - Flags: approval-comm-esr45? → approval-comm-esr45+

Updated

a year ago
status-thunderbird_esr45: affected → fixed
tracking-thunderbird_esr45: ? → 47+
You need to log in before you can comment on or make changes to this bug.