Thunderbird is limited to 10000 temp files, stops sending emails if nsmail-9999.tmp exists, and fails to open attachment if <attachment_filename>-9999.attachment_extension exists
Categories
(Thunderbird :: General, defect)
Tracking
(thunderbird_esr115? fixed, thunderbird118 fixed)
People
(Reporter: demon, Assigned: welpy-cw)
References
(Blocks 1 open bug)
Details
(Keywords: ux-error-prevention, Whiteboard: [has pointers to code])
Attachments
(1 file, 2 obsolete files)
48 bytes,
text/x-phabricator-request
|
wsmwk
:
approval-comm-beta+
wsmwk
:
approval-comm-esr115+
|
Details | Review |
Reporter | ||
Comment 1•13 years ago
|
||
Reporter | ||
Comment 2•13 years ago
|
||
Comment 3•13 years ago
|
||
Comment 4•13 years ago
|
||
Comment 5•13 years ago
|
||
Reporter | ||
Comment 6•13 years ago
|
||
Comment 7•13 years ago
|
||
Comment 8•13 years ago
|
||
Comment 10•13 years ago
|
||
Comment 11•13 years ago
|
||
Comment 12•13 years ago
|
||
Updated•13 years ago
|
Comment 15•12 years ago
|
||
Comment 17•12 years ago
|
||
Updated•11 years ago
|
Comment 18•11 years ago
|
||
Comment 19•11 years ago
|
||
Comment 21•4 years ago
|
||
(In reply to :aceman from comment #18)
Jonathan, do you still have 9999 nsmail temp files existing?
I think recently there were some fixes to not leave stray temp files around.
I can still reproduce sticking "nsqmail-*.tmp" files by placing mails in the Send-Later folder and then calling Send Unsend Messages while the server is down.
https://searchfox.org/comm-central/rev/68ba4decd0f0b2e4081c2be892fc12a5e8f44784/mailnews/compose/src/nsMsgSendLater.cpp#557
Comment 22•4 years ago
|
||
I just had this issue in 2021, "Assembling message" then "sending of the message failed" suddenly. After reading this thread, my nsemail* files were 9999 and nsmail was in the 3000s. Deleted them all, problem solved. Thanks and yes please write proper error message when this happens. Much appreciated!
Updated•3 years ago
|
Updated•2 years ago
|
Comment 23•2 years ago
|
||
The severity field for this bug is relatively low, S3. However, the bug has 4 duplicates.
:nika, could you consider increasing the bug severity?
For more information, please visit auto_nag documentation.
Comment 24•2 years ago
|
||
The 10,000 temp file limit still exists. It wouldn't be that hard to increase it, but if there's some kind of temp file leak in Thunderbird that's only going to delay the issue for people.
Comment 25•2 years ago
|
||
I can bump the limit to 99,999 and maybe that will help...
Comment 26•2 years ago
|
||
Apparently Thunderbird hits the current limit. Maybe increasing by a factor of 10 will help.
I'm not increasing it further so I don't have to worry about file length issues.
Comment 27•2 years ago
|
||
The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.
Comment 28•2 years ago
|
||
From a quick scan I don't see code in Thunderbird which handles deleting the temporary nsqmail.temp
files created here: https://searchfox.org/comm-central/rev/ddb726dfb927d29debb26fb4b7cb8508b0c38c97/mailnews/compose/src/nsMsgSendLater.cpp#556, which might be a cause of the temporary file build-up (at least for nsqmail*.temp
). I expect there are other things like that in thunderbird for the other types of temporary files which are leaking.
We might still do something on the xpcom side to allow large numbers of temporary files, but this bug specifically seems like it is probably thunderbird-specific.
Updated•2 years ago
|
Updated•2 years ago
|
Comment 29•1 years ago
|
||
I found code to clean up some temporary files when Thunderbird starts up.
https://searchfox.org/comm-central/source/mailnews/compose/src/nsMsgComposeService.cpp#121
nsresult nsMsgComposeService::Init() {
nsresult rv = NS_OK;
Reset();
AddGlobalHtmlDomains();
// Since the compose service should only be initialized once, we can
// be pretty sure there aren't any existing compose windows open.
MsgCleanupTempFiles("nsmail", "tmp");
MsgCleanupTempFiles("nsemail", "html");
MsgCleanupTempFiles("nscopy", "tmp");
return rv;
}
I think the following code should be added.
MsgCleanupTempFiles("nsemail", "eml");
MsgCleanupTempFiles("nsqmail", "tmp");
Comment 30•1 years ago
|
||
nsmail-xxxx.tmp
is cleaned up on Thunderbird startup. (since Thunderbird 16)
It is nsemail-xxxx.eml
that is causing the sending to stop with NS_ERROR_FILE_TOO_BIG error, and I don't think it is appropriate to include nsmail-9999.tmp
in this bug summary.
Comment 31•1 year ago
|
||
I checked the behavior of file removing by MsgCleanupTempFiles().
I created the following files in the temp folder and then started Thunderbird.
nscopy.tmp, nscopy-1.tmp, nscopy-2.tmp, ... , nscopy-9999.tmp
In Windows environment, it can be executed with the following PowerShell script.
# Testing MsgCleanupTempFiles()
New-Item $env:temp\nscopy.tmp -type file -Force
for ($i=1; $i -le 9999; $i++) {
$f = $env:temp + "\nscopy-" + $i + ".tmp"
New-Item $f -type file -Force
}
Start-Process -FilePath "C:\Program Files\Mozilla Thunderbird\thunderbird.exe"
Actual results:
Removed nscopy.tmp, nscopy-1.tmp, nscopy-2.tmp, ... , nscopy-9998.tmp, but nscopy-9999.tmp remains.
NS_ERROR_FILE_TOO_BIG
error occurs when the index reaches 9999, so files up to 9999 should be removed.
I think the code of MsgCleanupTempFiles() should be modified as below.
https://searchfox.org/comm-central/source/mailnews/base/src/nsMsgUtils.cpp#1271
} while (exists && ++index < 10000);
↓
} while (exists && index++ < 10000);
Comment 32•1 year ago
|
||
Comment 33•1 year ago
|
||
I don't actually work on Thunderbird, so I think somebody else would be in a better position to fix this. I forgot to clear myself from the assigned field when this got moved out of XPCOM.
Comment 34•1 year ago
|
||
The mechanism by which the nsemail-xxxx.eml file is increased is unknown, but I have seen reports of NS_ERROR_FILE_TOO_BIG errors that result in emails not being sent.
MozillaZine.jp forum (Japanese)
- https://forums.mozillazine.jp/viewtopic.php?f=3&t=20480 (2022-10-12)
- https://forums.mozillazine.jp/viewtopic.php?f=3&t=22110 (2023-05-13)
The NS_ERROR_FILE_TOO_BIG error can be prevented by automatically deleting the temporary file when Thunderbird starts.
I would appreciate it if someone could fix this bug.
I submitted a job with a patch that incorporate the proposed changes.
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=4166f87076876f916196758da9b6a60b636dc982
The patch:
https://hg.mozilla.org/try-comm-central/rev/f626bf44149071da679eb890b77743c0e7b87437
I feel a bit awkward when there are still so many OTHER errors to pronounce the health of the patch although I did not see anything that stood as new bugs.
Comment 36•1 year ago
|
||
ISHIKAWA, please submit the patch through phabricator. I don't see any new problems.
Comment 37•1 year ago
|
||
https://hg.mozilla.org/try-comm-central/rev/f626bf44149071da679eb890b77743c0e7b87437#l2.9
You've added
+ MsgCleanupTempFiles("nsemail", "eml");
+ MsgCleanupTempFiles("nsqmail", "tmp");
This is correct, since both temp file names are used (as already pointed out in comment #29):
https://searchfox.org/comm-central/search?q=nsemail.*eml&path=&case=false®exp=true
https://searchfox.org/comm-central/search?q=nsqmail.*tmp&path=&case=false®exp=true
Can you please also add nsemail.tmp
https://searchfox.org/comm-central/search?q=nsemail.*tmp&path=&case=false®exp=true
and remove nsemail.html which is not used any more:
https://searchfox.org/comm-central/search?q=nsemail.*html&path=&case=false®exp=true
See also comment in test_temporaryFilesRemoved.js. If you remove that, test_staleTemporaryFileCleanup.js needs to be adjusted accordingly.
We suggest to credit EarlgreyTea <earlgreypicard@gmail.com> with the analysis here.
(In reply to Magnus Melin [:mkmelin] from comment #36)
ISHIKAWA, please submit the patch through phabricator. I don't see any new problems.
Will do hopefully this week. I have not set up phabricator completely. I tried once several years ago, but it was too early . I mean the internal usage documentation was sparse and so I could not get it to work meaningfully.
(In reply to betterbird.project+6 from comment #37)
https://hg.mozilla.org/try-comm-central/rev/f626bf44149071da679eb890b77743c0e7b87437#l2.9
You've added+ MsgCleanupTempFiles("nsemail", "eml"); + MsgCleanupTempFiles("nsqmail", "tmp");
This is correct, since both temp file names are used (as already pointed out in comment #29):
https://searchfox.org/comm-central/search?q=nsemail.*eml&path=&case=false®exp=true
https://searchfox.org/comm-central/search?q=nsqmail.*tmp&path=&case=false®exp=trueCan you please also add nsemail.tmp
https://searchfox.org/comm-central/search?q=nsemail.*tmp&path=&case=false®exp=true
and remove nsemail.html which is not used any more:
https://searchfox.org/comm-central/search?q=nsemail.*html&path=&case=false®exp=true
See also comment in test_temporaryFilesRemoved.js. If you remove that, test_staleTemporaryFileCleanup.js needs to be adjusted accordingly.We suggest to credit EarlgreyTea <earlgreypicard@gmail.com> with the analysis here.
Let me digest this and incorporate necessary changes. Please wait for a few days.
Comment 39•1 year ago
|
||
Let me digest this and incorporate necessary changes.
You can see it in the updated version.
Comment 40•1 year ago
|
||
I think.
The patch to fix this bug has already been shown.
However, I do not have a build environment and no way to commit the patch.
Ishikawa-san also needs time to get phabricator set up and working.
If someone who can already use phabricator can submit the patch, we can move forward to the next step.
(In reply to EarlgreyTea from comment #40)
I think.
The patch to fix this bug has already been shown.
However, I do not have a build environment and no way to commit the patch.
Ishikawa-san also needs time to get phabricator set up and working.
If someone who can already use phabricator can submit the patch, we can move forward to the next step.
Copmment 41
My first attempt to use moz-phab seemed successful, except for the missing reviewer.
Someone can comment on the review, I think.
I am afraid I am a follower of the school of thought that favors independent tools rather than
intewound tool like moz-phab that is tied very deep with MERCURIAL, but that is because I have used HG MQ extension and git and like the ease of pushing and popping patches locally quite often.
Sure, once you are deep into the workflow of HG and other cloud services, I believe moz-phab may help a lot.
Except, I still can't figure out how to handle almost two dozen different patch sets that handle two dozen different bugs/issues neatly
once in my local queue without HG MQ.
Anyway, I will be learning a bit more about HG EVOLUTION, I suppose.
(It would be great if there is a testbed for moz-phab. I almost screwed up by posting many locally brewing patches without my intention.)
Comment 43•1 year ago
|
||
Chiaki, is the any issue following our advice in comment #37? See comment #39 for a full version of the patch.
The file types that should be added are nsemail.eml, nsemail.tmp and nsqmail.tmp. nsemail.html should be removed and the tests adjusted accordingly.
(In reply to betterbird.project+8 from comment #43)
Chiaki, is the any issue following our advice in comment #37? See comment #39 for a full version of the patch.
The file types that should be added are nsemail.eml, nsemail.tmp and nsqmail.tmp. nsemail.html should be removed and the tests adjusted accordingly.
That is a simple oversight. I thought I had modified the local patch accordingly, but obviously I forgot to do that.
Now, I have to learn the process of updating a patch submitted via phabricator. Please wait for a day or two.
Comment 45•1 year ago
|
||
To update a patch submitted with phab, just hg up <rev>
, do the changes, hg amend
and then moz-phab .
As long as the "Differential Revision: https:.... " line remains inside the commit, moz-phab will know it's the same, and send an update instead
Comment 46•1 year ago
|
||
(In reply to ISHIKAWA, Chiaki from comment #44)
Please wait for a day or two.
It seems it was two weeks, not two days. Or maybe two months.
Sorry my day job and other stuff keeps me busy. I wish there were 40 hours in a day.
Assignee | ||
Comment 48•1 year ago
|
||
Updated•1 year ago
|
Assignee | ||
Comment 49•1 year ago
|
||
(In reply to EarlgreyTea from comment #40)
The patch to fix this bug has already been shown.
However, I do not have a build environment and no way to commit the patch.
Ishikawa-san also needs time to get phabricator set up and working.
If someone who can already use phabricator can submit the patch, we can move forward to the next step.
Hope, that's ok, ISHIKAWA.
Updated•1 year ago
|
Assignee | ||
Updated•1 year ago
|
Comment 50•1 year ago
|
||
Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/57adabcbe8c7
Fix non-deletion of nsmail-9999.tmp. r=mkmelin
(In reply to Hartmut Welpmann [:welpy-cw] from comment #49)
(In reply to EarlgreyTea from comment #40)
The patch to fix this bug has already been shown.
However, I do not have a build environment and no way to commit the patch.
Ishikawa-san also needs time to get phabricator set up and working.
If someone who can already use phabricator can submit the patch, we can move forward to the next step.Hope, that's ok, ISHIKAWA.
Thank you. That is fine.
I have been down with Covid-19 and recover has been very slow.
Thank you for taking care of the pending patch.
Assignee | ||
Comment 52•1 year ago
|
||
(In reply to ISHIKAWA, Chiaki from comment #51)
I have been down with Covid-19 and recover has been very slow.
Thank you for taking care of the pending patch.
My pleasure. You could mark your revision as abandoned ("Add Action..." down below in Phabricator).
I wish you all the best for your further recovery!
Comment 53•1 year ago
|
||
Comment on attachment 9346419 [details]
Bug 673703 - remove temporary files
[Triage Comment]
Approved for beta
Comment 54•1 year ago
|
||
Comment on attachment 9352213 [details]
Bug 673703 - Fix non-deletion of nsmail-9999.tmp. r=mkmelin
[Triage Comment]
Approved for beta
Updated•1 year ago
|
Comment 55•1 year ago
|
||
Comment on attachment 9346419 [details]
Bug 673703 - remove temporary files
[Triage Comment]
welpy-cw, Thanks for the correction. I missed that only one had been checked in.
Comment 56•1 year ago
|
||
bugherder uplift |
Thunderbird 118.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/e997ca319a3a
Assignee | ||
Updated•1 year ago
|
Comment 57•1 year ago
|
||
Comment on attachment 9352213 [details]
Bug 673703 - Fix non-deletion of nsmail-9999.tmp. r=mkmelin
[Triage Comment]
Approved for esr115
Comment 58•1 year ago
|
||
bugherder uplift |
Thunderbird 115.3.0:
https://hg.mozilla.org/releases/comm-esr115/rev/7063abfca36b
Description
•