Closed Bug 1589890 Opened 5 years ago Closed 4 years ago

Temporary files not cleared upon close (nsemail.tml, nscopy.tmp, nsmail.tmp)

Categories

(MailNews Core :: Attachments, defect)

Product:
MailNews Core
Component:
Attachments
Platform:
All
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: hpfeil, Unassigned)

References

(
URL
)

Details

(Keywords: platform-parity)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

Version: 68.1.2 (64-bit) Linux
Open Thunderbird, read, respond, write new messages.

Actual results:

Depending on the session activity, lots of files left in /tmp. Typical files:
nscopy.tmp
nsemail-1.eml
nsemail-2.eml
nsemail.eml
nsemail.html
nsmail.tmp

These files are owner read/write only. If left alone, more such files accumulate during each Thunderbird session. Not a priority, just somewhat sloppy, like a memory leak.

Expected results:

Those files should have been deleted upon each session close.

Hmm, interesting. I played with temporary files in bug 1568896 comment #8 on Windows. They all got cleaned up when TB closed. That's not the case on Linux?

Flags: needinfo?(mkmelin+mozilla)

No, Jorg, not on Linux. Before I suggested that it might be a bug, I planned to write a script around Thunderbird to "rm /tmp/ns*" upon exit. This morning I found these. nsemail-0,1,2,3 were from yesterday, the rest today.
nscopy-1.tmp
nscopy-2.tmp
nscopy-3.tmp
nscopy-4.tmp
nscopy.tmp
nsemail-1.eml
nsemail-2.eml
nsemail-3.eml
nsemail-4.eml
nsemail-5.eml
nsemail-6.eml
nsemail-7.eml
nsemail-8.eml
nsemail.eml
nsmail-1.tmp
nsmail-2.tmp
nsmail-3.tmp
nsmail.tmp

Sorry about the double, I thought I was editing the message so you'd know the files were all created after I had deleted the ones in the original post.
The four nsmail*.tmp files are all copies of the same jpg image that I sent in one message as a lone attachment. In examining the other files, I noticed that I need to change my habit of replying to a recent message, then changing the subject and deleting the old content to create a new message, suffering from congenital indolence as I oftentimes do. I found huge reference lists of previous threads in the header. I was under the impression that a new Subject meant a new message. Apparently you can change the subject in any reply and it remains a reply to the previous message. Perhaps it is that habit that leaves behind the /tmp files?
The nscopy files are all the same message in text and html format with the attached image in mime format.
I cleared ns*, opened t-bird, sent myself a test message, then closed t-bird. These four are all from that message, in both txt and html format.
nscopy.tmp
nsemail.eml
nsemail.html
nsmail.tmp

I download the latest tarball, extract the Thunderbird folder, then move it to /usr/lib64. Would installing via a package manager change the result?

(In reply to Jorg K (GMT+2) from comment #1)

temporary files ... all got cleaned up when TB closed.

Me too, under controlled shutdown in Windows. However my %tmp% also has a couple hundred files over a period of a few months so there must be circumstances where cleanup doesn't happen.

Anyway, among the open bugs we have https://mzl.la/2MGutmu and the last bug fixed was perhaps Bug 235432 - Mailnews/Thunderbird leaves unused nsqmail.tmp (nsqmail-*.tmp, nsemail.eml) files in temporary folder (TEMP or /tmp) after quit

OK, I spontaneously checked C:\Users\jorgk\AppData\Local\Temp and found four left-over temp files :-(
nsemail.eml with versions -1 and -2 and a nsmail.eml. The former three only contained headers, the latter one contained a message attachment, so message which I had sent out as an attachment. So yes, there must be some hole somewhere.

Yeah I see the nsmail tmp files too. I also thought we put those in a per-user subdir of tmp at some point, or was that only for opened attachments?

Flags: needinfo?(mkmelin+mozilla)

(In reply to Magnus Melin [:mkmelin] from comment #7)

Yeah I see the nsmail tmp files too. I also thought we put those in a per-user subdir of tmp at some point, or was that only for opened attachments?

A number of issues have been fixed over the years: https://mzl.la/34uiBKd
And some not: https://mzl.la/2Nc90lr

bug 58979 suggests the use of a subdir.

One open bug is Bug 368380 - lots of temp files left around

In the intervening years, the following have been fixed (Plus some tmp bugs not related to sending mail - compact, etc)
Bug 235432 TB13 - Mailnews/Thunderbird leaves unused nsqmail.tmp (nsqmail-*.tmp, nsemail.eml) files in temporary folder (TEMP or /tmp) after quit
Bug 299655 TB15 - /tmp files left behind, some world-readable

These bugs have comments from some users that their issue was not fixed.

Component: Untriaged → Attachments
Product: Thunderbird → MailNews Core
Summary: Temporary files not cleared upon close → Temporary files not cleared upon close (nsemail.tml, nscopy.tmp)

i can confirm this behavior in Linux. i am running same version as reporter on debian unstable (bug 1602759). i would add that those files contain copies of sent emails. sometimes spotted even plain text where the sent email was encrypted. i tried running thunderbird --verbose but it did not help revealing more info. i tried thunderbird -g but thunderbird-dbgsym seems to be missing from debian.

Summary: Temporary files not cleared upon close (nsemail.tml, nscopy.tmp) → Temporary files not cleared upon close (nsemail.tml, nscopy.tmp, nsmail.tmp)

Confirming problem on Thunderbird 68.7.0 (32-bit) on Windows7.

Scary part is that I am not entirely sure that I was running Thunderbird when the files went into the temp-folder.

The files "nsemail.eml" to "nsemail-10.eml" contain emails I have sent the last months. I have had no reason to open them lately, so it feels like someone is spying on me.

Same here, Ubutnu 20.04 LTS with their Thunderbird 68.7.0

Isn't it a security issue that plain text is persisted on disk ?

I consider this a memory leak and/or design flaw. My feeble understanding of Thunderbird innards suggests that these files might be created to form the final message that is sent? If so, they should be deleted once the message is sent and the copy is in the Sent folder. They should exist only on the POP or IMAP server and ~/.thunderbird. Use of the file system should be restricted to the user's ~/.thunderbird folder, then deleted when finished. I don't know where the files appear on Windows 10, but in any case, delete any temporary files you need to create.

In case the system crashes or is shut off while Thunderbird is active, the next time Thunderbird gets launched those temporary files from the previous session could be used to reconstruct the interrupted message, should the user wish to retrieve it. The current design places permanent files in a location meant for temporary files.

Confirm this bug on Gentoo x86_64, thunderbird-bin 68.8.0.

It is not just Linux issue. Happens on Windows also. At Windows 10 2004, Thunderbird 78.2.2 (but was also previous versions (68).

I am seeing the same on my Windows 10 Pro 2004.
Thunderbird 78.3.2
Location of the nsemail & nsmail
C:\Users\Name\AppData\Local\Temp

Example of what I am seeing.
nsemail.eml
nsemail-1.eml
thru nsemail-10.eml

There are also nsmail.pdf, nsmail.mp4, nsmail.tmp, nsmail-1.pdf etc.

They are not being deleted on exit.

See Also: → 1595343

I also see a large number of ns* temporary files that Thunderbird leaves behind in /tmp/. This is quite annoying.

I've currently got almost 1000 ns* files there:

/tmp/nsemail-328.eml  /tmp/nsemail-622.eml  /tmp/nsmail-2.pdf
/tmp/nsemail-329.eml  /tmp/nsemail-623.eml  /tmp/nsmail-2.png
/tmp/nsemail-32.eml   /tmp/nsemail-624.eml  /tmp/nsmail-2.tmp
/tmp/nsemail-330.eml  /tmp/nsemail-625.eml  /tmp/nsmail-30.tmp
/tmp/nsemail-331.eml  /tmp/nsemail-626.eml  /tmp/nsmail-31.tmp
/tmp/nsemail-332.eml  /tmp/nsemail-627.eml  /tmp/nsmail-32.tmp
/tmp/nsemail-333.eml  /tmp/nsemail-628.eml  /tmp/nsmail-33.tmp
/tmp/nsemail-334.eml  /tmp/nsemail-629.eml  /tmp/nsmail-34.tmp
/tmp/nsemail-335.eml  /tmp/nsemail-62.eml   /tmp/nsmail-3.eml
/tmp/nsemail-336.eml  /tmp/nsemail-630.eml  /tmp/nsmail-3.jpeg
/tmp/nsemail-337.eml  /tmp/nsemail-631.eml  /tmp/nsmail-3.pdf
/tmp/nsemail-338.eml  /tmp/nsemail-632.eml  /tmp/nsmail-3.tmp
/tmp/nsemail-339.eml  /tmp/nsemail-633.eml  /tmp/nsmail-4.eml
/tmp/nsemail-33.eml   /tmp/nsemail-634.eml  /tmp/nsmail-4.jpeg
/tmp/nsemail-340.eml  /tmp/nsemail-635.eml  /tmp/nsmail-4.pdf
/tmp/nsemail-341.eml  /tmp/nsemail-636.eml  /tmp/nsmail-4.tmp
/tmp/nsemail-342.eml  /tmp/nsemail-637.eml  /tmp/nsmail-5.eml
/tmp/nsemail-343.eml  /tmp/nsemail-638.eml  /tmp/nsmail-5.jpeg
/tmp/nsemail-344.eml  /tmp/nsemail-639.eml  /tmp/nsmail-5.pdf
/tmp/nsemail-345.eml  /tmp/nsemail-63.eml   /tmp/nsmail-5.tmp
/tmp/nsemail-346.eml  /tmp/nsemail-640.eml  /tmp/nsmail-6.eml
/tmp/nsemail-347.eml  /tmp/nsemail-641.eml  /tmp/nsmail-6.jpeg
/tmp/nsemail-348.eml  /tmp/nsemail-642.eml  /tmp/nsmail-6.pdf
/tmp/nsemail-349.eml  /tmp/nsemail-643.eml  /tmp/nsmail-6.tmp
/tmp/nsemail-34.eml   /tmp/nsemail-644.eml  /tmp/nsmail-7.jpeg
/tmp/nsemail-350.eml  /tmp/nsemail-645.eml  /tmp/nsmail-7.pdf
/tmp/nsemail-351.eml  /tmp/nsemail-646.eml  /tmp/nsmail-7.tmp
/tmp/nsemail-352.eml  /tmp/nsemail-647.eml  /tmp/nsmail-8.pdf
/tmp/nsemail-353.eml  /tmp/nsemail-648.eml  /tmp/nsmail-8.tmp
/tmp/nsemail-354.eml  /tmp/nsemail-649.eml  /tmp/nsmail-9.pdf
/tmp/nsemail-355.eml  /tmp/nsemail-64.eml   /tmp/nsmail-9.tmp
/tmp/nsemail-356.eml  /tmp/nsemail-650.eml  /tmp/nsmail.asc
/tmp/nsemail-357.eml  /tmp/nsemail-651.eml  /tmp/nsmail.docx
/tmp/nsemail-358.eml  /tmp/nsemail-652.eml  /tmp/nsmail.eml
/tmp/nsemail-359.eml  /tmp/nsemail-653.eml  /tmp/nsmail.ics
/tmp/nsemail-35.eml   /tmp/nsemail-654.eml  /tmp/nsmail.jpeg
/tmp/nsemail-360.eml  /tmp/nsemail-655.eml  /tmp/nsmail.odt
/tmp/nsemail-361.eml  /tmp/nsemail-656.eml  /tmp/nsmail.pdf
/tmp/nsemail-362.eml  /tmp/nsemail-657.eml  /tmp/nsmail.png
/tmp/nsemail-363.eml  /tmp/nsemail-658.eml  /tmp/nsmail.tmp
/tmp/nsemail-364.eml  /tmp/nsemail-659.eml  /tmp/nsmail.zip
/tmp/nsemail-365.eml  /tmp/nsemail-65.eml

This is just an excerpt. There are many more, total 989 files. And that although I've had already cleaned them up relatively recently already.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: platform-parity
OS: Unspecified → Linux
Hardware: Unspecified → All

I can confirm this as well.
This appears to be in the message sending code. I would not be surprised if saving drafts also triggers these files as saving a draft shares a lot ofg the same code as actually sending.

Today at 14:31 local time, I sent the email about building Thunderbird 85.0b3. And I have several files in tmp from that time that contain that message in various states:

-rw------- 1 rob  rob     3959 Dec 30 14:31 key-3.asc
-rw------- 1 rob  rob    78973 Dec 30 14:31 nscopy-2.tmp
-rw------- 1 rob  rob     3753 Dec 30 14:31 nsemail-2.html
-rw------- 1 rob  rob    78973 Dec 30 14:31 nsemail-3.eml
-rw------- 1 rob  rob    46469 Dec 30 14:31 nsmail-2.tmp
-rw------- 1 rob  rob      517 Dec 30 14:31 nsmail-3.tmp
  • key-3.asc - This is a copy of my public key
  • nscopy-2.tmp - The entire email, headers and all attachments as one would expect to find in a maildir/mbox file
  • nsemail-2.html - The email body in HTML format
  • nsemail-3.eml - The entire email, same as nscopy-2.tmp
  • nsmail-2.tmp - The release notes that were included in the email (and html attachment, so this file is HTML)
    ** nsmail-3.tmp* - The email body in plain text

Based on the files left behind, it looks like when a message is sent, the various mime parts are written out to temporary files to assemble the final message before sending. That produces nsemail-3.eml. nscopy-2.tmp is maybe because of the copy to the sent folder.

This is with Thunderbird 78.6.0.

One standard way to deal with this kind of thing is to open the file - then immediately unlink the file. The file handle can continue to be used to read / write the file.

The file is removed from the directory list (so is no longer visible). It is removed completely when the file handle is closed - or when app stops running (which closes all open file handles).

May mean passing file handle around instead of filename but that's a trivial and automatable change.

Maybe you guys can do something similar?

FWIW, I don't have temp files left behind if using js smtp (on 86 beta).

Good to know - i'm on 78.7.1

Also, am not familiar with TB code - you mention js smtp - is that a config option or just different smtp implementation in 86 vs 78 or something else?

Yes, different/new implementation. I mention it since AIUI it's meant to replace the current C++ code.

Correct. Both the send back-end and the smtp implementation has been rewritten after 78.
I think we can close this. Please reopen if you can reproduce with a recent beta.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME

As an addenda (and because the current downloadable version is still 78.*) I have to mention THE real problem with this bug.
If Thunderbird is used long enough to accumulate 10000 temporary files in the TEMP directory, so it has all the files from nsemail.eml to nsemail-9999.eml, Thunderbird will suddenly stop sending e-mails with an error.
It can be solved by simply deleting the files.

The system was a Windows 7 SP1 64bit with now latest TB 78.8.0.
But we met this problem earlier on a few machines (Win 7 and 10 mixed) with older and latest TB versions, where unknowing the real cause we solved the problem by removing TB, cleaning every corner of the system and meanwhile cleaning the temp folders as well.

This still appears to be an issue on Windows and also Linux. Folders such as pid-32601 get created when you open an attachment. The number changes each time you open a new session of Thunderbird, but the folders and their contents are never cleared.

Alan is referring to the bug regarding appdata/local/temp/pidxxx folders : bug 1769423
This continues to be a problem in release version TB 102.2.2 win64
TB needs to stop creating permanent pid folders which contain read only copies of attachments that are stored permanently. Although you can edit the Properties to remove the read- only. But how many people know about the appdata>Local> Temp? Most people have appdata as a hidden folder and do not even know about it.

You need to log in before you can comment on or make changes to this bug.