1741517 - Email corruption, possibly leading to 100% storage utilization

Status: RESOLVED WORKSFORME

(Thunderbird :: Untriaged, defect)

Description

[Tristan Young]

Attachment: Thunderbird corrupted email.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0

Steps to reproduce:

Ongoing issue with email corruption. Thunderbird consuming 100% of storage space. Seen in Thunderbird v95 beta 1 and beta2, and I saw it happen on a different machine running Thunderbird v94.

Open an affected email, and scroll to the bottom

I have tried repairing the affected msf files, compacting the affected folders, wiping out all stored emails and re-downloading (imap).

No add-ons. Also happens in troubleshoot mode.

Actual results:

Evidence of corruption at the bottom of the email

With the global search and indexer on, nstmp (nstmp-1, nstmp-2, etc) are created in the profile's ImapMail folder, and grow in size until there's no room left on the drive.

With the global search and indexer off, so far, the munging of storage space doesn't happen, however the email corruption persists.

Not every email shows visible signs of corruption. Some emails cause Thunderbird to freeze for a short period of time and show corruption. Emails exhibiting corruption come from from a variety of senders.

These same emails, when viewed in K9 email on Android, show no corruption and cause no issues.

This causes additional write wear on an SSD drive.

Comment 1

[Tristan Young]

I notice on some emails that I'm seeing evidence of a different unrelated email in the bottom.

For example, one of the emails showing this behaviour is dated November 10 2021 8:40pm. At the bottom, I see a variation of a second email which is actually 1 email before this one, dated November 9 2021 7:56pm. The November 9th email also exists as a separate email and doesn't show corruption. There are no other emails between these two.

Again, everything looks normal when viewing with K9 email.

This is happening not only in my INBOX, but also in my archives folder.

Comment 2

[Tristan Young]

I should have scrolled down further on the example email I was using. I clicked on View Source, and I'm seeing more than 1 extra message at the bottom. It looks like quite a few extra emails have been tacked on to the bottom.

Comment 3

[Magnus Melin [:mkmelin]]

Fixeed on trunk. You'll have to do Repair Folder to fix the indexing.

Comment 4

[Tristan Young]

Perfect, thank you.

Sorry for the duplicate bug report. I tried to search before submitting, but I guess I didn't search for the right term.

I look forward to testing this on my setup.

Best regards,
Tristan Young

Comment 5

[Tristan Young]

Hello,

I'm wondering if there has been any progress on the issue.

I've installed every beta update since the problem began occurring, repaired folder each time, and even deleted the .msf file and re-downloaded the emails. I am still seeing this problem, and it's causing major issues. I'm currently running v98 beta 3.

This is not only a bug that affects performance and useability, but it could violate privacy if an email reply is created and the contents of other emails appears at the end of the email.

Thank you.

Comment 6

[WaltS48 [:walts48]]

Looks to me like this bug was resolved as a duplicate of bug 1734847, and you should follow along there.

You are Cc'd on that bug report.

Comment 7

[Magnus Melin [:mkmelin]]

Which is continued in bug 1742975, but we have no steps to reproduce. If you find such steps, please add them there.

Comment 8

[Wayne Mery (:wsmwk)]

Per Tristian's comment 5, Resetting status

Comment 9

[Wayne Mery (:wsmwk)]

Tristian, does this still reproduce for you?

Comment 10

[Tristan Young]

(In reply to Wayne Mery (:wsmwk) from comment #9)

Tristian, does this still reproduce for you?

I am still experiencing freezing and multiple emails merged as one. Forcing Thunderbird to terminate is a common occurrence.

I haven't seen my storage space disappear, but honestly I've been managing things a lot more closely, and I think disabled global indexing. Still seeing nstmp files being generated with large file sizes.

Repair Folder doesn't fix the problem. Deleting all the email files and re-downloading from my personal email server doesn't solve the problem. Still occurring on multiple Windows PCs, with the exact identical bad effects.

I wonder if I were to give someone access to my inbox, or made a new mailbox and cloned the majority of non-private emails to it, whether they'd experience the same problem.

Comment 11

[Tristan Young]

In fact, I had to force-close Thunderbird moments ago. After restarting it, I was able to read one email. Clicked on a different email, and freeze. Had to force-close TB again.

So I read most of my emails with K9 Mail on my phone and tablet. No problems there.

Comment 12

[Wayne Mery (:wsmwk)]

Suggest you test tomorrow's beta 102.0b7 and post your results.

Comment 13

[Tristan Young]

(In reply to Wayne Mery (:wsmwk) from comment #12)

Suggest you test tomorrow's beta 102.0b7 and post your results.

Ok, will do.

Comment 14

[Tristan Young]

(In reply to Wayne Mery (:wsmwk) from comment #12)

Suggest you test tomorrow's beta 102.0b7 and post your results.

I've been testing 102.0b7 this week, and so far, seems like it's working fine on multiple machines.

Thank you!

I'll continue testing, and will report any unusual behaviour.

Cheers!