Downloading new emails from pop3 server rapidly slows down with each downloaded email when quarantining turned on

RESOLVED FIXED in Thunderbird 14.0

Status

MailNews Core
Networking: POP
--
major
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: mayhemer, Assigned: Bienvenu)

Tracking

({perf, regression})

Thunderbird 14.0
perf, regression
Bug Flags:
in-testsuite +

Thunderbird Tracking Flags

(thunderbird12+ fixed, thunderbird13+ fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
http://hg.mozilla.org/releases/mozilla-aurora/rev/5c4189019bc1 (I don't know what comm-central revision it is)

I recently lost my TB profile, so I'm redownloading all messages from a pop3 
server.  There is over some 36 thousands emails.

I noticed that after some 5000 email aprox. had been downloaded, the process slowed rapidly down.  At some 15000 email each mail takes about 1 second to download.

nsPop3Sink::IncorporateBegin is the culprit.  It call CreateUnique to create a temp download file.  Problem is that from some reason the prior temp files are not deleted.

I have no idea how nsPop3Sink exactly works, but both places where m_tmpDownloadFile->Remove(false) is called are not hit.
(Reporter)

Comment 1

6 years ago
Looks like if (m_pop3ConData->last_accessed_msg >= m_pop3ConData->number_of_messages) at nsPop3Protocol::GetMsg() never passes...

I get back to this, if this is not known or someone else doesn't take, what I would prefer, later next week maybe.

Till that time I'll delete the temp files manually....
(Assignee)

Comment 2

6 years ago
oh, interesting. I should fix that.

Comment 3

5 years ago
Can you tell us what are the names of the temp files? What is the pattern?
There are still several of these "not cleaning temp files"-bugs open but they  did presumably happen at other actions of TB (not POP3 download).
(Reporter)

Comment 4

5 years ago
(In reply to :aceman from comment #3)
> Can you tell us what are the names of the temp files? What is the pattern?

newmsg, and as CreateUnique creates, followings are newmsg-1, newmsg-2 ... newmsg-999.

> There are still several of these "not cleaning temp files"-bugs open but
> they  did presumably happen at other actions of TB (not POP3 download).

This is specific to POP3 as I understood the code.  It still happens, I still have to delete the files manually.

Just a reminder: this happens on a fresh profile.  I use global inbox for the account.  At the times I was able to reproduce this there were just a single pop3 account.

Comment 5

5 years ago
Good, so it is another type of temp files and are not covered by bug 299655.
(Assignee)

Comment 6

5 years ago
Created attachment 611620 [details] [diff] [review]
proposed fix with unit test

Ghe quarantining stuff works by creating one temp file per get new mail session and re-using it by truncating it for every message downloaded, but we were creating a tmp file for every message. I probably messed this up during the pluggable store work. This fixes the leak, and adds a unit test.
Assignee: nobody → dbienvenu
Attachment #611620 - Flags: review?(mbanner)
(Assignee)

Updated

5 years ago
Keywords: regressionwindow-wanted
OS: Windows 7 → All
Hardware: x86_64 → All
Summary: Downloading new emails from pop3 server rapidly slows down with each downloaded email → Downloading new emails from pop3 server rapidly slows down with each downloaded email when quarantining turned on
(Assignee)

Comment 7

5 years ago
we'll want this fix landed pretty soon on all branches.
Status: NEW → ASSIGNED
status-thunderbird12: --- → affected
status-thunderbird13: --- → affected
tracking-thunderbird12: --- → +
tracking-thunderbird13: --- → +
Attachment #611620 - Flags: review?(mbanner)
Attachment #611620 - Flags: review+
Attachment #611620 - Flags: approval-comm-beta+
Attachment #611620 - Flags: approval-comm-aurora+
(Assignee)

Comment 8

5 years ago
fixed on trunk - http://hg.mozilla.org/comm-central/rev/27aac7b2e676

I'll transplant to branches in a minute.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 14.0
(Assignee)

Comment 9

5 years ago
fixed for beta - http://hg.mozilla.org/releases/comm-beta/rev/dd4c013f2477
aurora - http://hg.mozilla.org/releases/comm-aurora/rev/b022aa765e51
status-thunderbird12: affected → fixed
status-thunderbird13: affected → fixed
Blocks: 402392
status-thunderbird12: fixed → affected
status-thunderbird13: fixed → affected
status-thunderbird12: affected → fixed
status-thunderbird13: affected → fixed
You need to log in before you can comment on or make changes to this bug.