removing many expired messages can be very slow

RESOLVED FIXED

Status

MailNews Core
Networking: NNTP
--
major
RESOLVED FIXED
17 years ago
10 years ago

People

(Reporter: Mark Hanson, Assigned: Bienvenu)

Tracking

({perf})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

654 bytes, patch
(not reading, please use seth@sspitzer.org instead)
: superreview+
Details | Diff | Splinter Review
(Reporter)

Description

17 years ago
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221
BuildID:    2001122108

In high traffic newsgroups, clicking on the link "click here to remove all
expired articles" can be extremely slow.  It can take tens of minutes to finish
on a 500MHz Celeron with plenty of memory (no swapping).  During this time, a
mozilla-bin process hogs the CPU and strace shows that it is executing many,
many redundant __llseek() and fstat() calls with the same arguments.

_llseek(60, 0, [835014], SEEK_CUR)      = 0
fstat(60, {st_mode=S_IFREG|0644, st_size=835014, ...}) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
fstat(60, {st_mode=S_IFREG|0644, st_size=835014, ...}) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
fstat(60, {st_mode=S_IFREG|0644, st_size=835014, ...}) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
_llseek(60, 835014, [835014], SEEK_SET) = 0
write(60, "\n@$${B031{@\n<(A4D9=5b7a)>[4A:^80"..., 55) = 55
_llseek(65, 0, [14783003], SEEK_CUR)    = 0
fstat(65, {st_mode=S_IFREG|0644, st_size=14783003, ...}) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
fstat(65, {st_mode=S_IFREG|0644, st_size=14783003, ...}) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
fstat(65, {st_mode=S_IFREG|0644, st_size=14783003, ...}) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
fstat(65, {st_mode=S_IFREG|0644, st_size=14783003, ...}) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
_llseek(65, 14783003, [14783003], SEEK_SET) = 0
write(65, "\n@$${2538{@\n[244C:m(^91=0)]\n{1:^"..., 220) = 220

What benefit is there to seeking to the same point in a file three times in a
row?  Looks like this can be optimized quite a bit.

Reproducible: Sometimes
Steps to Reproduce:
1. Find a newsgroup with a lot (thousands) of expired articles.
2. Click on an expired article.
3. When the message appears, click on the link to remove all expired articles.
4. Run strace on the mozilla-bin process which is using all the CPU.

I believe that the reproducability of this depends on the number of articles it
needs to process.  If there aren't many, the process can only take a few seconds.

Actual Results:  A bit of the strace output is in the Description section.

Expected Results:  It should have completed the task more quickly.
(Assignee)

Comment 1

17 years ago
Rob, that's not a database issue; it's a news issue. It sounds like news is
committing the db way too often in this scenario.
Status: UNCONFIRMED → NEW
Component: Mail Database → Networking - News
Ever confirmed: true

Updated

17 years ago
QA Contact: esther → stephend
-> seth, me
Assignee: bienvenu → sspitzer

Updated

17 years ago
Severity: critical → major
(Assignee)

Comment 3

16 years ago
Created attachment 82125 [details] [diff] [review]
proposed fix

don't commmit the db for every header removed - the db will be committed at the
end of the process anyway
Comment on attachment 82125 [details] [diff] [review]
proposed fix

sr=sspitzer

excellent!
Attachment #82125 - Flags: superreview+
(Assignee)

Comment 5

16 years ago
taking
Assignee: sspitzer → bienvenu
(Assignee)

Comment 6

16 years ago
*** Bug 76232 has been marked as a duplicate of this bug. ***
Dup'd bug was win2k, marking os-->all.
OS: Linux → All
*** Bug 124673 has been marked as a duplicate of this bug. ***
David - you landed this on the trunk but haven't yet marked it, are you
intending this to be landed on the branch, too?
(Assignee)

Comment 10

16 years ago
no, I just forgot to mark it fixed. I'll nominate it for the branch, since it's
such a trivial fix, but I don't think anyone in charge is going to care.
Status: NEW → RESOLVED
Last Resolved: 16 years ago
Keywords: nsbeta1
Resolution: --- → FIXED
Keywords: perf
Hardware: PC → All

Comment 11

16 years ago
This occurs still in RC2 (build 2002051005) on MacOS9.2. I see only "Loading
document ..." and then nothing more. I needed to use Force to quit. This sucks!
(Assignee)

Comment 12

16 years ago
the fix is only on the trunk, not in rc1 or rc2, as my previous comment should
have made clear.

Comment 13

16 years ago
What needs then to be done to get it into the releases?

Updated

16 years ago
Keywords: nsbeta1 → nsbeta1-
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.