178153 - Changing 1199 bugs took 75 minutes

Status: RESOLVED FIXED

(Bugzilla :: Creating/Changing Bugs, defect)

Description

[timeless]

well um. i just tried to kill off UID. (I failed; there are 13 bugs i can't
see.) I'm filing this bug to complain about how long it took for me to move 1200
bugs :-)

steps: collect one very large useless component 'uid'. collect ~1200 resolved
bugs (resolution!=---).
query for all bugs (that you can see...) in the component
select change several bugs at once
check all
select a new component (i used browser-general).

Here's the first bugmail i got:
Date: Sun, 3 Nov 2002 02:33:02 -0800 (PST)
Message-Id: <[e-mail]>
From: [e-mail]
To: [e-mail]
Subject: [Bug 130580] Following link and then "Back"ing displays previous page
at top.
X-Bugzilla-Reason: AssignedTo QAContact

http://bugzilla.mozilla.org/show_bug.cgi?id=130580

[e-mail] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|User Interface Design       |Browser-General

And the last one:
Date: Sun, 3 Nov 2002 03:47:50 -0800 (PST)
Message-Id: <[e-mail]>
From: [e-mail]
To: [e-mail]
Subject: [Bug 49201] Print button should be hidden by default
X-Bugzilla-Reason: AssignedTo QAContact CC

http://bugzilla.mozilla.org/show_bug.cgi?id=49201

[e-mail] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|User Interface Design       |Browser-General

Comment 1

[Bradley Baetz (:bbaetz)]

This sounds about right. 1-1.5 seconds to kick off processmail (remember, it
loads global.spl which loads Template + friends) gives about 3000 processmail
calls in 75 minutes.

If you include 1.5 dependancy per bug, (remember, we run processmail each way)
that gives 3000.

See bug 124174 for making processmail run in the same process.

Comment 2

[Bradley Baetz (:bbaetz)]

.. and even better, we don't send dependency mails from mass change - see bug
178157, so thats not it.

I'm guessing that sendmail reached some sort of throttle limit due to the
massive abuse timeless was doing

Comment 3

[timeless]

Received: from mothra.mozilla.org not authenticated [207.200.81.216]
	by smtp-send.myrealbox.com with NetMail SMTP Agent $Revision:   3.14  $ on
Novell NetWare;
	Sun, 03 Nov 2002 03:29:21 -0700
Received: (from nobody@localhost)
	by mothra.mozilla.org with  id gA3AX2w13629;
	Sun, 3 Nov 2002 02:33:02 -0800 (PST)
--
Received: from mothra.mozilla.org not authenticated [207.200.81.216]
	by smtp-send.myrealbox.com with NetMail SMTP Agent $Revision:   3.14  $ on
Novell NetWare;
	Sun, 03 Nov 2002 04:44:09 -0700
Received: (from nobody@localhost)
	by mothra.mozilla.org with  id gA3BloW10980;
	Sun, 3 Nov 2002 03:47:50 -0800 (PST)

Comment 4

[J. Paul Reed [:preed]]

Bug 124174 went in today; this is encouraging:

* justdave does a bulk change of 576 bugs on landfill
<justdave> holy cow is it flying.
<justdave> that's an order of magnitude faster :)

Comment 5

[Dave Miller [:justdave]]

I just made a bulk change of 576 bugs on landfill (after it picked up the fix
for bug 124174) and it took 15 minutes on the nose.  Lots of people sign up for
Bugzilla with fake addresses (with bad domains on them even).  They can't sign
in with them, but people still use them for CCs or assigned to.  There were
noticible pauses when it hit a bug with one of those addresses on it (presumably
DNS timeouts).  When the bug had good addresses, they all flew past so fast I
had a hard time scrolling fast enough to keep up with them.

This kind of performance, even with the occassional pauses for bad addresses,
would get your 1199 bug bulk change down to 30 minutes (instead of 75) which is
a pretty marked improvement.  I'd bet it'd be even faster on b.m.o since most
people actually use real addresses and not stupid stuff to play around :)

The new Bugzilla::BugMail module is a few orders of magnitude faster then
processmail was. :)  I'm blaming the new performance gains on the fact that
Bugzilla no longer has to spawn a new Perl interpreter process for each bug that
it sends mail on :)