233645 - Bugzilla *very* slow when changing > 10 bugs at once

Status: RESOLVED FIXED

(Bugzilla :: Creating/Changing Bugs, defect)

Description

[Thor List]

User-Agent:       
Build Identifier: 

Gerv wrote:

This is looking like a bug. Please file one, so we can track this 
properly :-)

I suspect lines 1118 and 1119 of process_bug.cgi, where in the multiple 
case, $::FORM{'product'} may well be undef.



Initial description:

Today I had to close 91 bugs that were verified, and I chose to "Edit
multiple bugs", selected all and asked to change their status to closed.

It changes and prints 10 bugs really quickly, then stops for several
minutes, while Bugzilla is completely unresponsive (Apache and the computer
is less than 1% busy and mysqld takes about 0.1% or 0.0% of CPU in top).

After several minutes I can get access to bugzilla again and about 10-15
bugs might have been changed out of the 91. So I repeat, and the same
happens.

I have changed the bugzilla parameter sendmailnow to off, but this doesn't
seem to change anything.

While this is going on, I can easily use MySQL on other tables, but the one
actually being modified. I assume that Bugzilla locks the bugs table while
it is editing, so I have tried leaving my select statement waiting and it
takes many minutes before it is allowed in.

It seems to be stuck in (using ps axf):

     8940 ?        S      0:02  |   \_ /usr/bin/perl -wT
/home/httpd/bugzilla/process_bug.cgi

Otherwise, bugzilla is really fast and works beautifully.



Further info:

> ISTR that there is some oddity where Apache will block a process if
> there is too much going to STDERR.  Look at the apache error log as you
> do a small mass-change.  If there is stuff going into the log on a
> per-bug basis, see if supressing some of that mitigates the problem.

You are quite right. When I do a post I get hundreds of:

process_bug.cgi: Use of uninitialized value in
pattern match (m//) at globals.pl line 832., referer:
http://xxxxxxxxxxxxx/buglist.cgi?query_format=&short_desc_type=allwordss
ubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type
=allwordssubstr&bug_file_loc=&status_whiteboard_type=all
wordssubstr&status_whiteboard=&bug_status=VERIFIED&emailtype1=exact&email1=&
emailtype2=substring&email2=&bugidtype=include&bug_id=&v
otes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&remaction=run&namedcmd=All%20
Bugs&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.creation_ts%2Cbug
s.bug_status%2C%20bugs.priority%2C%20map_assigned_to.login_name%2C%20bugs.bu
g_id&tweak=1

process_bug.cgi: Use of uninitialized value in
string eq at /xxxxxxxx/bugzilla/process_bug.cgi line 369., referer:
http://xxxxxxxxxxxxxxx/buglist.cgi?query_format=&short_des
c_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&b
ug_file_loc_type=allwordssubstr&bug_file_loc=&status_whi
teboard_type=allwordssubstr&status_whiteboard=&bug_status=VERIFIED&emailtype
1=exact&email1=&emailtype2=substring&email2=&bugidtype=i
nclude&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&remaction=run
&namedcmd=All%20Bugs&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.c
reation_ts%2Cbugs.bug_status%2C%20bugs.priority%2C%20map_assigned_to.login_n
ame%2C%20bugs.bug_id&tweak=1

Mostly those two are repeating. Do you have any idea why this is happening?
The code at those points do not exactly hint at a solution...:-)


Reproducible: Always
Steps to Reproduce:
1. Edit 15+ bugs at a time
2. Commit any change to be made to all
3. For some this is a problem even for 5+ bugs

Actual Results:  
Bugzilla becomes non-responsive for 3-5 minutes and afterwards only about 10-12 
bugs have been changed, regardless of how many were initially selected to be 
edited.

Expected Results:  
Done this in less than a seconds time, I would imagine...

See details and email thread "Bugzilla *very* slow when changing > 10 bugs at 
once" in netscape.public.mozilla.webtools

Comment 1

[Gervase Markham [:gerv]]

Attachment: Patch v.1

This fixes the problem on my machine - I can update 379 bugs in 60 seconds with
the patch; it hangs after ten bugs without.

The patch adds three checks for a value being defined in strategic places. For
the component_id one, the trick is not to allow the form field to exist at all
unless there's a value to go in it.

Gerv

Comment 2

[Gervase Markham [:gerv]]

Comment on attachment 141271 [details] [diff] [review]
Patch v.1

Dave: this is a fairly simple one, with a big performance gain.

Gerv

Comment 3

[Dave Miller [:justdave]]

Comment on attachment 141271 [details] [diff] [review]
Patch v.1

>Index: globals.pl
>===================================================================
>@@ -829,7 +829,7 @@
> 
> sub get_component_id {
>     my ($prod_id, $comp) = @_;
>-    return undef unless ($prod_id =~ /^\d+$/);
>+    return undef unless ($prod_id && ($prod_id =~ /^\d+$/));
>     PushGlobalSQLState();
>     SendSQL("SELECT id FROM components " .
>             "WHERE product_id = $prod_id AND name = " . SqlQuote($comp));

This is okay, but couldn't this easily be
detaint_natural($prod_id) || return undef;
?

r=justdave anyway.  If you want to make that change, go for it, but it works
like this, too. :)

Comment 4

[Dave Miller [:justdave]]

actually, detaint_natural throws a warning if you pass it an undef, too, doesn't
it...

Comment 5

[Gervase Markham [:gerv]]

Fixed.

Checking in buglist.cgi;
/cvsroot/mozilla/webtools/bugzilla/buglist.cgi,v  <--  buglist.cgi
new revision: 1.244; previous revision: 1.243
done
Checking in process_bug.cgi;
/cvsroot/mozilla/webtools/bugzilla/process_bug.cgi,v  <--  process_bug.cgi
new revision: 1.199; previous revision: 1.198
done
Checking in globals.pl;
/cvsroot/mozilla/webtools/bugzilla/globals.pl,v  <--  globals.pl
new revision: 1.254; previous revision: 1.253
done

Gerv

Comment 6

[Dave Miller [:justdave]]

*** Bug 201775 has been marked as a duplicate of this bug. ***

Comment 7

[Dave Miller [:justdave]]

*** Bug 234507 has been marked as a duplicate of this bug. ***

Comment 8

[Frédéric Buclin]

*** Bug 215320 has been marked as a duplicate of this bug. ***