279309 - Bugzilla *very* slow when changing > 6 bugs at once

Status: RESOLVED DUPLICATE of bug 215320

(Bugzilla :: Creating/Changing Bugs, defect)

Description

[Michael Hamilton]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

This looks like a regression of bug 233645.
Apache Error:
[Fri Jan 21 14:27:39 2005] [error] [client 192.168.50.49] (70007)The timeout
specified has expired: ap_content_length_filter: apr_bucket_read() failed, referer:
 http://SOMEIPADDRESS/bugzilla/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&product=Trash%20BIN&long_desc_type=substring&long_d
esc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=UNCONFI
RMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&emailassigned_to1=1&emailtype1=substring&e
mail1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=N
ow&chfieldvalue=&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.bug_severity%2Cbugs.priority%2Cbugs.priority%2Cbugs.priority%2Cbugs.priority%2Cbugs.resol
ution&tweak=1

Reproducible: Always

Steps to Reproduce:
1. Do a search which produces more than 6 bugs.
2. Click on Change Several Bugs at Once
3. Select more than 6 bugs, and change their component to another component.
4. Click Commit.

Actual Results:  
It quickly finnishes up 6 bugs then hangs till it times out. Sometimes the
bugzilla system itself becomes unusable. The rest of the OS is fine and mostly idle.

Expected Results:  
quickly process the listed bugs.

Comment 1

[Max Kanat-Alexander]

Weird. I thought bug 233645 fixed this.

Actually, the error message you're reporting is just that the timeout occurred.
The previous bug (bug 233645) was caused by a lot of repeating error messages in
the Apache log. Are there a lot of repeating messages in the log? If so, could
you attach a few lines of it?

Comment 2

[Michael Hamilton]

No repeating errors, just the timeout.
I found a number of similar bugs on this problem, and most of them where marked
duplicates of bug 233645.

Comment 3

[Michael Hamilton]

The other thing to keep in mind is that I did this test with 7 bugs. So perhaps
it enters and error per bug over 6. I will further test.

Comment 4

[Michael Hamilton]

It only produces the one timeout.

Comment 5

[Dan Holmes]

I have found this too.  I posted my error log in the NG.

news://news.mozilla.org:119/ctrgf6$[e-mail]

Comment 6

[Dave Miller [:justdave]]

(In reply to comment #5)
> I have found this too.  I posted my error log in the NG.
> 
> news://news.mozilla.org:119/ctrgf6$[e-mail]

For people who don't have something that resolves news: links, here's the
article on Google:
http://groups-beta.google.com/group/netscape.public.mozilla.webtools/browse_thread/thread/11833cb25a3ebe5d/583199fbb1d74946#583199fbb1d74946

Comment 7

[Frédéric Buclin]

any relation between this bug and bug 215320?

Comment 8

[Dan Holmes]

Attachment: apache error_log

this is a modified copy of my error_log.  I removed the lines about not finding
favicon but that is all i removed.

I was trying to remove a group from a list of bugs.  The list 65 long.	I used
the change several bugs at once.  Any number higher than 10 caused bugz to be
unresponsive to others and it never finished the process.

I know it did some of them because when i went to search again there were only
47 bugs in the group i was trying to remove.

Comment 9

[Max Kanat-Alexander]

Ahh, that's very helpful. What is line 317 of process_bug.cgi on your
installation (and the lines surrounding it)?

Comment 10

[Dan Holmes]

(In reply to comment #9)
> Ahh, that's very helpful. What is line 317 of process_bug.cgi on your
> installation (and the lines surrounding it)?

317 is white space.  with this above it

# Figure out whether or not the user is trying to change the product
# (either the "product" variable is not set to "don't change" or the
# user is changing a single bug and has changed the bug's product),
# and make the user verify the version, component, target milestone,
# and bug groups if so.
if ( $::FORM{'id'} ) {


and this below it.

# Checks that the user is allowed to change the given field.  Actually, right
# now, the rules are pretty simple, and don't look at the field itself very
# much, but that could be enhanced.

my $lastbugid = 0;
my $ownerid;
my $reporterid;
my $qacontactid;


That is probably not helpful.  Want me to attach the file?

Comment 11

[Dan Holmes]

> 
> That is probably not helpful.  Want me to attach the file?

I never said this but i am using 2.19.2.  That is what this bug is versioned
against but i have not updated my bugzilla since the pull with the 2.19.2 tag.

Comment 12

[Christian Reis]

I have a similar problem on a customer site, but his code is based on 2.17.x.

Comment 13

[Frédéric Buclin]

(In reply to comment #9)
> Ahh, that's very helpful. What is line 317 of process_bug.cgi on your
> installation (and the lines surrounding it)?

mkanat meant line 367 of course (read your error log)

Comment 14

[Dan Holmes]

> mkanat meant line 367 of course (read your error log)

    # Return true if they haven't changed this field at all.
367->    if ($oldvalue eq $newvalue) {
        return 1;
    }    
    elsif (trim($oldvalue) eq trim($newvalue)) {
        return 1;
    # numeric fields need to be compared using == 
    } elsif (($field eq "estimated_time" || $field eq "remaining_time") &&
             $oldvalue == $newvalue) {
        return 1;
    }

Comment 15

[Max Kanat-Alexander]

Yep. That's what I thought. We need to check against /\d+/ for:

$oldvalue == $newvalue) {

Comment 16

[Frédéric Buclin]

(In reply to comment #15)
> Yep. That's what I thought. We need to check against /\d+/ for:
> 
> $oldvalue == $newvalue) {

The point is that when changing several bugs at once with the time tracking
group being defined, the "estimated_time" and "remaining_time" fields may
contain "--do_not_change--", responsible for this error message. I will fix that
in bug 215320.

If this is the only reason why Bugzilla is slow, then this is a dupe of bug 215320.

Comment 17

[Frédéric Buclin]

The fix for bug 215320 has been checked in. Reporter, can you reproduce the
problem using the latest CVS version? (note that 2.19.3 does *not* contain this
fix; will be in 2.19.4 or 2.20rc1).

If not, please close this bug as dupe of 215320.

Comment 18

[Dan Holmes]

Attachment: apache error log

I changed about 20 bugs at once this time changing the target milestone.  The
throbber eventually quit trobbing (or whatever it does) and i checked my bugs
an only have of them changed.  This was in the error log.

I did this twice that is why the timeout is in the error log twice.  The second
time all the bugs were changed.

Comment 19

[Dan Holmes]

(In reply to comment #17)
> The fix for bug 215320 has been checked in. Reporter, can you reproduce the
> problem using the latest CVS version? (note that 2.19.3 does *not* contain this
> fix; will be in 2.19.4 or 2.20rc1).
> 
> If not, please close this bug as dupe of 215320.

Comment #18 doesn't include this change.  This is a production system and i saw
that 2.19.3 was spoken of as unstable so i figured the tip was too.  It doesn't
appear to be related to bug 215320.

Comment 20

[Frédéric Buclin]

We cannot help you if you stay with an older version. Couldn't you at least
apply the patch from bug 215320 ? It's a one-liner patch.

Comment 21

[Frédéric Buclin]

Comment on attachment 184344 [details]
apache error log

>The timeout specified has expired: ap_content_length_filter: apr_bucket_read() failed

I don't see what we can do here. Is it really a bugzilla bug?
My vote to remove it from the blocker list. Or even "invalid" if it's not a
bugzilla bug.

Comment 22

[Dave Miller [:justdave]]

Paul, ever seen this?  any clue what it means?  I find lots of references to
people running into it on Google, but no solutions.

Comment 23

[Paul Querna]

Re: The Apache Error.

The CGI pipe is timing out.  This happens when no data is sent by the CGI for
the admin configured Timeout.  This bug might be more common if your Timeout is
set very low. I have seen some people setting it to less than 10 seconds. So, if
bugzilla takes more than 10 seconds between output for any operation, this will
trigger the Apache timeout.

Possible solutions include:

1) Raising the Timeout value. (Default is 300)
2) Sending output more often. (Even whitespace every few seconds would let
apache know that the CGI is still working, and stop it from timing out)
3) Make bugzilla faster :)

Comment 24

[Frédéric Buclin]

(In reply to comment #23)
> Possible solutions include:
> 
> 1) Raising the Timeout value. (Default is 300)

This is something we have no control on it. All we can do here is to mention it
in the docs.

> 2) Sending output more often. (Even whitespace every few seconds would let
> apache know that the CGI is still working, and stop it from timing out)

I guess we cannot do that while we are waiting for sendmail to finish. Actually,
an output is displayed once for each bug. Maybe could we send a whitespace (are
empty output allowed too?) after each N calls to sendmail, i.e. after each group
of N emails sent, with N = ? (would N = 1 be too high?). This would highly
decrease the time between two successive calls to the CGI.

> 3) Make bugzilla faster :)

Thanks for the advice! :-p

Comment 25

[Frédéric Buclin]

*** Bug 253411 has been marked as a duplicate of this bug. ***

Comment 26

[Max Kanat-Alexander]

I'm fairly certain, having read over all the comments again, that this indeed
*is* a dup of bug 215320.

Dan -- if an upgrade to 2.20rc1 doesn't resolve the problem, feel free to leave
a comment here with that information and we can reopen the bug.

*** This bug has been marked as a duplicate of 215320 ***