471880 - More scripts should use the shadow DB instead of the master DB

Status: RESOLVED FIXED

(Bugzilla :: Bugzilla-General, defect, P2)

Description

[Frédéric Buclin]

This is a copy of my last email sent to developers@:

> Yeah, the master DB for Bugzilla gets more traffic than the slave DB
> does on bmo right now, which makes it kind of meaningless to add slaves,

IMO, that's because query.cgi uses the master DB to populate the search form, and buglist.cgi prepares everything using the master DB, and only shifts to the shadow DB at the very end when it's time to run the huge query. I think both should shift to the shadow DB, the only exceptions being when they write to the DB (when updating the default query or deleting/updating a saved search).

I also suspect more and more third-party tools call config.cgi to know the config of Bugzilla, but this script uses the master DB too, despite it's pure read-only. So we could shift to the shadow DB without problem.

describecomponents.cgi and describekeywords.cgi are also good candidates to use the shadow DB. It doesn't matter if the number of bugs reported by keyword is off by a few minutes, really.

Maybe more controversial is show_activity.cgi, which *could* eventually use the shadow DB too. Note that BugMail.pm doesn't call show_activity.cgi to generate bugmail, so this wouldn't break anything.

Other scripts look fine to me.

Comment 1

[Frédéric Buclin]

Oh, and now that Bugzilla supports Atom Feeds, many people use them to track changes in their favorite (saved) searches. In my case, Firefox 3 runs 8 saved searches every XX minutes (no idea what the refresh time in Firefox 3 is). Multiply this by the number of users also having feeds in their favourite browser and you will notice that buglist.cgi generates a lot of query, many of which happen on the master DB.

Comment 2

[Frédéric Buclin]

mkanat suggests that show_bug.cgi also uses the shadow DB for logged out users.

Comment 3

[Frédéric Buclin]

(In reply to comment #0)
> and buglist.cgi prepares everything using the master DB, and only shifts to the
> shadow DB at the very end when it's time to run the huge query.

I take that back. Most of the internal stuff done by buglist.cgi is unrelated to the DB, but mainly has to do with parsing CGI parameters and preparing the query to run later. So this script seems fine. /me continues investigation.

Comment 4

[Frédéric Buclin]

Attachment: patch, v1

justdave said he has a test installation with a slave.

Comment 5

[Max Kanat-Alexander]

Comment on attachment 356019 [details] [diff] [review]
patch, v1

I think in show_activity we could do Bugzilla::Bug->check on the shadow DB. Everything else looks fine to me, though.

Comment 6

[Frédéric Buclin]

I'm going to commit my patch without justdave's testing.

Comment 7

[Frédéric Buclin]

Checking in config.cgi;
/cvsroot/mozilla/webtools/bugzilla/config.cgi,v  <--  config.cgi
new revision: 1.31; previous revision: 1.30
done
Checking in describecomponents.cgi;
/cvsroot/mozilla/webtools/bugzilla/describecomponents.cgi,v  <--  describecomponents.cgi
new revision: 1.39; previous revision: 1.38
done
Checking in describekeywords.cgi;
/cvsroot/mozilla/webtools/bugzilla/describekeywords.cgi,v  <--  describekeywords.cgi
new revision: 1.22; previous revision: 1.21
done
Checking in query.cgi;
/cvsroot/mozilla/webtools/bugzilla/query.cgi,v  <--  query.cgi
new revision: 1.183; previous revision: 1.182
done
Checking in request.cgi;
/cvsroot/mozilla/webtools/bugzilla/request.cgi,v  <--  request.cgi
new revision: 1.49; previous revision: 1.48
done
Checking in show_activity.cgi;
/cvsroot/mozilla/webtools/bugzilla/show_activity.cgi,v  <--  show_activity.cgi
new revision: 1.26; previous revision: 1.25
done
Checking in show_bug.cgi;
/cvsroot/mozilla/webtools/bugzilla/show_bug.cgi,v  <--  show_bug.cgi
new revision: 1.56; previous revision: 1.55
done

Comment 8

[Reed Loden [:reed]]

How possible would it be to backport this to 3.2? This is a performance fix, so it should be acceptable on the branches.

Comment 9

[Max Kanat-Alexander]

(In reply to comment #8)
> How possible would it be to backport this to 3.2? This is a performance fix, so
> it should be acceptable on the branches.

  Possible, but too risky, I think. Particularly as it may silently cause strange things to happen with customizations. You're welcome to backport it for bmo, though.

Comment 10

[Max Kanat-Alexander]

Added to the release notes for Bugzilla 3.4 in bug 494037.