Entering something into shutdownhtml doesn't completely disable Bugzilla as there are some queries that are run before bugzilla calls PutHeader() [which is where shutdownhtml takes effect]. This causes issues with the number of available connections when bugzilla is trying to do things like resync the shadow db. Also, when bugzilla is shut down, you see all the normal page headers (FE, the last time bugzilla was shut down I was in the middle of making a change and the page header said that the bug was processed, but immediatly underneath that it said bugzilla was down).
As much as I hate to add something else to 2.14, this causes issues when mozilla.org is attemping to recover from unplanned downtime. It may also be contributing to the failure of resyncing the shadow database.
Well, if syncshadowdb actually did shut down Bugzilla when they resyncing the database, that might be right. But they don't, the backup just locks the tables. See bug #75840.
That's true and it is something to be addressed. I don't know for sure if this is what's causing the nightly resync to fail, but it seems likely. If someone else agrees that this is the likely cause, we should probably up the target for bug 75840 to 2.14 as it is causing mozilla.org many headaches. The bulk of what this report is about was discovered when Dawn manually put some text in shutdownhtml this morning so she could rebuild the shadow database. She had to shut down httpd in order to kill all the stray MySQL connections that kept appearing (she was getting 'too many users' errors).
As far as I can tell the nightly process locks down the database just fine. Maybe there is a bug there that you can still access certain tables, but I doubt it. If that was the case, the correct fix is to lock the tables down properly, not to shutdown Bugzilla. For databases of sufficiently small size, shutting down is not appropriate and the queries can just wait for a minute or two. As far this bug, it's certainly a bug, but my assertion was the two have no relationship and hence this isn't a 2.14 blocker.
When making this patch, I went under the assumption that when bugzilla is shut down, the only pages that should be accessed are editparams.cgi and doeditparams.cgi (it is possible for the administrator to log into bugzilla using only editparams.cgi). It looks like that's what $ignoreshutdown was attempting to do, but it also allowed access to any page that required a login (as logging in was considered a reason to ignore the shutdown param). To that end, this patch will not send anything to the SQL server unless the page being viewed is editparams.cgi or doeditparams.cgi. It also always sets the page title to "Bugzilla is Down" when there is something in the shutdownhtml param.
r=justdave in IRC. Checked in.
Was looking at the code for syncshadowdb (re bug 75840) and relized that this patch would prevent the shadowdb from being rebuilt if bugzilla is shut down. The fix is easy, we just have to tell SendSQL() that it's OK to pass on the query if the command being run is syncshadowdb.
I would expect that it wouldn't run if Bugzilla is shut down.
r=justdave it's in.
Moving to Bugzilla product