720885 - Kuma: Need ability to quickly lock site to prevent changes

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Wiki pages, defect, P2)

Description

[Eric Shepherd [:sheppy]]

Occasionally, a technical or user error occurs that can best be handled if the site is locked to prevent non-admin users from making changes for a while. For example, a technical failure requiring rolling back to a backup. If we file an IT request to do this, changes made in the interim get lost. It would be best if site admins could lock the site to prevent changes until that work is done, so that no changes get made to wind up being lost.

Having an easy way to quickly prevent all non-admin changes without going through IT would be very helpful in cases like these. For example, today we had a case where a combination of admin error and a wiki crash resulted in a large set of docs getting mangled. We're gradually working on manually fixing the problem, but it would have been much easier to just lock the site and ask IT to roll us back instead.

This is something that would be extremely worthwhile to have before we launch, since initial launch is obviously a likely time where problems of consequence might arise in which having this feature would be helpful.

Comment 1

[Les Orchard [:lorchard]]

This could be very handy as a July launch oh-crap button.

Comment 2

[James Socol [:jsocol, :james]]

Something very close to this already exists.

https://github.com/mozilla/kuma/blob/master/settings.py#L769

At the bottom of the settings_local, simply add:

read_only_mode(globals())

Done.

Comment 3

[James Bennett [:ubernostrum]]

So, after poking and prodding and trying various tricks, I don't think there's a simple way to make this happen through a web interface.

The read_only_mode() trick, for example, has two problems. One is that putting it into constance creates a chicken/egg situation where we need settings available to access the dynamic config, but we want to change settings before they're available. A bigger issue is that the read-only middleware indiscriminately disallows POST requests, and the read_only_mode() toggle also turns off auth and sends all DB traffic to a slave. So if we get to a point where we can turn read-only *on*, we won't be able to turn it *off* again since we have no way of logging in and no way of writing to a DB even if we manage that.

I think an all-around better solution would be to roll this into what I suggested in bug 757951 -- rather than a "read-only mode", we should implement a permissions system that lets us ban users from making changes, and have read-only mode basically be a temporary ban of all non-superuser accounts. That's something that's far easier to toggle, and gives us the ability to turn the site back to normal.

Comment 4

[James Socol [:jsocol, :james]]

I should point out the original point of Kitsune's (from Zamboni's) read_only_mode() method, and why I think it's worth keeping around.

http://coffeeonthekeyboard.com/surviving-pac-man-422/

Traffic patterns are unpredictable, and a mostly working site is better than a partly working site is better than a not working site.

The read_only_mode() in settings is not something you can configure via Constance--i.e., making the settings.READ_ONLY setting a Constance setting is insufficient. You really need to ask IT to add the read_only_mode() call to settings_local and deploy.

It is a sledgehammer to stop database writes. Not to slow them down, or restrict them, but to stop them. This is, however, a really nice panic-button to have. (Particularly on support, where occasionally emergency issues cause so many questions to be asked that rep lag gets unbearable.)

So, while I support James' suggestion in the second half of comment 3, I would discourage you from dropping read_only_mode() as it exists now. You hope you never have to use it, but when you do, it's a life-saver. Or an uptime saver, at least.

Comment 5

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/fd334267b165be0b9d8c1cc552b36f4483f34da8
fix bug 720885 bug 671783 - lock site with waffle

* waffle flags 'kumaediting' and 'kumabanned' to control edit views

* ReadOnlyMiddleware to show appropriate messages on 403 page

* check_readonly decorator and tests

https://github.com/mozilla/kuma/commit/8992c2dfb4c33ad257a9f3f7c73e6fcea515f74e
Merge pull request #274 from groovecoder/lock-site-720885

fix bug 720885 bug 671783 - lock site with waffle