Please deploy server-storage 1.15-8 and server-core 2.12-8 to sync server stage environment. Build command: make build PYPI=http://pypi.build.mtv1.svc.mozilla.com/simple PYPIEXTRAS=http://pypi.build.mtv1.svc.mozilla.com/extras PYPISTRICT=1 SERVER_STORAGE=rpm-1.15-8 SERVER_CORE=rpm-2.12-8 CHANNEL=prod RPM_CHANNEL=prod build_rpms This includes a single bugfix over the previous release, for Bug 853032. But since this bug deals with mysql connection-handling, I would still recommend a full 24-hour loadtest before rolling it out to prod. To test whether the bug has been resolved, restart mysql on one of the db machines while a loadtest is in progress. We should see a brief spike of 503s until the database comes back up, then a return to successful request handling. If the bug is not fixed, the failures will continue even after mysql comes back up. Since there are no user-facing problems associated with this bug, there's no great urgency from my end to get this out to production. Ops have requested a new tag so it can be rolled out at their convenience.
Yes. QA will do the usual harm to Stage before this goes to Production. I will coordinate with OPs on the testing since we need to restart mysql on a db.
Oh, yikes, I linked the wrong bug here. Copy-paste fail! It should be Bug 857323.
This version has been pushed to Stage, and is ready for QA.
Verified the following on the webheads: processes logging versions: python26-services-2.12-8.noarch python26-syncstorage-1.15-8.noarch Moving on to a 24-hour load test...
Results of load test were mixed. Nothing new to report, but nothing really to fix or break the release. So, not worth rolling out to Production. :rfkelly to continue investigations through https://bugzilla.mozilla.org/show_bug.cgi?id=857323
Verified as finished, but not a complete push to Production.