723755 - QA and deploy BrowserID train-2012.02.02 to production

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Lloyd Hilaiel [:lloyd]]

ChangeLog including issues resolved: 
https://github.com/mozilla/browserid/blob/train-2012.02.02/ChangeLog#L1-17

[QA] Suggested areas of focus for QA:
  * regression of all train-2012.01.18 hotfixes, to ensure they were properly ported
  * testing of different supported locales, no english strings should appear (in dialog).
  * Full and complete regression testing across all devices (i18n touches lots of code)
  * testing of required email features via myfavoriteshow.org
  * testing of persistent sign in features via myfavoritebooze.org
  * testing of primary support using an @eyedee.me address.

[ops] deployment issues:
  * configuration is completely changed.  NODE_EXTRA_CONFIG is no longer supported, instead you can configure via env or by providing JSON configuration files specified as csv in CONFIG_FILES.  examples are in tree at config/*.json and (almost) all supported configuration params are in lib/configuration.js
  * ensure use_minified_resources is true
  * ensure BROWSERID_FAKE_VERIFICATION is true in stage, and not set in prod

Comment 1

[Lloyd Hilaiel [:lloyd]]

version is 0.2012.02.02.1 - sha 9396814

Comment 2

[James Bonacci [:jbonacci]]

BrowserID code deployed to Stage:
https://diresworb.org/ver.txt
f5a74b1 allow host and password to be specified in database configuration
locale svn r101115

OPs is still working on the l10n stuff.

QA accepts this build for testing.

Comment 3

[James Bonacci [:jbonacci]]

Locale stuff should be done.
39+ locales should be in Stage and testable.

Hot Fixes coming in today:
#1 for #1059
1059: Use l10n-all config file when building locale data.
#2 for #1063/#1058
1063: Hotfix 1058 forgot password
1058: Two new issues with the dialogs for 'forgot password' (train-2012.02.02)

Comment 4

[James Bonacci [:jbonacci]]

Hotfixes are pushed.
QA is working on hotfix verification, sanity testing, and low-level load test.

Comment 5

[James Bonacci [:jbonacci]]

QA has done a very thorough sanity/smoke test of Stage.
Load testing continues, noting here a continuous/consistent increase in webhead mysql query time values. OPs is tracking.
Bug verification is up next, then new features testing: localization, primaries...

Comment 6

[James Bonacci [:jbonacci]]

+Zak

Comment 7

[James Bonacci [:jbonacci]]

I have noticed some real problems with flows using Android 2.2 plus the stock browser (not set a default). I will investigate this more tomorrow.

Comment 8

[James Bonacci [:jbonacci]]

Initial load testing has been completed.
Another load test for failover checks will be done on Monday.
Bug verification (36+ closed issues) has been completed.
Testing of primaries is in progress.
Testing of localization is also in progress.

Comment 9

[Lloyd Hilaiel [:lloyd]]

petef, a hotfix ready for immediate deployment for issue GH-1087 - 570ba72c6974e88e87ae0e2078fcafcb55dc909c - 0.2012.02.02.4 - https://github.com/mozilla/browserid/commit/570ba72c6974e88e87ae0e2078fcafcb55dc909c

This is a 2 line CSS tweak for IE users, jbonacci digs it: http://irclog.gr/#show/irc.mozilla.org/identity/55304

Comment 10

[James Bonacci [:jbonacci]]

HotFix pushed:
570ba72 update changelog and version with fix for issue #1087
locale svn r101559

Comment 11

[James Bonacci [:jbonacci]]

HotFix #3 tested and passed.

Load testing and locale testing continue through tomorrow.

Also, for reference:
https://intranet.mozilla.org/Services/Ops/ChangeWindow_20120215

Comment 12

[James Bonacci [:jbonacci]]

QA signs off on this train.
There is still a lot of localization testing that needs to happen, but it is not blocking the deployment to Production.
We will continue this on the next train.
Also, QA is concerned about the following issue showing up in Production:
1057: stage JS error for db_LB
So, we will be tracking that carefully through the deployment...

Comment 13

[James Bonacci [:jbonacci]]

One other issue came up, covered in this Bugzilla ticket:
Bug 727578 - BrowserID: The sv directory missing in Stage
OPs pushed a final RPM candidate to Stage to cover this before we started deployment to Production (with the same change pushed there as well).

Comment 14

[James Bonacci [:jbonacci]]

This ticket is to remain open while we continue to debug/resolve the following GitHub issue:
Issue 1129: the cache issue with rollout of train-2012.02.02

Comment 15

[Pete Fritchman [:petef]]

This is pushed to production.  We pushed a 302 out for the dialog.js thing, and I'm not sure we can do too much more (and the 30-hour possible impact window is almost over).

Comment 16

[James Bonacci [:jbonacci]]

Closing down and moving on to next train.