727995 - QA and deploy BrowserID train-2012.02.16 to production

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Lloyd Hilaiel [:lloyd]]

ChangeLog including issues resolved: 
https://github.com/mozilla/browserid/blob/train-2012.02.16/ChangeLog#L1-17

[QA] Suggested areas of focus for QA:
  * normal stuff - resolved issue verification, load testing, and full regression testing
  * given issue 1129 and caching changes, check cache headers of important resources to ensure no regression.
  * test failure modes with cookies disabled on all environments

[ops] deployment issues:
  * no changes present which should affect deployment

Comment 1

[Lloyd Hilaiel [:lloyd]]

petef - this is train-2012.02.16 - sha fffb471f - v0.2012.02.16.1

Comment 2

[James Bonacci [:jbonacci]]

RPM: browserid-server-0.2012.02.16-1.el6_101802.x86_64.rpm

Comment 3

[Pete Fritchman [:petef]]

on stage, ready for QA.

Comment 4

[Pete Fritchman [:petef]]

there were some RSBAC-related deploy issues on the sweb layer (dbwriter) which is better now (dbwriter now needs to talk in /opt/browserid/resources, rsbac was preventing).

Signing in now on beta.myfavoritebeer.org gets me:

We are very sorry, there has been an error!
To retry, you will have to close this window and try again.
    Action: Checking Authentication

which is weird, because I only see success codes on server requests.  looking in my browser, the only error I see is:

[09:51:34.426] Unexpected end of file while searching for end of @media or @-moz-document rule.  Unexpected end of file while searching for '}'. @ https://diresworb.org/v/70dcf81bb6d158f3ddfc1aa4cca0c1e2/production/dialog.css:1

Comment 5

[Pete Fritchman [:petef]]

(In reply to Pete Fritchman [:petef] from comment #4)
> there were some RSBAC-related deploy issues on the sweb layer (dbwriter)
> which is better now (dbwriter now needs to talk in /opt/browserid/resources,
> rsbac was preventing).
> 
> Signing in now on beta.myfavoritebeer.org gets me:
> 
> We are very sorry, there has been an error!
> To retry, you will have to close this window and try again.
>     Action: Checking Authentication
> 
> which is weird, because I only see success codes on server requests. 
> looking in my browser, the only error I see is:
> 
> [09:51:34.426] Unexpected end of file while searching for end of @media or
> @-moz-document rule.  Unexpected end of file while searching for '}'. @
> https://diresworb.org/v/70dcf81bb6d158f3ddfc1aa4cca0c1e2/production/dialog.
> css:1

restarted browser and can't reproduce the error screen.

anyway, QA, stage should be ready for you.

Comment 6

[James Bonacci [:jbonacci]]

QA accepts this train for testing:
https://diresworb.org/ver.txt
fffb471 document changes in train-2012.02.16, also fix broken dates in past changelog entries
locale svn r101802

Comment 7

[James Bonacci [:jbonacci]]

Processes, startup times, logs all look as expected.
Quick sanity test was good.

More sanity testing will continue today.
Also, QA will be running a load test over the three-day...

Comment 8

[John Morrison [:jrgm]]

I've started a 1M ADU loadtest from client{5,6,7,8}.scl2.smc. Will run over the 
weekend, but may restart the clients after a day or two.

Comment 9

[John Morrison [:jrgm]]

restarted 1M ADU loadtest from client{5,6,7,8}.scl2.smc after the clients died at about 2012-02-19 22:00

Comment 10

[James Bonacci [:jbonacci]]

Second load test is still in progress...
Sanity testing is done.
Local installs and configurations (unit tests, load, AWS) are done.
Bug verification is in progress.

Current list of locales (needs verification):
"af", "ca", "cs", "da", "de", "el", "en-US", "es", "et", "eu",
"fi", "fr", "fy", "ga", "hr", "it", "lij", "nl", "pa", "pl",
"ru", "sk", "sl", "sq", "sr", "sv", "tr", "zh-CN", "zh-TW

Comment 11

[James Bonacci [:jbonacci]]

Bug verification for this train is done, although there are a lot of closed bugs from previous trains that will try to get to during this train.

Functional testing continues...

Comment 12

[James Bonacci [:jbonacci]]

New RPM to cover all locales in config/l10n-all.json
1177: more rpm fixes: instead of pulling in all config/, just l10n-all.json

https://diresworb.org/ver.txt
Updated 2/22/2012:
4612fd9 include configs in RPM, so we can reference l10n-all.json in stage
locale svn r102046

Comment 13

[James Bonacci [:jbonacci]]

Current config/l10n-all.json from a local install - 
{
  "supported_languages": [
    "af", "ca", "cs", "da", "de", "el", "en-US", "eo", "es", "es-MX",
    "et", "eu", "fi", "fr", "fy", "ga", "gd", "gl", "he", "hr", "hu",
    "it", "ja", "lij", "lt", "ml", "nl", "pa", "pl", "pt", "pt-BR",
    "rm", "ro", "ru", "sk", "sl", "son", "sq", "sr", "sv", "tr", "zh-CN",
    "zh-TW",
    "it-CH", "db-LB"
  ]
}

Comment 14

[John Morrison [:jrgm]]

2nd loadtest clients passed away about 4am PST this morning, not restarting. 
In general the loadtest passes in that the server is fine at this level of load. 
There are sporadic clientside errors, but seem to happen more as the load_gen ages,
so for now, taking that as a side-effect of the load_gen mem issues.

Comment 15

[Pete Fritchman [:petef]]

deploy is this afternoon.

Comment 16

[James Bonacci [:jbonacci]]

QA does indeed sign off on this train!
Already assigned to petef for afternoon deployment...

Comment 17

[James Bonacci [:jbonacci]]

ChangeWindow for today:
https://intranet.mozilla.org/Services/Ops/ChangeWindow_20120229

Comment 18

[Pete Fritchman [:petef]]

browserid-server-0.2012.02.16-2.el6_102380.x86_64.rpm built and ready

Comment 19

[Pete Fritchman [:petef]]

deployed

Comment 20

[James Bonacci [:jbonacci]]

Verified deployment to Production.