787708 - Deploy train-2012.08.17 to staging environment

Status: RESOLVED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Lloyd Hilaiel [:lloyd]]

train: train-2012.08.17
git SHA: 80480e1a480cc9f85a425eeee718353e914f0b6d
http://ozten.com/random/identity/browserid-server-0.2012.08.17-1.el6_108803.x86_64.rpm
rpm md5: 77c0888e03fe9364eff909c7da1d5942

Note: This rpm was built on a non-RHEL box by ozten.  If there are problems and we must build RPMs on adm1, we needs some software installed on that box (expat-devel and node-0.6.17 at least).

Look ok? (and yes, I know we need to move to automatic generation of RPMs and have a better hand-off location)

Comment 1

[James Bonacci [:jbonacci]]

QA Release Wiki:
https://wiki.mozilla.org/QA/Browser_Technologies/Services/Releases/BrowserID_08172012_Stage

QA Test Spreadsheet:
https://docs.google.com/spreadsheet/ccc?key=0As8GUPwNNWMFdDRCMVo3bWJQNjBVZk8wUTNUb21DdGc#gid=0

Comment 2

[Bob Micheletto [:bobm]]

The RPM has been added to mrepo.

Comment 3

[Austin King [:ozten]]

jrgm and gene noticed locale_directory has been removed as a valid config property.

This type of detail should go into this deployment ticket.

I think this was an earlier train that was derailed, which is probably how it fell through the cracks.

Comment 4

[Austin King [:ozten]]

static_url is a new config property.

This is present in config/production.json, but should also be called out explicitly (noting for later process improvement)

Comment 5

[Gene Wood [:gene]]

I made these changes to svn.mozilla.org/sysadmins
r46864
r46865

for production.json.stage

Comment 6

[Austin King [:ozten]]

Testing http://beta.123done.org/ I get an error after entering a password.

Looks like a 503 to /wsapi/list_emails

    Response Text: Service Unavailable: database unavailable

Comment 7

[Gene Wood [:gene]]

Staging is updated with train 08.17

Comment 8

[Gene Wood [:gene]]

web1.idweb was still "draining" in zeus due to how bidpush works.
web2.idweb and web3.idweb : daemontools wasn't able to start browserid-static reporting : "fatal: unable to start browserid-static/run: file does not exist?". I ran "sudo svc -u /service/browserid-static/" on each and got them working.

We don't have any browserid-static monitors in nagios.

Comment 9

[Austin King [:ozten]]

@lloyd - are these details right?

Note: Before prod schema migration, please do:

1) Note current value of max_query_time_ms
2) set max_query_time_ms to 20000 and restart services

max_query_time_ms is in milliseconds, we want 20 second timeouts

3) Run migrations
4) set max_query_time_ms to original value and restart services

Services which use DB queries - browserid, dbwriter

Comment 10

[Austin King [:ozten]]

Please ignore Comment #9

Due to the small window of time, it's not worth touching configs and restarting servers twice.

Comment 11

[John Morrison [:jrgm]]

Here's my status. I am pretty much signing off on this at this point, but will
be continuing test work for the next few days:
- unit tests pass (including run frontend tests on firefox
  {ESR,15,16,17,18}, safari, chrome, opera, IE {8,9,10}, safari mobile,
  android stock browser, android chrome, android firefox on android 2.2,
  android 4.0, osx 10.7, winxp, win7, win8, linux/ubuntu)
- extended load test complete at up to 1.3M ADU with no noted issues. RSS
  or node process steady at <64MB. (Actually, the zeus upgrade is much
  better than the previous version).
- bug verifications complete for current and prior trains (but I am going
  to continue going back over major items from this set of trains today,
  tomorrow and a bit on the weekend).

Comment 12

[Austin King [:ozten]]

https://ozten.com/random/identity/browserid-server-2.2012.08.17-1.el6_109133.x86_64.rpm

Updates - latest SVN strings and %post daemon restart hook

Comment 13

[John Morrison [:jrgm]]

QA signs off on deploying train-2012.08.17 to production, but there are two items that need to happen first:
1) respin and stage deploy of L10N bits. 
2) A detailed change plan for prod, especially the timing of database alters.

Comment 14

[Austin King [:ozten]]

bobm or atoll - as gene takes some rest, can one of you push the latest rpm out to stage for jrgm?

Comment 15

[Bob Micheletto [:bobm]]

I am investigating this.

Comment 16

[Bob Micheletto [:bobm]]

The latest RPM has been pushed.

Comment 17

[John Morrison [:jrgm]]

I don't have localized strings on stage (e.g., if I set my language to be 'de', I don't get German in the dialogs). 

https://login.anosrep.org/ver.txt says:
80480e1 lockdown v0.0.2 - uses a more specific ENV var than PORT to fix conflict in deployment environment
locale svn r109133

Shouldn't that SHA be different?

Comment 18

[Austin King [:ozten]]

(In reply to John Morrison [:jrgm] from comment #17)

> I don't have localized strings on stage (e.g., if I set my language to be 'de', I 
> don't get German in the dialogs). 

there are no locales on stage but en localizers use http://translate.123done.org/ to test their work

Same SHA, no new code. (Actually I should have built the rpm with the rpm spec changes, but I didn't, that is why there is no SHA change.)

Comment 19

[Gene Wood [:gene]]

Looks like this is done