895077 - please deploy browserid train-2013.07.17 to staging

Status: RESOLVED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

sha: 978c603ba0a5fd5f6531c952aba0bbdfb97f447e

ChangeLog: https://github.com/mozilla/browserid/blob/train-2013.07.17/ChangeLog#L1-L9

Schedule: http://personatra.in/#2013.07.17

Comment 1

[John Morrison [:jrgm]]

travis build passing: https://travis-ci.org/mozilla/browserid/builds/9202136

Comment 2

[John Morrison [:jrgm]]

There is a problem with building this rpm - https://github.com/mozilla/browserid/issues/3683, so hopefully I can get an rpm for this built tomorrow when that issue is fixed.

Comment 3

[John Morrison [:jrgm]]

Okay, problem is thought to be resolved.

New SHA on train-2013.07.17 is 495c12fa85c0039598d79a2b71c87c98b5c66781 

Building rpm now.

Comment 4

[John Morrison [:jrgm]]

Okay, new rpm:

> Build command: sh -x ./build.sh browserid train-2013.07.17 495c12fa85c0039598d79a2b71c87c98b5c66781

> rpm md5sum: 4d72f6729ac1ce86eb717d715695a6f2

> Output rpm on r6: /home/jrgm/workspace/browserid/rpmbuild/RPMS/x86_64/browserid-server-0.2013.07.17-2.el6_118220.x86_64.rpm

Comment 5

[Gene Wood [:gene]]

Stack created and DNS switched to it :

https://github.com/mozilla/identity-ops/wiki/Interim-AWS-Stack-List#stack-0727--us-west-2--stage

Comment 6

[John Morrison [:jrgm]]

Gene: DNS does not appear to be pointing at the 0727 stack.

$ dig @8.8.8.8 login.anosrep.org CNAME +short
w-anosrep-org-0725-514332909.us-west-2.elb.amazonaws.com.

$ curl -k https://w-anosrep-org-0725-514332909.us-west-2.elb.amazonaws.com/ver.txt
b419bd2 Merge pull request #3653 from mozilla/train-2013.07.03-i18n-abide-0.0.8
locale svn r118142

$ git branch --contains b419bd2
  train-2013.07.03

Comment 7

[Gene Wood [:gene]]

Damn. Dynect was stuck on the 3rd publish page (it takes a "save", "publish", "publish") for the change to take effect. Must've missed the last page. Sorry about that, it's switched now.

Comment 8

[John Morrison [:jrgm]]

Yep, fixed.

$ dig @8.8.8.8 login.anosrep.org CNAME +short
w-anosrep-org-0727-1042885409.us-west-2.elb.amazonaws.com.

Comment 9

[John Morrison [:jrgm]]

Gene, I	have a new build of train-2013.07.17 to	pickup this fix:

- https://github.com/mozilla/browserid/pull/3732

Please build new *stage* stacks using this rpm.

This also picks up L10N changes to most recent - r118455

New SHA: bc4be73a9d9eb73f47461882cc2956c9b4c978fd

build command: ./build.sh browserid train-2013.07.17 bc4be73a9d9eb73f47461882cc2956c9b4c978fd

md5sum of output rpm : 5a7ba134a6eee83db2dae91b78197677

output rpm on r6: /home/jrgm/workspace/browserid/rpmbuild/RPMS/x86_64/browserid-server-0.2013.07.17-3.el6_118455.x86_64.rpm

Comment 10

[Gene Wood [:gene]]

Ok, new stack is built (0730) and DNS is pointed to it.

Comment 11

[John Morrison [:jrgm]]

Gene, I appear to getting the wrong output of /ver.txt 

$ curl -k https://w-anosrep-org-0730-478183029.us-west-2.elb.amazonaws.com/ver.txt

495c12f Merge pull request #3687 from jrgm/train-2013.07.17-fix-l10n
locale svn r118220

That should be:
bc4be73 Merge pull request #3734 from jrgm/bump-ver-train-2013.07.17
locale svn r118455

Is the right rpm installed on stack 0730?

Comment 12

[Gene Wood [:gene]]

No, my fault. I'd built the new AMI's using the new RPMs but failed to reference them in the stack definition so the stack came up with the original AMIs.

I've build a new stack 0731 and pointed DNS to it, confirming that it has the right AMIs now.

Comment 13

[John Morrison [:jrgm]]

Yep, /ver.txt shows bc4be73 and r118455 now. Thanks.

Comment 14

[John Morrison [:jrgm]]

Hi, Gene. I have an amended hotfix for train-2013.07.17	to pick	up fixes for mixed case	in domain names	with sideshow.

The new SHA: 6ade72755bac774becda2060d72620253ecda538

build command (keeping the same locale):  sh -x ./build.sh browserid train-2013.07.17 6ade72755bac774becda2060d72620253ecda538 118455

output rpm on r6: /home/jrgm/workspace/browserid/rpmbuild/RPMS/x86_64/browserid-server-0.2013.07.17-4.el6_118455.x86_64.rpm

md5sum of rpm: 6a1b8ff2612e7ed44504403157994c0d

Can you	please build a new stack with the rpm above (and the previous stage stack 0731 can be discarded	after that).

(Meta question:	do you want these requests in separate tickets?).

Comment 15

[Gene Wood [:gene]]

New stack 0801 built and DNS has been pointed to it.

Comment 16

[John Morrison [:jrgm]]

Hey Gene, I have another small fix on top of train-2013.07.17. Can you build a new stack in stage (sorry) for this build.

new git SHA: f4012cb2d8033f7e314afee549a7ec9380592b92

build command: sh -x ./build_private.sh browserid train-2013.07.17 f4012cb2d8033f7e314afee549a7ec9380592b92 118455

md5sum of rpm: fe1509051753d4ed94cf37f5b44a3222

output rpm on r6: /home/jrgm/workspace/browserid/rpmbuild/RPMS/x86_64/browserid-server-0.2013.07.17-5.el6_118455.x86_64.rpm

Comment 17

[Gene Wood [:gene]]

stack 0802 built and DNS has been pointed to it

Comment 18

[John Morrison [:jrgm]]

train-2013.07.17:
https://bugzilla.mozilla.org/show_bug.cgi?id=895077
Target sha from above is f4012cb, locale svn r118455
Against aws stage stack: Stack 0802 : Staging

Notes:
- No Travis-CI due to private repo. Tests run on awsbox pass with no failures.
- for i in `seq 200`; do curl -s https://login.anosrep.org/ver.txt; sleep 0.5; done | sort | uniq -c
 200 f4012cb Merge pull request #11 from jrgm/bumpver-and-changelog-train-2013.07.17
 200 locale svn r118455
- for i in `seq 200`; do curl -s https://login.anosrep.org/sign_in | egrep 'Current|dialog.js|dialog.css'; sleep 0.5; done | sort | uniq -c
 200         <script src="https://static.login.anosrep.org/v/f98278d58f/production/en/dialog.js"></script>
 200     <link href="https://static.login.anosrep.org/v/844b6f3f18/production/ie8_dialog.css" rel="stylesheet" type="text/css">
 200    - Current Commit: https://github.com/mozilla/browserid/commit/f4012cb
 200   <link href="https://static.login.anosrep.org/v/2bf9f39515/production/en/dialog.css" rel="stylesheet" type="text/css">
- for i in `seq 200`; do curl -s https://login.anosrep.org/.well-known/browserid; echo; sleep 0.5; done | sort | uniq -c
  returns a public-key, no 500 error.
 200 {"public-key":{"algorithm":"RS","n":"24675054630711933868774826698221262583022959838922715089308093557771535226157283798420755285528809601560803946672224335269223905409755784163774694226606575102480326358073775068815605414664478919457740194999867465701123595549248903773608746524260866171172164883179416750582603848947125459918437866751902576315529350110189557631388145098647767060983460212352836162158121131461597572429103480324094008478124008821851189122205273102639257759224296668314172945539499932445070143661155462045140023832666883732781606060636986328479344187760928917781755007104902663769321280335237300185799120549336300548062873067162015605387","e":"65537"}}
- for i in `seq 100`; do curl -s https://yahoo.login.anosrep.org/ver.txt; sleep 1; done | sort | uniq -c
 100 0a81d5f Preparing train-2013.05.29 Release 2
 100 locale svn r116978
 - should match the production version if not explicitly shipping new bigtent.
- for i in `seq 200`; do curl -s https://yahoo.login.anosrep.org/.well-known/browserid; echo; sleep 0.5; done | sort | uniq -c
 200     "authentication": "/authentication",
 200     "provisioning": "/provision"
 200     "public-key": {"algorithm":"RS","n":"13688119101238081271351442723674676434770300234892243778987099842550314810167052836204245268309831620476763737802838143890374809474754776358838371280337039881742878190936304894032210920493992319741500577190457494000154350446141267951522092793262048050476287115644745783766060952789405240481567205576079217990023203919323544773941131073409575122052543705727693268243600671829358376488926602543663905612053137845422922085587845164580139485152516981679161382863876414611998324028321872020967684896382955365818179310712886086861055505578548278384846483021345326244461254362577471864446059981515806090589078087126108150453","e":"65537"},
- for i in `seq 100`; do curl -s https://gmail.login.anosrep.org/static/ver.txt; sleep 1; done | sort | uniq -c
 100 [b18e22e] Hardcode some version data temporarily
 100 locale svn r118029
- for i in `seq 200`; do curl -s https://gmail.login.anosrep.org/.well-known/browserid; echo; sleep 0.5; done | sort | uniq -c
 200     "algorithm": "RS",
 200     "e": "65537"
 200     "n": "10661385041953378496139948687004413045689056137872786573403286272712403651213837166316146560955495233274293537941687019063378161041101113704937379601112418914535388044172652578240469865436730455466265320149658626701551157820806850950005487062694743206849757155434689255702047500868009541193522666199905549542119101523140140988057589714319229970972170558948226479555013246787106394397572261165519657879565603817052916139685663788090947848847332714867891205515158025589518906909519301644533326018153939644567892449993613227347775733164477830664507357001513925251346965410550082497540885842782126872078633080190268381227",
 200   "authentication": "/authenticate",
 200   "provisioning": "/provision"
 200   "public-key": {
- for f in `get_hosts 0802`; do ssh -oStrictHostKeyChecking=no $f node --version; done | sort | uniq -c
     19 v0.8.24

- for f in `get_hosts 0802`; do ssh -oStrictHostKeyChecking=no $f svstat /service/*; done | sort
- for f in `get_hosts 0802`; do echo $f; \
ssh root@$f "ps -eo pid,lstart,bsdtime,pcpu,pri,rsz,vsz,command | grep 'node bin/' | grep -v grep"; done
  shows the processes restarted and not flapping

- On 5 iterations of 17 suites for these platforms (more flaky in 07.17 than 07.03)
  linux_firefox_21 - 79/85 suite runs pass
  win7_chrome        68/85 suite runs pass
  win7_firefox_21    80/85 suite runs pass
  win8_ie_10         76/85 suite runs pass
  win7_ie_9          75/85 suite runs pass
  winxp_ie_8         71/85 suite runs pass
So, passing but with flakiness on some tests.
The biggest difference is that Chrome is failing always for two tests:
https://github.com/mozilla/browserid/issues/3747

- load_gen @ 250K ADU showed good response times, no RSS growth and few errors.

Comment 19

[John Morrison [:jrgm]]

Hey Gene, I have another small fix on top of train-2013.07.17. Can you build a new stack in stage.

new git SHA: 849f8df5918d52a3720544feb8c0d20e65e294d1

sh -x ./build_private.sh browserid train-2013.07.17 849f8df5918d52a3720544feb8c0d20e65e294d1 118455

md5sum of rpm: 44db517df7fba10efd53a2330a076d8f

output rpm on r6: /home/jrgm/workspace/browserid/rpmbuild/RPMS/x86_64/browserid-server-0.2013.07.17-6.el6_118455.x86_64.rpm

Comment 20

[Gene Wood [:gene]]

stack 0803 built and dns updated