960284 - please deploy browserid train-2013.10.23 to production

Status: RESOLVED WONTFIX

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[John Morrison [:jrgm]]

Gene:

Now, new and improved.

Please build browserid-server with sha 
3c066b17681f09f86c49e0080d757058348643ad
and locale svn version 122101, from the browserid_private repo (on the train-2013.10.23, to be complete).

And then build a __production__ stack with that rpm.

Comment 1

[Gene Wood [:gene]]

Stacks 0121 are up in both datacenters. Take a look and I'll put traffic on them.

Comment 2

[Edwin Wong [:edwong]]

jrgm and karl are looking at it.

Comment 3

[John Morrison [:jrgm]]

Gene: you can switch DNS traffic to the 0121 stacks.

Comment 4

[Edwin Wong [:edwong]]

setting needinfo on gene

Comment 5

[Gene Wood [:gene]]

2:46pm stacks 0121 have been brought live in DNS

Comment 6

[Gene Wood [:gene]]

John report marketplace issues. 4:55pm PST I rolled back to 1203 stacks

Comment 7

[François Marier [:francois]]

It looks like a problem with this security fix:

  https://github.com/mozilla/browserid_private/commit/11e48d48b7eb75b674408a71159c696fe711368d

I have not yet been able to reproduce the problem with Marketplace, but if removing the above commit fixes it, I think we should just push that security fix to the next train and push that revised stack to prod.

Comment 8

[Dan Callahan [:callahad]]

:gene, please build a production stack from SHA 2be2d217c93e6a12e74bd59eb48edaa64601d797 in browserid_private, and locale svn version 122101.

Thank you,
-Callahad

Comment 9

[Edwin Wong [:edwong]]

Gene: can you update us on your status on this so QA can schedule testing and prod deploy.  Thanks.

Comment 10

[Gene Wood [:gene]]

I'm planning to build these stacks today. :callahad's request came in at 8:44pm Friday night and I had not seen it until now, Monday morning.

Comment 11

[Gene Wood [:gene]]

To clarify, this from a different branch

Commit : 2be2d217c93e6a12e74bd59eb48edaa64601d797
Branch : train-2013.10.23-release

Comment 12

[Dan Callahan [:callahad]]

Yep! Sorry I didn't make that clear. Thanks!

Comment 13

[Gene Wood [:gene]]

Stacks 0127 are up in both datacenters. QA, take a look

Comment 14

[John Morrison [:jrgm]]

:callahad - not understanding what is being shipped here. Tests do not pass.

$ git symbolic-ref --short -q HEAD ; git rev-parse --short HEAD
train-2013.10.23-release
2be2d21

$ node tests/unverified-email-test.js
Setting NODE_ENV to test_json to test with the local JSON database
To test with a local mysql database, setup mysql and set NODE_ENV to test_mysql
· · · ···· · · ·· ·verifier (86283): error: uncaughtException: Object #<Object> has no method 'currentSupport' ...
...
stack=[TypeError: Object #<Object> has no method 'currentSupport',
  at /Users/jmorrison/github/mozilla/browserid_private/lib/verifier/certassertion.js:183:19,
  at /Users/jmorrison/github/mozilla/browserid_private/node_modules/jwcrypto/lib/cert.js:183:7
·

Closing this bug. File a new ticket.

Comment 15

[Dan Callahan [:callahad]]

This train is cursed. There's been a ridiculous amount of cherry-picking in the commit graph, and it's making it hard for us to reason about what we're actually doing. The lack of Travis on the private repo also bit us, as John found.

francois: I suggest we abandon 2013.10.23 (change this from INVALID to WONTFIX) and prepare a new train based on the current tip of the dev branch. Since you proposed a new release process, I'll leave the abandon/fix-again decision to you.

Comment 16

[François Marier [:francois]]

While I agree that this train has run into a lot of problems, adding more changes to it (i.e. getting all of the stuff that's on dev but not on 10.23) is only going to make things worse.

It sounds like we forgot to fix the test suite after we reverted the commit that broke forceIssuer, so that should be fairly easy to fix.

Comment 17

[Gene Wood [:gene]]

I've torn down stacks 0127 in both datacenters