deploy train-2012.11.23 to production

RESOLVED INCOMPLETE

Status

Cloud Services
Operations: Deployment Requests
RESOLVED INCOMPLETE
6 years ago
6 years ago

People

(Reporter: jrgm, Assigned: gene)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
deploy train-2012.11.23 version 3, sha b74cf6ead to production.

Note: I'm going to recheck some stuff tonight, and we have a meeting together in the morning. I'm thinking we're good to go at this point, but will give the final signoff tomorrow.
(Reporter)

Updated

6 years ago
Depends on: 818651
(Assignee)

Updated

6 years ago
Assignee: nobody → gene
(Reporter)

Comment 1

6 years ago
I didn't set gene as the assigned, so he didn't have this on his radar. We synced up at 3pm and I wasn't feeling to good (physically) to push this through. So we will get this out on Monday. My fault.
(Reporter)

Updated

6 years ago
Depends on: 819506
(Assignee)

Updated

6 years ago
Status: NEW → ASSIGNED
(Assignee)

Comment 2

6 years ago
12:59pm dns drained traffic at SCL2
(Assignee)

Comment 3

6 years ago
1:06pm began scl2 code deploy
(Assignee)

Comment 4

6 years ago
1:10pm scl2 updated, please test and approve
(Reporter)

Comment 5

6 years ago
This change is similarly required in production puppet for config/production.json

https://bugzilla.mozilla.org/show_bug.cgi?id=816208#c2
(Assignee)

Comment 6

6 years ago
Config change has been made (sysadmins r54178) and deployed at scl2. All processes have been restarted.
(Reporter)

Comment 7

6 years ago
Okay, I have login/logout with bugz,mdn,affiliates,etherpad.mozilla.org,marketplace,mozillians,mfbeer,123done,voost,dolumar,times-xword,openphoto with scl2.
I can do signup,forgetpass,changepass,addemail,cancelaccount.
https://login.persona.org/.well-known/browserid returns a pubkey, and verifier /verifier returns a response.
And I get l10n content when I choose a non-en language.
Not seeing errors on pencil.

Okay gene, we can switch traffic to phx1.
(Assignee)

Comment 8

6 years ago
3:37 switched from scl2 to phx1
(Assignee)

Comment 9

6 years ago
3:42 switched back to phx1 per jrgm
(Assignee)

Comment 10

6 years ago
rolling back to browserid-server-0.2012.10.12-6.el6_110339.x86_64 at scl2 per jrgm
(Reporter)

Comment 11

6 years ago
Yes, thanks gene. Also let's undo the puppet change in prod for adding proxy_idps
https://bugzilla.mozilla.org/show_bug.cgi?id=816208#c2

We'll need to figure out why bugzilla was unhappy.
(Assignee)

Comment 12

6 years ago
4:41 rollback at scl2 complete
(Reporter)

Comment 13

6 years ago
Okay, starting validation. Will update soon.
(Reporter)

Comment 14

6 years ago
Okay, repeated validations noted above. Gene you can add scl2 back into public operation.
(Assignee)

Comment 15

6 years ago
5:44pm scl2 added back to dns load balance group.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → INCOMPLETE
(Reporter)

Comment 16

6 years ago
The reason for the rollback was that we had a change in the verifier service that was not compatible with how we were being called by some RPs, including bugzilla. The fix for that is in bug 820446 and https://github.com/mozilla/browserid/pull/2855 and a new build of train-2012.11.23 is in stage for testing now.
You need to log in before you can comment on or make changes to this bug.