Closed Bug 1566184 Opened 7 years ago Closed 7 years ago

Deploy Autograph 3.4.0 train-1

Categories

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Product:
Cloud Services
Component:
Operations: Deployment Requests - DEPRECATED
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: u581815, Assigned: u581815)

References

(Blocks 1 open bug)

Details

Please deploy Autograph 3.4.0 to staging and perform QA testing.
https://mana.mozilla.org/wiki/display/SVCOPS/Autograph#Autograph-QA

Week starting 2019-07-22

Build CI job: https://circleci.com/gh/mozilla-services/autograph/2990
Docker image listed at: https://hub.docker.com/r/mozilla/autograph/tags/

Code changes from 3.3.2: https://github.com/mozilla-services/autograph/compare/3.3.2...3.4.0

  • add python sign-tool back for SAO signing (no server component)
  • CI and lint fixes; some refactoring / renaming
  • DB errors on disconnect rather than crashing the app
  • heartbeat handler checks HSM connectivity and DB connectivity (but DB disconnect doesn't count as unhealthy since DB config is just read at startup)

Config changes:

  • 88510c7d80fd545c96aab4ef52b61e8c9d38943a add db monitor poll interval of 10 seconds (same as old watchdog) to stage and prod app configs
  • bc93bb2d7ef629fd4fb95dd2d727ecd1c1a942fa copy signer, create creds and rename keys for bug 1564119
  • a21bba87ef0038c675f17c53c0343ac10c3d56e1 add releng langpack creds for bug 1566197
  • 65189944ba20b270b17257aefa489d86328f832e csigpki changes for bug 1533177
Blocks: 1533177
Depends on: 1566197
Summary: Deploy Autograph {{TAG}} train-1 → Deploy Autograph 3.4.0 train-1

Stage is deployed. For QA:

a. the monitor is passing
b. the test addon looks good
c. kinto refresh lambda ran successfully (run: ea7279b0-4df2-425a-93d7-4185e1410441) and stage and stage preview sync in about:remotesettings
d. :bpitts can you run "./manage.py update_signatures --force" on the SHIELD stage admin?
e. :aki can you kick off the stage mar signing job?

Flags: needinfo?(bpitts)
Flags: needinfo?(aki)
Status: NEW → ASSIGNED

The stage mar signing job went green.

Flags: needinfo?(aki)

I've run "./manage.py update_signatures --force" on the SHIELD stage admin node.

Flags: needinfo?(bpitts)

Thanks :aki and :miles! I set is_current = false in the DB to force a csig renewal, so we're ready to deploy prod.

Prod is deployed. Starting QA.

Prod QA:

a. We're seeing pkcs7: failed to verify certificate chain: x509: certificate signed by unknown authority errors for the SAO and webext dep signers, which makes sense given that they use the stage root. This should be fixed by adding the stage root to the prod monitor roots. Everything else is passing and looks good.
b. AMO signed the test webextension without issues
c. kinto refresh lambda ran successfully (run: rid 59e0c0e4-e6ed-4c8e-95aa-fd82a1ad3251) and in about:remotesettings for prod preview we get MissingSignatureError: Missing signature (main-preview/normandy-recipes) from polling but otherwise it syncs without issues and prod syncs all collections without issues
d. :bpitts or :miles can you run "./manage.py update_signatures --force" on the SHIELD prod admin?
e. :aki can you let me know if the next prod mar signing job passes and check if the whitelisted mac minis can access autograph?

Flags: needinfo?(miles)
Flags: needinfo?(bpitts)
Flags: needinfo?(aki)

I retriggered a nightly mac build-signing and mar-signing task. They both went green.

Flags: needinfo?(aki)

Thanks :aki!

:ulfr wants to do a full normandy resign to make sure the csig changes are working. :miles can you do that (and the SHIELD resigning) since bpitts is on PTO?

Flags: needinfo?(bpitts)

I've successfully run ./manage.py update_signatures --force on the SHIELD prod admin.

Flags: needinfo?(miles)

Great thanks :miles!

We're all set here.

Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.