1578785 - Deploy Autograph 3.7.0 train-4

Status: RESOLVED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Julien Vehent]

Tracker for autograph train-4 to be deployed the week of september 23rd.

Comment 1

[Greg G]

Code changes:

• bump golang 1.12 to 1.13
• update signers to use the HSM RNG when available not just when their private key is in the HSM (this should be more secure since it's a better entropy source and performant since it's higher bandwidth)
• add a check to error initializing signers with malformed public keys (too short <50 bytes) usually indicating an HSM fetch failure
• log DB and HSM latencies from heartbeat check
• log output SHA256 sums for /sign/file and /sign/data

full diff: https://github.com/mozilla-services/autograph/compare/3.6.0...3.7.0
image tag: https://hub.docker.com/r/mozilla/autograph/tags/?page=1&name=3.7.0

Comment 2

[Greg G]

Config changes:

• add dep and rel signing keys for Firefox for Fire TV APKs
• increase the SAO and other non-AMO XPI signer RSA cache sizes and key generation rate

Comment 3

[Greg G]

Deployed to stage.

QA:

stage proxy monitor passes (run-id 9aa45095-0943-4f27-90db-8e64586955e9)
monitor passes (run-id a70f89df-e5ef-4d74-b23c-770597c893ec)
test addons signed and verified on AMO

:bpitts or :miles can you run "./manage.py update_signatures --force" on the normandy stage admin?
:aki can you run the relevant MAR/widevine/mac signing jobs from https://moz-releng-docs.readthedocs.io/en/latest/procedures/Testing_Autograph.html ?

Comment 4

[Aki Sasaki (not active)]

The staging mar task went green. Did we need widevine/mac tests? Those aren't set up currently.

Comment 5

[Greg G]

(In reply to Aki Sasaki [:aki] (he/him) (UTC-7) from comment #4)

The staging mar task went green.

Sweet, thanks!

Did we need widevine/mac tests? Those aren't set up currently.

Those might be prod only, so I think we're good.

Comment 6

[Brian Pitts]

Stage signed ok.

https://earthangel-b40313e5.influxcloud.net/d/FZfPnJcWz/normandy-signing?orgId=1&from=now-1h&to=now&var-env=stage

Comment 7

[Greg G]

Thanks :bpitts!

Continuing with Kinto QA the kinto refresh lambda completed successfully (run id: 18ccda06-3477-4f99-ab00-22d0f3697cd5)

The stage and stage preview settings sync, but we get a MissingSignatureError: Missing signature (main-preview/normandy-recipes) in stage preview.

Comment 8

[Greg G]

:leplatrem :mythmon is the MissingSignatureError: Missing signature (main-preview/normandy-recipes) error in the stage preview collection something we should worry about?

Comment 9

[Mathieu Leplatre [:leplatrem]]

No, not really an issue. This collection is not in use (normandy does not use the multi-signoff/preview feature).

It should be gone with https://bugzilla.mozilla.org/show_bug.cgi?id=1575182

Comment 10

[Greg G]

(In reply to Mathieu Leplatre [:leplatrem] from comment #9)

No, not really an issue. This collection is not in use (normandy does not use the multi-signoff/preview feature).

It should be gone with https://bugzilla.mozilla.org/show_bug.cgi?id=1575182

Awesome! We'll deploy prod later today then.

Comment 11

[Greg G]

Prod is deployed.

:bpitts can you run "./manage.py update_signatures --force" on the normandy prod admin?

:aki can you run a MAR signing job from https://moz-releng-docs.readthedocs.io/en/latest/procedures/Testing_Autograph.html and any other signing jobs releng wants to test (widevine, authenticode, omni.ja)? Otherwise those can wait 'til the next Fx signing run.

Comment 12

[Greg G]

Prod QA:

monitor is passing and I see the new fx fire apk keys
ditto for monitor proxy

AMO signed and verified the test addon

TODO Kinto

Comment 13

[Brian Pitts]

I resigned prod successfully.

Comment 14

[Aki Sasaki (not active)]

A mar-signing retrigger went green.
The other signing types happen on-push in CI, and appear to be good.

Comment 15

[Greg G]

Thanks!

Kinto refresh lambda passed (run-id: bc5edb3e-d676-4727-ac80-7c58f90ca366)

Getting:

MissingSignatureError: Missing signature (main-preview/normandy-recipes)
MissingSignatureError: Missing signature (main-preview/rocket-releases)

with other main-preview/ collections passing for prod preview and all collections in prod passing.

Comment 16

[Greg G]

I think we're good here, but we can reopen if those preview collections are an issue.