1096102 - Migrate msisdn-gateway from jwcrypto to browserid-crypto, including cert format changes

Status: RESOLVED FIXED

(Cloud Services Graveyard :: MobileID, defect)

Description

[Ryan Kelly [:rfkelly]]

The upstream "jwcrypto" library has been renamed to "browserid-crypto" and trimmed down somewhat, in an attempt to increase maintainability of the code.  Specifically, the idea is to make it less like a general-purpose JWT-handling library and more like a special-purpose library for doing BrowserID crypto.

We should migrate msisdn-gateway to use the latest upstream release of browserid-crypto, so that all our projects are working from a consistent version of a maintained codebase.

Unfortunately, one of the upstream simplifications was the removal of "new-style" browserid certificate formats, as described here:

  https://github.com/mozilla/browserid-crypto/issues/97.

We thought that nobody was using this format in production, but the msisdn-gateway is, since it ships with jwcrypto==0.5.x which uses the new formats by default.

This can be confirmed by viewing the public key and noting that it contains a "kty" field:

  https://msisdn.services.mozilla.com/.well-known/browserid

By contrast, the old-style format uses an "algorithm" field for this information (among other less-obvious differences):

  https://api.accounts.firefox.com/.well-known/browserid

As far as I can tell, this is the *only* piece of production software that uses the new-style formats.  Even the MobileID client code that talks to this server doesn't use them - it uses gecko's builtin jwcrypto routines that only support the old-style formats.  (This strange mis-match only works because the client doesn't try to introspect the certificate it gets back from the server, and our hosted verifier supports both formats).

So I propose that we migrate production msisdn-gateway to use the old-style formats and upgrade it to use the latest release of browserid-crypto.  This will put all of our production browserid-using services on a single consistent data format.

To do so, we will need to:

  1. Confirm that all clients of msisdn-gateway are in fact using the old-style data formats, either because they're on jwcrypto < 0.5.0 or because they're using the native jwcrypto code in gecko.

  2. Convert the production msisdn-gateway keypair from new-style to old-style format.  We should be able to do this without deploying new code, since it's using a version of jwcrypto that supports both formats.

  3. Update msisdn-gateway to use browserid-crypto == 0.7.0 and deploy it through to production.  I will work on the PR for this.

  4. Wait for any certs issued by previous versions of msisdn-gateway to expire.

  5. At this point we are free to deploy a version of our hosted verifier using browserid-crypto ==0.7.0, which will not longer support the new-style format.

I don't think this will involve much actual code, but will need to be a carefully staged and managed process.

:natim :alexis :ferjm - thoughts?

Comment 1

[Ryan Kelly [:rfkelly]]

> 1. Confirm that all clients of msisdn-gateway are in fact using the old-style data formats, either
> because they're on jwcrypto < 0.5.0 or because they're using the native jwcrypto code in gecko.

This is almost certainly true, as I see the msisdn-gateway server contains a "validateJWCryptoKey" function which asserts that all submitted keys are in the old-style format.  Whoever's idea it was to add this, THANK YOU :-)

Comment 2

[Ryan Kelly [:rfkelly]]

> 2. Convert the production msisdn-gateway keypair from new-style to old-style format.  We should be
> able to do this without deploying new code, since it's using a version of jwcrypto that supports both
> formats.

By way of confirmation, the test keys found in "test.json" in the repo are old-style and evidently load fine.  So I think this should be a simple matter of re-formatting the "BIDPublicKey" and "BIDSecretKey" fields in our various configs.

I can provide a script that will do this automatically, if that would be helpful.

Comment 3

[Ryan Kelly [:rfkelly]]

Attachment: PR updating msisdn-gateway to use browserid-crypto

Given the above two comments, the PR against msisdn-gateway is a simple search-and-replace to change the module name.

Comment 4

[Ryan Kelly [:rfkelly]]

> So I think this should be a simple matter of re-formatting the
> "BIDPublicKey" and "BIDSecretKey" fields in our various configs.

Actually, for DSA keys the required changes are pretty simple:

  * "kty": "DSA"   =>   "algorithm": "DS"
  * base64-encoded fields   =>   hex-encoded fields

Comment 5

[Rémy Hubscher (:natim)]

https://github.com/mozilla-services/msisdn-gateway/commit/89041cd9e4d4c4dffbf88366124209a6b910e713

Comment 6

[Ryan Kelly [:rfkelly]]

Thanks Rémy.

FWIW, I think we should keep this open until we confirm that prod keys are converted over to the other format where necessary.

Comment 7

[Rémy Hubscher (:natim)]

I think they still are using the old format in stage and production. I am asking ops to verify this.

Dean or Bobm could you please make sure that the "BIDPublicKey" and "BIDSecretKey" configuration variables for MSISDN stage and MSISDN prod use {"algorithm": "DS"} and hex-encoded fields?

Comment 8

[Dean Wilson (:deanw)]

"algorithm": "DS" is present for both keys in stage and prod. The values also appear to be hex.

Comment 9

[Rémy Hubscher (:natim)]

Ok great

Comment 10

[Ryan Kelly [:rfkelly]]

Great!  It sounds like we can safely push out this change with no config change and no hassle:

> 3. Update msisdn-gateway to use browserid-crypto == 0.7.0 and deploy it through to production.

I'm not sure what the release schedule is for msisdn-gateway, but it would be great to get the change out fairly soon in order to fix Bug 1096106, and also since this:

>  4. Wait for any certs issued by previous versions of msisdn-gateway to expire.

Currently blocks new deployments of the verifier.

Comment 11

[Rémy Hubscher (:natim)]

I have tagged the 0.5.1 release and Dean deployed it to stage.
QA told me they are going to take care of it just after the Loop room release which should happens this week.

See Bug 1100296 to follow this.

Comment 12

[Ryan Kelly [:rfkelly]]

As of Bug 1108679, viewing the browserid public key at:
  
  https://msisdn.services.mozilla.com/.well-known/browserid

Correctly shows the key in old-style format, with "algorithm":"DS" rather than "kty":"DSA".  So I think we can mark this verified.