1529860 - Add new breaches to Remote Settings automatically

Status: RESOLVED FIXED

(Firefox :: Firefox Monitor, enhancement)

Description

[(inactive) Nihanth Subramanya [:nhnt11]]

This will involve a few steps:

1. Set up a new Kinto account for a script to authenticate and edit records.
2. Update the script I've been locally running to update breaches to use credentials from the environment, support a dry-run mode, and include some more logging/feedback.
3. Drop the script into the blurts-server repo and notify ops - they'll need to set up automation to run the script periodically.

Comment 1

[(inactive) Nihanth Subramanya [:nhnt11]]

:leplatrem on #storage identified some questions we need to answer around this.

1. How often do you want it to run?
2. Should a human being approve the changes (recommended)?
3. How do you want to receive alerts when the script fails?
4. What is your process to verify that the script is doing the job (duplicates etc.) ?

My tentative answers are:

1. Every day
2. Yes
3. An email to the "automation owner" (maybe Bob?) as well as the fxmonitor engineering team
4. The script already does a diff against HIBP - we could extend that to do a full check of our records, compare against HIBP, and check for dupes. leplatrem suggested that the script could have two modes - sync and check (or something like that).

Luke, your input would be valuable here - could you chime in please? :)

Thanks!

Comment 2

[Luke Crouch [:groovecoder]]

Been a while, but all those answers sound good. I'm making a modified version of :nhnt11's original script in a pull request here:

https://github.com/mozilla/blurts-server/pull/855

cc'ing :oremj on this as a heads-up for when we activate this on the live Monitor service.

Comment 3

[Luke Crouch [:groovecoder]]

Just FYI here, I ran the script from the PR and it loaded the newest breach exactly as it should. Will plan to merge, deploy, and configure cron job early next week.

Comment 4

[Luke Crouch [:groovecoder]]

Merged PR and tags v5.1.0 of blurts-server repo.

To deploy it to stage, we need these env var values:

FX_REMOTE_SETTINGS_WRITER_SERVER="https://settings-writer.prod.mozaws.net/v1"
FX_REMOTE_SETTINGS_BEARER_TOKEN="<get from storage/kinto team>"

:leplatrem - can you help us with that?

Comment 5

[Mathieu Leplatre [:leplatrem]]

The stage writer is https://settings-writer.stage.mozaws.net/v1

We can't really provide a FX_REMOTE_SETTINGS_BEARER_TOKEN.

We can create a Kinto account, allow this account to edit records on your collection. And your script will use this account credentials as basic auth to authenticate.

Also, preferably, the account credentials remain secret to the «human» reviewers, so that nobody could bypass multi-signoff.

Comment 6

[Luke Crouch [:groovecoder]]

Oh, I can change the code to use basic auth then. How would you like to communicate stage auth creds?

Comment 7

[Luke Crouch [:groovecoder]]

I got stage creds and I was able to sign in at https://settings-writer.stage.mozaws.net/v1/admin/#/ but I wasn't able to create the fxmonitor-breaches collection.

Comment 8

[Mathieu Leplatre [:leplatrem]]

The collection exists already:
https://settings.stage.mozaws.net/v1/buckets/main/collections/fxmonitor-breaches

But you'll have to wait for https://github.com/mozilla-services/remote-settings-permissions/pull/37 before being able to create records with that new user

Comment 9

[Mathieu Leplatre [:leplatrem]]

Should be ok now.

Comment 10

[(inactive) Nihanth Subramanya [:nhnt11]]

Luke has taken over this effort.

Comment 11

[Jon Buckley [:jbuck]]

PR up at https://github.com/mozilla-services/cloudops-infra/pull/878 and deployed in stage

Comment 12

[Luke Crouch [:groovecoder]]

I saw the new Appartoo added by the updatebreaches.js script in stage. It triggered a review request which I approved. This looks good to deploy to prod (via https://bugzilla.mozilla.org/show_bug.cgi?id=1545507)