1497693 - Firefox Monitor: add job to delete unverified subscribers to stage, then prod

Status: RESOLVED FIXED

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Luke Crouch [:groovecoder]]

To delete expired unverified subscribers, we have a script:

node scripts/delete-unverified-subscribers.js

On Heroku, we run this every 10 minutes via Heroku Scheduler.

We should add a corresponding cron(?) job to the stage environment, and when we verify it's working correctly, add the same job to production.

Note: Make sure DELETE_UNVERIFIED_SUBSCRIBERS_TIMER env var is set to 86400 (24 hours).

Comment 1

[Luke Crouch [:groovecoder]]

ping: any ETA for this?

Comment 2

[Bob Micheletto [:bobm]]

I would like to avoid creating an administration node for this single job.  Does it have a locking feature where it'll refuse to run if another instance is already running?  Is it very resource intensive?

Comment 3

[Luke Crouch [:groovecoder]]

No locking feature in the script. But the DB will lock the table, so each script run would wait until the previous one finishes for that. The script should also be idempotent, so running it many times shouldn't hurt anything.

I'm afraid it may be resource-intensive, unless/until we add another DB index on the created_at field, since the delete statement has a WHERE created_at < expiredTimeStamp. Should add that index before we turn this on.

Comment 4

[Bob Micheletto [:bobm]]

(In reply to Luke Crouch [:groovecoder] from comment #3)

> I'm afraid it may be resource-intensive, unless/until we add another DB
> index on the created_at field, since the delete statement has a WHERE
> created_at < expiredTimeStamp. Should add that index before we turn this on.

Do you want to handle that through a knex db migration, or add it manually?

Comment 5

[Luke Crouch [:groovecoder]]

knex db migration for sure.

I'm planning to file a deploy bug on Monday for next week. It will include lots of changes already:

* l10n (using an env var to keep it en-US only until we're ready to launch the other locales)
* sentry functionality
* potentially a bug fix on the HIBP callback endpoint

So it will already need some hands-on deployment help. An extra db migration should be relatively minor part of it.

Comment 6

[Luke Crouch [:groovecoder]]

ni? :jbuck here too - could we tackle this cron job while we're setting up the one for remote settings?

Comment 7

[Jon Buckley [:jbuck]]

This has been deployed in stage, PR is up at https://github.com/mozilla-services/cloudops-infra/pull/855