Hard Delete Makes after 14 days

RESOLVED FIXED

Status

Webmaker
MakeAPI
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: cade, Assigned: cade)

Tracking

Details

(Whiteboard: [MakeAPI])

Attachments

(1 attachment)

(Assignee)

Description

5 years ago
Makes marked as deleted should be removed Permanently every 14 days.

Updated

4 years ago
Duplicate of this bug: 922815
(Assignee)

Comment 2

4 years ago
Created attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

I wrote a script that can be used to remove all makes that were deleted 2 weeks or more before the scripts invocation.

How could we set this up to run periodically on our infra?
Attachment #8340034 - Flags: review?(jon)

Comment 3

4 years ago
Comment on attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

This script won't exit properly
Attachment #8340034 - Flags: review?(jon) → review-
(Assignee)

Comment 4

4 years ago
Comment on attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

I forgot to remove a function from the script I copied from :D
Attachment #8340034 - Flags: review- → review?(jon)

Comment 5

4 years ago
Comment on attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

lookin' good
Attachment #8340034 - Flags: review?(jon) → review+

Comment 6

4 years ago
Commit pushed to master at https://github.com/mozilla/MakeAPI

https://github.com/mozilla/MakeAPI/commit/659324602ae7e93c934f890e2acf23f9bc2f9c72
Bug 867982 - Add script for deleting makes deleted more than 2 weeks prior to the current time at the scripts execution
(Assignee)

Comment 7

4 years ago
Merged into master. Now we can set up cron jobs to execute the script. need-info-ing jp.
Flags: needinfo?(johns)

Comment 8

4 years ago
We can setup a worker node which receives MakeAPI but does not sit in the load balancer, and then runs a cron job.

Let me know what the cronjob is going to need to run, how often, and I'll get it setup.
Flags: needinfo?(cade)

Updated

4 years ago
Flags: needinfo?(johns)
(Assignee)

Comment 9

4 years ago
(In reply to JP Schneider :jp from comment #8)
> We can setup a worker node which receives MakeAPI but does not sit in the
> load balancer, and then runs a cron job.
> 
> Let me know what the cronjob is going to need to run, how often, and I'll
> get it setup.

It should run the `scripts/cleanup.js` script.

The exit code will be 0 if successful, 1 if not.

Once a day seems fine. As for when, ideally when our traffic is lightest, but I cannot say with certainty when that is.
Flags: needinfo?(cade)

Comment 10

4 years ago
I am working on this here: https://github.com/mozilla/mofo_system_configurations/compare/webmakercontroller

I'm creating a new controller role / node, makeapi will be the first app I have put into that by creating a new nodetype variable which is set based on if something is a controller or web server in the MakeAPI manifest.

Comment 11

4 years ago
I've got it to a review-worthy state.  I have additionally added the mofo-techops/creds/app/staging(production)/webmaker-controller folder.  The lame part here will be ensuring if we update makeapi configs for usual nodes, we do the same for this folder.  I'll think on how to change that so it works better.

Tomorrow, I'll be ready to create IAM creds and rights for this new node type, and test it out while makeapi-production puppet is stopped.  Staging...of course. :)

Comment 12

4 years ago
PR with puppet diff is here: https://github.com/mozilla/mofo_system_configurations/pull/208

You can see I added a variable into Makeapi, 'nodetype', to determine which services/files each server had setup.

Comment 13

4 years ago
I've created a staging and prod node, in an autoscaling group (we want the node replaced if it dies).

In staging, I ran the job with these results:
root@i-834865fb:/var/www/makeapi# cd /var/www/makeapi && . ./.env && /usr/bin/node scripts/cleanup.js 
82 Makes deleted successfully.

I'm bringing prod up now, and it should log to this stream:
http://loggins.mofoprod.net/streams/52a0a7dce4b00e9f99fec7a8/messages?q=*&rangetype=relative&relative=300

Comment 14

4 years ago
The actual logging stream is here: http://loggins.mofoprod.net/streams/52a0c068e4b00e9f99fecc93/messages?q=*&rangetype=relative&relative=300

Closing this out!
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.