Closed Bug 867982 Opened 8 years ago Closed 8 years ago

Hard Delete Makes after 14 days

Categories

(Webmaker Graveyard :: MakeAPI, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: cade, Assigned: cade)

References

Details

(Whiteboard: [MakeAPI])

Attachments

(1 file)

Makes marked as deleted should be removed Permanently every 14 days.
Duplicate of this bug: 922815
I wrote a script that can be used to remove all makes that were deleted 2 weeks or more before the scripts invocation.

How could we set this up to run periodically on our infra?
Attachment #8340034 - Flags: review?(jon)
Comment on attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

This script won't exit properly
Attachment #8340034 - Flags: review?(jon) → review-
Comment on attachment 8340034 [details] [review]
https://github.com/mozilla/MakeAPI/pull/177

I forgot to remove a function from the script I copied from :D
Attachment #8340034 - Flags: review- → review?(jon)
Commit pushed to master at https://github.com/mozilla/MakeAPI

https://github.com/mozilla/MakeAPI/commit/659324602ae7e93c934f890e2acf23f9bc2f9c72
Bug 867982 - Add script for deleting makes deleted more than 2 weeks prior to the current time at the scripts execution
Merged into master. Now we can set up cron jobs to execute the script. need-info-ing jp.
Flags: needinfo?(johns)
We can setup a worker node which receives MakeAPI but does not sit in the load balancer, and then runs a cron job.

Let me know what the cronjob is going to need to run, how often, and I'll get it setup.
Flags: needinfo?(cade)
Flags: needinfo?(johns)
(In reply to JP Schneider :jp from comment #8)
> We can setup a worker node which receives MakeAPI but does not sit in the
> load balancer, and then runs a cron job.
> 
> Let me know what the cronjob is going to need to run, how often, and I'll
> get it setup.

It should run the `scripts/cleanup.js` script.

The exit code will be 0 if successful, 1 if not.

Once a day seems fine. As for when, ideally when our traffic is lightest, but I cannot say with certainty when that is.
Flags: needinfo?(cade)
I am working on this here: https://github.com/mozilla/mofo_system_configurations/compare/webmakercontroller

I'm creating a new controller role / node, makeapi will be the first app I have put into that by creating a new nodetype variable which is set based on if something is a controller or web server in the MakeAPI manifest.
I've got it to a review-worthy state.  I have additionally added the mofo-techops/creds/app/staging(production)/webmaker-controller folder.  The lame part here will be ensuring if we update makeapi configs for usual nodes, we do the same for this folder.  I'll think on how to change that so it works better.

Tomorrow, I'll be ready to create IAM creds and rights for this new node type, and test it out while makeapi-production puppet is stopped.  Staging...of course. :)
PR with puppet diff is here: https://github.com/mozilla/mofo_system_configurations/pull/208

You can see I added a variable into Makeapi, 'nodetype', to determine which services/files each server had setup.
I've created a staging and prod node, in an autoscaling group (we want the node replaced if it dies).

In staging, I ran the job with these results:
root@i-834865fb:/var/www/makeapi# cd /var/www/makeapi && . ./.env && /usr/bin/node scripts/cleanup.js 
82 Makes deleted successfully.

I'm bringing prod up now, and it should log to this stream:
http://loggins.mofoprod.net/streams/52a0a7dce4b00e9f99fec7a8/messages?q=*&rangetype=relative&relative=300
The actual logging stream is here: http://loggins.mofoprod.net/streams/52a0c068e4b00e9f99fecc93/messages?q=*&rangetype=relative&relative=300

Closing this out!
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.