666282 - adjust clobberer cleanup crontasks

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

These now need to point to MySQL instead of sqlite

And be documented.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Uh, I can't find any related crontasks on dm-wwwbuild01.  Any idea where these are hidden?

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Bug 656307 set it cron up as root@dm-wwwbuild01.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: m666282-tools-p1-r1.patch

I think this is a better way to keep the logic where we can edit it, and also make sure the right db is being edited.

I'll document this as it lands.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Docs are here btw:
  https://wiki.mozilla.org/ReleaseEngineering/Applications/Clobberer

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 541564 [details] [diff] [review]
m666282-tools-p1-r1.patch

Switching review to someone who knows more php+db.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

OK, this is deployed now, and the cleanup crontask script is living in ~dmitchell/cleanup-clobberer.sh.  That's probably not the best place, and it needs to get plugged into the nightly crontab.  Amy, can you help with both of those? where would be a good place to put this script?

Comment 7

[Amy Rich [:arr] [:arich]]

I disabled the following in root's crontab:

MAILTO=[e-mail]
@hourly                $HOME/bin/cleanup_clobberer.sh /var/www/html/build/clobberer/db/clobberer.db

I made a backup of the old script in /root/bin/cleanup_clobberer.sh (/root/bin/cleanup_clobberer.sh.sqlite)

I copied the version from ~dmitchell/cleanup_clobberer.sh to /root/bin/cleanup_clobberer.sh.

And, lastly, I added a file called cleanup_clobberer to /etc/cron.hourly which calls /root/bin/cleanup_clobberer.sh if it exists.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

This isn't working because /clobberer/ requires LDAP auth, which the wget scripts aren't providing.  Hmm.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

So, I think that the best option is to just run cleanup.php directly using the CLI SAPI.  It will still need some kind of check to make sure it's not run from Apache, although that isn't so critical.  Back to the drawing board, then :)

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: m666282-tools-p2-r1.patch

So this allows the cleanup to be run via the PHP cli sapi, with a password required in an env var to discourage people from hitting cleanup.php with their web browsers.

The cleanup.php part of this is already deployed, and works from the command line.  So this is really just a review before deploying the crontab.

Should we have a similar crontab for clobberer-staging?

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

I'll land this as part of my transition to IT.

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Crontab deployed.  I also committed this:

diff --git a/clobberer/cleanup_clobberer.cron b/clobberer/cleanup_clobberer.cron
--- a/clobberer/cleanup_clobberer.cron
+++ b/clobberer/cleanup_clobberer.cron
@@ -1,2 +1,2 @@
 MAILTO=[e-mail]
-@hourly CLEANUP_PASSWORD=sekrits /usr/bin/php /var/www/html/build/clobberer/cleanup.php
+@hourly root CLEANUP_PASSWORD=sekrits /usr/bin/php /var/www/html/build/clobberer/cleanup.php

(oops!)

Otherwise, this is finished.  I'll watch over the next hour to make sure it runs.  Catlee, let me know if you get email (you shouldn't).

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

Lookin' good.