1219390 - Start producing allthethings.json based on generate_allthethings_json.sh

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

By doing this, the community and releng can produce exactly the same file and we can easily restore the set up.

Steps taken:
* Disable cronjobs on armenzg's and catlee's account
* Cloned braindump under /home/buildduty/braindump
 # As root
 chown buildduty /var/www/html/builds/allthethings.json*
 # As buildduty; we specify which python to use so it is 2.7.3 rather than 2.6
 ~/braindump/community/setup_buildbot_environment.sh -p /usr/local/bin/python2.7
 # First run
 ~/braindump/community/generate_allthethings_json.sh
* Verify that the file shows in here with an updated date [1]
* Added this cron to buildduty's:
 */15 * * * *  $HOME/braindump/community/generate_allthethings_json.sh
 5 0 * * *      find /var/www/html/builds/ -type f -name "allthethings._*" -mtime +90 -delete

The latter clean is for files that are used to determine changes in list of builders.

[1] http://cruncher.build.mozilla.org/builds/
[2] https://hg.mozilla.org/build/braindump/file/default/community/generate_allthethings_json.sh

Comment 1

[Armen [:armenzg]]

Please let me know if the steps above make sense to you and we can close this bug.

Comment 2

[Armen [:armenzg]]

Attachment: setup_buildbot_environment.txt

Log of the venv setup for posterity.

Comment 3

[Amy Rich [:arr] [:arich]]

If this is something that should persist, please make sure that it's added to cron by puppet and that it's done so in a way that sends any stdout and stderr to papertrail using logger. Ideally this should probably go on aws-manager, not cruncher (unless there's a pressing need to ahve it there), so it's not a spof.

Comment 4

[Armen [:armenzg]]

We don't want to put anymore effort in here than necessary.
I only looked into it since I needed the file to be up-to-date.

Nevertheless, I've removed needing to check the file for pulse_actions (used for automated backfilling and TH-sherrifs actions: add missing jobs; manual backfill; trigger all talos jobs).

Comment 5

[Chris AtLee [:catlee]]

makes sense to me armen, thanks

Comment 6

[Armen [:armenzg]]

You're welcome! Thanks Chris.

Comment 7

[Armen [:armenzg]]

Fallout from this can be found in bug 1232080.
It was failling silently.