Closed Bug 1183858 Opened 4 years ago Closed 4 years ago

The purge-builds mozharness can delete a users home folder


(Release Engineering :: Applications: MozharnessCore, defect)

Not set


(Not tracked)



(Reporter: cmanchester, Unassigned)



I was running

python scripts/scripts/ --cfg --firefox-ui-branch mozilla-beta --update-verify-config mozBeta-firefox-linux.cfg --tools-tag FIREFOX_40_0b4_RELEASE_RUNTIME --total-chunks 6 --this-chunk 1

in order to reproduce the failure in bug 1176358. There's a purge-builds step that runs, and succeeds in the following:

14:09:53     INFO - #####
14:09:53     INFO - ##### Running purge-builds step.
14:09:53     INFO - #####
14:09:53     INFO - Running main action method: purge_builds
14:09:53     INFO - Running command: ['/home/chris/mozharness/external_tools/', '-s', '2', '--max-age', '14', '--not', 'info', '--not', 'rel-*', '--not', 'tb-rel-*', '/home/chris']
14:09:53     INFO - Copy/paste: /home/chris/mozharness/external_tools/ -s 2 --max-age 14 --not info --not rel-* --not tb-rel-* /home/chris
14:09:53     INFO - Using env: {'PATH': '/home/chris/git-cinnabar:/usr/local/heroku/bin:/home/chris/B2G/out/host/linux-x86/bin:/home/chris/.cabal/bin:/home/chris/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games'}


14:12:34     INFO -  Deleting /home/chris/Documents
14:12:34     INFO -  Deleting /home/chris/Music
14:12:34     INFO -  Deleting /home/chris/Public
14:12:34     INFO -  Deleting /home/chris/Templates
14:12:34     INFO -  Deleting /home/chris/Videos


14:12:34     INFO -  Deleting /home/chris/.pip
14:12:34     INFO -  Deleting /home/chris/.ssh
14:12:34     INFO -  Deleting /home/chris/.cabal


Fortunately I don't keep anything important on my laptop that isn't in version control, but wow.

so purge_builds considers anything free game from the base_work_dir:

in your case that's '/home/chris' from the looks of your call in comment 0

base_work_dir defaults to os.cwd() or .. wherever you call mozharness from. So i think you must have ran:
    python mozharness/scripts/ from /home/chris

This seems pretty fragile. It supports how we do things in automation right now but maybe we should change base_work_dir to work_dir and only purge things within 'build'. In your case, that would have been anything in /home/chris/build/* and not one level back. Though, I can't quite grasp the effect that would have in automation.

armen: iirc - you have been conscious of running this locally. How do we run it locally and is there a way we can remove purge_build action from the list of actions?
Flags: needinfo?(armenzg)
catlee tells me that will actually crawl up one (maybe two?) directories from the target path its given as that's how folders/builddirs are structured in buildbot automation.

safest thing to do would be to remove this action when run locally and always run mh from a nested location from anything important.
The purging was added since the Windows builders are not by default cleaned up properly by anything (maybe runner is now in effect and this might not be an issue).

I backed this out for now until someone has time to look into it:

Takers to fix the root issue are welcome. I don't have time at the moment and purging builds is not a hard requirement for me.
Flags: needinfo?(armenzg)
Summary: The purge-builds mozharness step will run when testing locally and delete large portions of a users home folder → The purge-builds mozharness can delete a users home folder
I've made this block bug 1182796 (make it production ready) instead of bug 1176358 (determine why Linux jobs are crashing).
Blocks: 1182796
No longer blocks: 1176358
We don't have the purge builds step anymore. I'm not planning on adding it unless really needed.
We clean up the machines without purge-builds before running the actual job.
I don't know why there is purge-builds to beging with.
Closed: 4 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.