The purge-builds mozharness can delete a users home folder

RESOLVED INVALID

Status

Release Engineering
Mozharness
RESOLVED INVALID
3 years ago
2 years ago

People

(Reporter: chmanchester, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
I was running

python scripts/scripts/firefox_ui_updates.py --cfg generic_releng_config.py --firefox-ui-branch mozilla-beta --update-verify-config mozBeta-firefox-linux.cfg --tools-tag FIREFOX_40_0b4_RELEASE_RUNTIME --total-chunks 6 --this-chunk 1

in order to reproduce the failure in bug 1176358. There's a purge-builds step that runs, and succeeds in the following:

14:09:53     INFO - #####
14:09:53     INFO - ##### Running purge-builds step.
14:09:53     INFO - #####
14:09:53     INFO - Running main action method: purge_builds
14:09:53     INFO - Running command: ['/home/chris/mozharness/external_tools/purge_builds.py', '-s', '2', '--max-age', '14', '--not', 'info', '--not', 'rel-*', '--not', 'tb-rel-*', '/home/chris']
14:09:53     INFO - Copy/paste: /home/chris/mozharness/external_tools/purge_builds.py -s 2 --max-age 14 --not info --not rel-* --not tb-rel-* /home/chris
14:09:53     INFO - Using env: {'PATH': '/home/chris/git-cinnabar:/usr/local/heroku/bin:/home/chris/B2G/out/host/linux-x86/bin:/home/chris/.cabal/bin:/home/chris/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games'}

<snip>

14:12:34     INFO -  Deleting /home/chris/Documents
14:12:34     INFO -  Deleting /home/chris/Music
14:12:34     INFO -  Deleting /home/chris/Public
14:12:34     INFO -  Deleting /home/chris/Templates
14:12:34     INFO -  Deleting /home/chris/Videos

<snip>

14:12:34     INFO -  Deleting /home/chris/.pip
14:12:34     INFO -  Deleting /home/chris/.ssh
14:12:34     INFO -  Deleting /home/chris/.cabal

etc.

Fortunately I don't keep anything important on my laptop that isn't in version control, but wow.

Comment 1

3 years ago
wow...

so purge_builds considers anything free game from the base_work_dir: http://mxr.mozilla.org/build/source/mozharness/mozharness/mozilla/purge.py#48

in your case that's '/home/chris' from the looks of your purge_builds.py call in comment 0

base_work_dir defaults to os.cwd() or .. wherever you call mozharness from. So i think you must have ran:
    python mozharness/scripts/firefox_ui_updates.py from /home/chris

This seems pretty fragile. It supports how we do things in automation right now but maybe we should change base_work_dir to work_dir and only purge things within 'build'. In your case, that would have been anything in /home/chris/build/* and not one level back. Though, I can't quite grasp the effect that would have in automation.

armen: iirc - you have been conscious of running this locally. How do we run it locally and is there a way we can remove purge_build action from the list of actions?
Flags: needinfo?(armenzg)

Comment 2

3 years ago
catlee tells me that purge_builds.py will actually crawl up one (maybe two?) directories from the target path its given as that's how folders/builddirs are structured in buildbot automation.

safest thing to do would be to remove this action when run locally and always run mh from a nested location from anything important.

Comment 3

3 years ago
The purging was added since the Windows builders are not by default cleaned up properly by anything (maybe runner is now in effect and this might not be an issue).

I backed this out for now until someone has time to look into it:
https://hg.mozilla.org/build/mozharness/rev/0dc76e63c4a6

Takers to fix the root issue are welcome. I don't have time at the moment and purging builds is not a hard requirement for me.
Flags: needinfo?(armenzg)
Summary: The purge-builds mozharness step will run when testing locally and delete large portions of a users home folder → The purge-builds mozharness can delete a users home folder

Comment 4

3 years ago
I've made this block bug 1182796 (make it production ready) instead of bug 1176358 (determine why Linux jobs are crashing).
Blocks: 1182796
No longer blocks: 1176358

Comment 5

2 years ago
We don't have the purge builds step anymore. I'm not planning on adding it unless really needed.
We clean up the machines without purge-builds before running the actual job.
I don't know why there is purge-builds to beging with.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.