1911968 - Running `./mach puppeteer-test` deletes the entire source repository

Status: RESOLVED FIXED

(Remote Protocol :: Agent, defect, P2)

Description

[Dave Townsend [:mossop]]

I just ran ./mach puppeteer-test to attempt to reproduce a test failure I was backed out for. It sat there for quite a while with no output running ../../tools/clean.mjs. Eventually I gave up and killed the process and then tried running ./mach help puppeteer-test at which point I discovered the mach file no longer existed. In fact almost my entire source repository, including the .hg directory had been deleted.

I'm not entirely sure how many days of work I've lost here. But a mach command shouldn't be wiping large amounts of data like this without some kind of confirmation step.

Comment 1

[Dave Townsend [:mossop]]

Looks like what is happening is the clean command is running git clean. I have a git repository above my source repository, it holds a bunch of scripts and my build configurations. So this command deleted everything not checking into or ignored by that repository. Which was more than just my source repository.

Comment 2

[Julian Descottes [:jdescottes]]

Attachment: repro_git_clean_issue.zip

Here's a reduced test case that reproduces the issue.

Edit: I think we can even reduce it further, no need for workspaces or for nesting the clean script in a subfolder.

Comment 3

[Julian Descottes [:jdescottes]]

I was under the impression that we had the full git repo vendored, but it's not the case. So of course here git clean will run on whatever git repo happens to be in scope. Still odd that this didn't cause issues earlier?

Comment 4

[James Graham [:jgraham]]

Ouch, this is not good.

I've filed https://github.com/puppeteer/puppeteer/issues/12917 to make the Puppeteer team aware of this bug. We should also do something on our side to avoid the issue e.g. maybe we don't need to run clean at all, or maybe we can patch in a list of paths that actually need to be deleted.

Comment 5

[Dave Townsend [:mossop]]

(In reply to Julian Descottes [:jdescottes] from comment #3)

I was under the impression that we had the full git repo vendored, but it's not the case. So of course here git clean will run on whatever git repo happens to be in scope. Still odd that this didn't cause issues earlier?

After switching to git for my mozilla-central clone there is no longer an issue. I'm guessing the number of people that use a mercurial clone of mozilla inside a git repository is fairly small, perhaps even only me, and even I'm not now.

Comment 6

[James Graham [:jgraham]]

I think the question is why it hasn't affected people who are using git. by deleting things in the m-c .gitignore files. Most of those are .pyc and similar, which might go unnoticed, but it might include mozconfig files which people likely would notice going missing.

Comment 7

[James Graham [:jgraham]]

mossop: which git version are you running? Puppeteer have landed a patch to make this safer, but it should already have been running git clean only in the puppeteer directory (but maybe the command syntax changed in recent versions or something)

Comment 8

[Dave Townsend [:mossop]]

(In reply to James Graham [:jgraham] from comment #7)

mossop: which git version are you running? Puppeteer have landed a patch to make this safer, but it should already have been running git clean only in the puppeteer directory (but maybe the command syntax changed in recent versions or something)

v2.45.2 from homebrew.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The fix will be in Puppeteer v23.1.0 and it will be vendored in via bug 1912239.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Fixed with the landing of bug 1912239.