1991063 - Artifact builds from jj repositories are busted - Error: There is no jj repo in "." (when using objdir that isn't inside / descendant of the srcdir)

Status: RESOLVED FIXED

(Firefox Build System :: General, task)

Description

[:Gijs (he/him)]

Unsure if bug 1989942 regressed this. But today I'm on my mac, not using workspaces, and I updated to today's main (https://github.com/mozilla-firefox/firefox/commit/43831fc5 ), and this happened:

Configure complete!
Be sure to run |mach build| to pick up any changes
 0:14.57 W Adding make options from /Users/gkruitbosch/dev/configs/frontend
    MOZ_OBJDIR=/Users/gkruitbosch/dev/builds/frontend
    OBJDIR=/Users/gkruitbosch/dev/builds/frontend
    FOUND_MOZCONFIG=/Users/gkruitbosch/dev/configs/frontend
    export FOUND_MOZCONFIG
  Parallelism determined by memory: using 10 jobs for 10 cores based on 32.0 GiB RAM and estimated job size of 1.0 GiB
 0:14.57 /usr/bin/make -f client.mk -j10 -s
 0:14.81 /Users/gkruitbosch/dev/firefox/third_party/python/urllib3/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
 0:14.81   warnings.warn(
 0:14.88 Creating the 'common' site at /Users/gkruitbosch/.mozbuild/srcdirs/firefox-cee71bc967db/_virtualenvs/common
 0:14.94 Using JujutsuRepository because a ".jj/" directory was detected!
 0:14.94 Warning: jj support is currently experimental, and may be disabled by setting the
 0:14.94 environment variable MOZ_AVOID_JJ_VCS=1. (This warning may be suppressed by
 0:14.94 setting MOZ_AVOID_JJ_VCS=0.)
 0:15.57 Error: There is no jj repo in "."
 0:15.91 Error running mach:
 0:15.91     mach --log-no-times artifact install
 0:15.91 The error occurred in code that was called by the mach command. This is either
 0:15.91 a bug in the called code itself or in the way that mach is calling it.
 0:15.91 You can invoke ``./mach busted`` to check if this issue is already on file. If it
 0:15.91 isn't, please use ``./mach busted file artifact`` to report it. If ``./mach busted`` is
 0:15.91 misbehaving, you can also inspect the dependencies of bug 1543241.
 0:15.91 If filing a bug, please include the full output of mach, including this error
 0:15.91 message.
 0:15.91 The details of the failure are as follows:
 0:15.91 Traceback (most recent call last):
 0:15.91   File "/Users/gkruitbosch/dev/firefox/python/mach/mach/main.py", line 390, in _run
 0:15.91     return Registrar._run_command_handler(
 0:15.91   File "/Users/gkruitbosch/dev/firefox/python/mach/mach/registrar.py", line 124, in _run_command_handler
 0:15.92     result = fn(instance, **kwargs)
 0:15.92   File "/Users/gkruitbosch/dev/firefox/python/mozbuild/mozbuild/artifact_commands.py", line 197, in artifact_install
 0:15.92     artifacts = _make_artifacts(
 0:15.92   File "/Users/gkruitbosch/dev/firefox/python/mozbuild/mozbuild/artifact_commands.py", line 123, in _make_artifacts
 0:15.92     artifacts = Artifacts(
 0:15.92   File "/Users/gkruitbosch/dev/firefox/python/mozbuild/mozbuild/artifacts.py", line 1220, in __init__
 0:15.92     self._git_root = subprocess.check_output(
 0:15.92   File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 424, in check_output
 0:15.92     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
 0:15.92   File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
 0:15.92     raise CalledProcessError(retcode, process.args,
 0:15.92 subprocess.CalledProcessError: Command '['/opt/homebrew/bin/jj', 'git', 'root']' returned non-zero exit status 1.
 0:15.92 Sentry event ID: c817cafbd70c42658f60d07e61b48b64
 0:16.05 make[3]: *** [recurse_artifact] Error 1
 0:16.05 make[2]: *** [artifact] Error 2
 0:16.05 make[1]: *** [default] Error 2
 0:16.05 make: *** [build] Error 2

This is very confusing - ./mach build seems to both believe that this is a jj repo, and that it isn't. I mean, it definitely is, but I don't know why it's confused.

Comment 1

[:Gijs (he/him)]

From some googling, I guess the "there is no jj repo" output is coming from jj itself, with the python stack pointing to the call at https://searchfox.org/firefox-main/rev/fd531836ca5e48e18a5afc171418bcbcb9d268e2/python/mozbuild/mozbuild/artifacts.py#1220 .

But running jj git root from the terminal in the repo dir works fine...

Comment 2

[:Gijs (he/him)]

diff --git a/python/mozbuild/mozbuild/artifacts.py b/python/mozbuild/mozbuild/artifacts.py
index cabac505cf..814d6927e5 100644
--- a/python/mozbuild/mozbuild/artifacts.py
+++ b/python/mozbuild/mozbuild/artifacts.py
@@ -1218,7 +1218,7 @@
         self._jj = jj
         if self._jj:
             self._git_root = subprocess.check_output(
-                [self._jj, "git", "root"], universal_newlines=True
+                [self._jj, "git", "root"], universal_newlines=True, cwd=topsrcdir
             ).strip()
         else:
             self._git_root = None

seems to be enough to fix it? I don't know if that's expected, or what other cwd it's using here (or how I'd find out, my python-fu is weak).

Comment 3

[Steve Fink [:sfink] [:s:]]

Attachment: Bug 1991063 - Force running `jj git root` from the srcdir r=ahochheiden

(Patch from :Gijs)

Comment 4

[:Gijs (he/him)]

We figured out on slack that the reason this broke for me but not others was that the artifact install code runs in the objdir, and works if the objdir is somewhere "underneath" the srcdir, but not if it's a sibling dir or similar (which it is for me).

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/f1c01352027d
https://hg.mozilla.org/integration/autoland/rev/5376a07f2383
Force running `jj git root` from the srcdir r=ahochheiden

Comment 6

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/5376a07f2383