Closed Bug 1989942 Opened 28 days ago Closed 26 days ago

Artifact builds don't work in jj workspaces

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Type:
defect

Tracking

(firefox145 fixed)

Status:
RESOLVED FIXED
Milestone:
145 Branch
Tracking Flags:
Tracking Status
firefox145 --- fixed

People

(Reporter: Gijs, Assigned: sfink)

References

(Blocks 2 open bugs)

Details

Attachments

(2 files)

Bug 1989942 - Allow artifact builds from jj workspaces r=ahochheiden
48 bytes, text/x-phabricator-request
Details | Review
Bug 1989942 - Fix MOZ_ARTIFACT_REVISION handling r=ahochheiden
48 bytes, text/x-phabricator-request
Details | Review

Configure is fine, but then:

 0:26.15 W Adding make options from D:\config\frontend
    MOZ_OBJDIR=D:/builds/frontend
    OBJDIR=D:/builds/frontend
    FOUND_MOZCONFIG=D:/config/frontend
    export FOUND_MOZCONFIG
  Parallelism determined by memory: using 64 jobs for 64 cores based on 63.9 GiB RAM and estimated job size of 1.0 GiB
 0:26.15 C:/Users/Gijs/.mozbuild/mozmake/mozmake.exe -f client.mk -j64 -s
 0:27.00 Creating the 'common' site at C:\Users\Gijs\.mozbuild\srcdirs\fx-frontend-0965d90a5039\_virtualenvs\common
 0:27.14 Using JujutsuRepository because a ".jj/" directory was detected!
 0:27.15 Warning: jj support is currently experimental, and may be disabled by setting the
 0:27.15 environment variable MOZ_AVOID_JJ_VCS=1. (This warning may be suppressed by
 0:27.15 setting MOZ_AVOID_JJ_VCS=0.)
 0:28.67 fatal: not a git repository (or any of the parent directories): .git
 0:29.29 Error running mach:
 0:29.29     mach --log-no-times artifact install
 0:29.29 The error occurred in code that was called by the mach command. This is either
 0:29.29 a bug in the called code itself or in the way that mach is calling it.
 0:29.29 You can invoke ``./mach busted`` to check if this issue is already on file. If it
 0:29.29 isn't, please use ``./mach busted file artifact`` to report it. If ``./mach busted`` is        
 0:29.29 misbehaving, you can also inspect the dependencies of bug 1543241.
 0:29.30 If filing a bug, please include the full output of mach, including this error
 0:29.30 message.
 0:29.30 The details of the failure are as follows:
 0:29.30 Traceback (most recent call last):
 0:29.30   File "D:\fx-frontend\python\mach\mach\main.py", line 390, in _run
 0:29.30     return Registrar._run_command_handler(
 0:29.30            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.30   File "D:\fx-frontend\python\mach\mach\registrar.py", line 124, in _run_command_handler       
 0:29.30     result = fn(instance, **kwargs)
 0:29.30              ^^^^^^^^^^^^^^^^^^^^^^
 0:29.30   File "D:\fx-frontend\python\mozbuild\mozbuild\artifact_commands.py", line 203, in artifact_install
 0:29.31     return artifacts.install_from(source, distdir or command_context.distdir)
 0:29.31            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.31   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1819, in install_from      
 0:29.31     return self.install_from_recent(distdir)
 0:29.31            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.31   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1714, in install_from_recent
 0:29.31     return self._install_from_hg_pushheads(hg_pushheads, distdir)
 0:29.31            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.31   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1686, in _install_from_hg_pushheads
 0:29.31     for trees, hg_hash in hg_pushheads:
 0:29.31                           ^^^^^^^^^^^^
 0:29.31   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1522, in _find_pushheads   
 0:29.32     last_revs = self._get_recent_public_revisions()
 0:29.32                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.32   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1461, in _get_recent_public_revisions
 0:29.32     return self._get_revisions_from_git()
 0:29.32            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.32   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1410, in _get_revisions_from_git
 0:29.32     rev_list = subprocess.check_output(
 0:29.32                ^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.32   File "D:\mozilla-build\python3\Lib\subprocess.py", line 466, in check_output
 0:29.32     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
 0:29.32            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:29.32   File "D:\mozilla-build\python3\Lib\subprocess.py", line 571, in run
 0:29.33     raise CalledProcessError(retcode, process.args,
 0:29.33 subprocess.CalledProcessError: Command '['D:/mozilla-build/msys2/mingw64/bin/git.exe', 'rev-list', '--topo-order', '--max-count=500', 'HEAD']' returned non-zero exit status 128.
 0:29.33 Sentry event ID: 294a1cd50bcc4897a904b3ec8b8fc14a
 0:29.40 Sentry is attempting to send 2 pending events
 0:29.40 Waiting up to 2 seconds
 0:29.40 Press Ctrl-Break to quit
 0:29.65 mozmake[3]: *** [Makefile:128: recurse_artifact] Error 1
 0:29.66 mozmake[2]: *** [D:/fx-frontend/config/recurse.mk:34: artifact] Error 2
 0:29.66 mozmake[1]: *** [D:/fx-frontend/config/rules.mk:359: default] Error 2
 0:29.66 mozmake: *** [client.mk:60: build] Error 2
Single process terminated successfullyport misc libs tools
 0:29.76 W 0 compiler warnings present.

JJ workspace directories don't have a git directory. Pointing at the original repo's git dir also would not work as it would produce a git rev that doesn't match the current workspace commit...

Blocks: mach-busted

Trying to use MOZ_ARTIFACT_REVISION with either a git or hg rev doesn't appear to work either:

$ MOZ_ARTIFACT_REVISION=767c44c1cde821258288378998f4bb481bec8908 ./mach build
Using JujutsuRepository because a ".jj/" directory was detected!

Warning: jj support is currently experimental, and may be disabled by setting the
environment variable MOZ_AVOID_JJ_VCS=1. (This warning may be suppressed by
setting MOZ_AVOID_JJ_VCS=0.)
 0:01.42 W Clobber not needed.
 0:01.57 W Adding make options from D:\config\frontend
    MOZ_OBJDIR=D:/builds/frontend
    OBJDIR=D:/builds/frontend
    FOUND_MOZCONFIG=D:/config/frontend
    export FOUND_MOZCONFIG
  Parallelism determined by memory: using 64 jobs for 64 cores based on 63.9 GiB RAM and estimated job size of 1.0 GiB
 0:01.58 C:/Users/Gijs/.mozbuild/mozmake/mozmake.exe -f client.mk -j64 -s
 0:02.47 Using JujutsuRepository because a ".jj/" directory was detected!
 0:02.47 Warning: jj support is currently experimental, and may be disabled by setting the
 0:02.47 environment variable MOZ_AVOID_JJ_VCS=1. (This warning may be suppressed by
 0:02.47 setting MOZ_AVOID_JJ_VCS=0.)
 0:03.82 Will only accept artifacts from a pushhead at 767c44c1cde821258288378998f4bb481bec8908 (matched revset "767c44c1cde821258288378998f4bb481bec8908").
 0:04.51 Error running mach:
 0:04.51     mach --log-no-times artifact install
 0:04.51 The error occurred in code that was called by the mach command. This is either
 0:04.51 a bug in the called code itself or in the way that mach is calling it.
 0:04.51 You can invoke ``./mach busted`` to check if this issue is already on file. If it
 0:04.51 isn't, please use ``./mach busted file artifact`` to report it. If ``./mach busted`` is        
 0:04.51 misbehaving, you can also inspect the dependencies of bug 1543241.
 0:04.53 If filing a bug, please include the full output of mach, including this error
 0:04.53 message.
 0:04.53 The details of the failure are as follows:
 0:04.53 Traceback (most recent call last):
 0:04.53   File "D:\fx-frontend\python\mach\mach\main.py", line 390, in _run
 0:04.53     return Registrar._run_command_handler(
 0:04.53            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:04.53   File "D:\fx-frontend\python\mach\mach\registrar.py", line 124, in _run_command_handler       
 0:04.53     result = fn(instance, **kwargs)
 0:04.53              ^^^^^^^^^^^^^^^^^^^^^^
 0:04.53   File "D:\fx-frontend\python\mozbuild\mozbuild\artifact_commands.py", line 203, in artifact_install
 0:04.54     return artifacts.install_from(source, distdir or command_context.distdir)
 0:04.54            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:04.54   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1810, in install_from      
 0:04.54     return self.install_from_revset(source, distdir)
 0:04.54            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:04.54   File "d:\fx-frontend\python\mozbuild\mozbuild\artifacts.py", line 1770, in install_from_revset
 0:04.54     self._artifact_job.candidate_trees + [self._artifact_job.try_tree],
 0:04.54                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 0:04.54 AttributeError: 'WinArtifactJob' object has no attribute 'try_tree'
 0:04.54 Sentry event ID: 2336d1b1793e48509eef2132e3248e6a
 0:04.62 Sentry is attempting to send 2 pending events
 0:04.62 Waiting up to 2 seconds
 0:04.62 Press Ctrl-Break to quit
 0:04.88 mozmake[3]: *** [Makefile:128: recurse_artifact] Error 1
 0:04.88 mozmake[2]: *** [D:/fx-frontend/config/recurse.mk:34: artifact] Error 2
 0:04.89 mozmake[1]: *** [D:/fx-frontend/config/rules.mk:359: default] Error 2
 0:04.89 mozmake: *** [client.mk:60: build] Error 2
Single process terminated successfullylibs tools
 0:05.00 W 0 compiler warnings present.

I don't know why accessing try_tree would ever have worked - it looks like it is an attribute on *JobConfiguration classes, but not on job instances...

Blocks: artifact
Assignee: nobody → sphink
Status: NEW → ASSIGNED

Oh, I should read the bug more carefully. My patch fixes the basic problem, but MOZ_ARTIFACT_REVISION looks to be a separate problem, and is what Gijs already figured out above.

https://hg.mozilla.org/mozilla-central/rev/a7295f63f2b5
https://hg.mozilla.org/mozilla-central/rev/9d57ef350eb8

Status: ASSIGNED → RESOLVED
Closed: 26 days ago
status-firefox145: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 145 Branch
See Also: → 1991063
Blocks: 1993219
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: