KeyError: 'HG' doing "./mach build" for artifact build on Ubuntu 20.04 using Python 3.11.0
Categories
(Firefox Build System :: Mach Core, defect, P3)
Tracking
(Not tracked)
People
(Reporter: john, Unassigned)
References
Details
Steps to reproduce:
[sorry I can't find how to determine the build system version so I just chose 1.0 because it's a required field]
$ python3 bootstrap.py
$ cat >> mozconfig
mk_add_options MOZ_OBJDIR=./objdir-frontend
$ cd mozilla-unified
$ ./mach build
Actual results:
john@sup:~/dev/ff/mozilla-unified$ ./mach build
0:00.63 Clobber not needed.
0:00.65 Adding make options from /media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/mozconfig
MOZ_OBJDIR=/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/objdir-frontend
OBJDIR=/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/objdir-frontend
FOUND_MOZCONFIG=/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/mozconfig
export FOUND_MOZCONFIG
Parallelism determined by memory: using 4 jobs for 4 cores based on 31.2 GiB RAM and estimated job size of 1.0 GiB
0:00.65 /usr/bin/make -f client.mk -j4 -s
0:01.45 Error running mach:
0:01.45 ['--log-no-times', 'artifact', 'install']
0:01.45 The error occurred in the implementation of the invoked mach command.
0:01.45 This should never occur and is likely a bug in the implementation of that
0:01.45 command.
0:01.45 You can invoke |./mach busted| to check if this issue is already on file. If it
0:01.45 isn't, please use |./mach busted file artifact| to report it. If |./mach busted| is
0:01.45 misbehaving, you can also inspect the dependencies of bug 1543241.
0:01.45 If filing a bug, please include the full output of mach, including this error
0:01.45 message.
0:01.45 The details of the failure are as follows:
0:01.45 KeyError: 'HG'
0:01.45 File "/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/python/mozbuild/mozbuild/artifact_commands.py", line 177, in artifact_install
0:01.45 artifacts = _make_artifacts(
0:01.45 ^^^^^^^^^^^^^^^^
0:01.45 File "/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/python/mozbuild/mozbuild/artifact_commands.py", line 96, in _make_artifacts
0:01.45 hg = command_context.substs["HG"]
0:01.45 ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
0:01.52 make[3]: *** [Makefile:122: recurse_artifact] Error 1
0:01.52 make[2]: *** [/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/config/recurse.mk:34: artifact] Error 2
0:01.52 make[1]: *** [/media/john/0891fedf-3029-4f23-8de2-26a33dc9b6d71/john/dev/ff/mozilla-unified/config/rules.mk:361: default] Error 2
0:01.52 make: *** [client.mk:63: build] Error 2
0:01.52 0 compiler warnings present.
|
||
Comment 1•3 years ago
|
||
The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Mach Core' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
|
||
Comment 2•3 years ago
•
|
||
I can't reproduce this. Artifact builds on Ubuntu 20.04.3 with Python3.11 (through pyenv) work fine for me (Regular builds are broken, though, fix coming in bug 1799982).
I used pyenv to install Python3.11, then installed Mercurial (as per Building Firefox On Linux docs. Ran ./mach bootstrap selected 1 (Artifact Builds) and ran ./mach build. Build succeeded in just over a minute:
1:05.75 Swap in/out (MB): 0.015625/29.8671875
To view resource usage of the build, run |mach resource-usage|.
1:05.76 Your build was successful!
To take your build for a test drive, run: |mach run|
Can you run the following 3 commands and paste the results here to confirm your environment?
which python3
python3 --version
python3 -m mercurial
The mercurial one will come back with an error, but the error lets us know if it's installed for Python3.11.
My results:
ahochheiden@ubuntu:~/mozilla-unified$ which python3
/home/ahochheiden/.pyenv/shims/python3
ahochheiden@ubuntu:~/mozilla-unified$ python3 --version
Python 3.11.0
ahochheiden@ubuntu:~/mozilla-unified$ python3 -m mercurial
/home/ahochheiden/.pyenv/versions/3.11.0/bin/python3: No module named mercurial.__main__; 'mercurial' is a package and cannot be directly executed
Thanks Alex! I too used pyenv:
$ which python3
/home/john/.pyenv/shims/python3
$ python3 --version
Python 3.11.0
$ python3 -m mercurial
/home/john/.pyenv/versions/3.11.0/bin/python3: No module named mercurial
Should I "pip install mercurial"?
|
|
||
Comment 4•3 years ago
|
||
Yes, mercurial is required.
Try python3 -m pip install --user mercurial (As per the Building Firefox On Linux docs).
Ah, I was using this page: https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html
and apparently the line that says "More information for Linux and for MacOS" and points to the page you cite is more critical than it sounds, and should come before, not after, the bootstrap step. [Installing mercurial after bootstrapping gave me the same build error--I needed to re-bootstrap.]
I'm happy to submit a docs PR [tho I don't see where the docs source is] or a separate issue describing this edit, as you suggest.
My build is now successful! Thanks again Alex!
|
|
||
Comment 6•3 years ago
|
||
(In reply to john from comment #5)
Ah, I was using this page: https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html
and apparently the line that says "More information for Linux and for MacOS" and points to the page you cite is more critical than it sounds, and should come before, not after, the bootstrap step. [Installing mercurial after bootstrapping gave me the same build error--I needed to re-bootstrap.]I'm happy to submit a docs PR [tho I don't see where the docs source is] or a separate issue describing this edit, as you suggest.
That's fair. I can see the confusion of having some of the instructions in the 'quick reference', but not all. Feel free to update the docs, but raise a separate bug to do so. My recommendation would be to add a section guiding the user to install mercurial in the 'quick reference'. Note that MacOS shouldn't use 6.2 or greater, so both Linux/Mac instructions could indicate 6.1.4 (Which is what the Mac instructions say).
You can find the relevant docs pages here:
https://searchfox.org/mozilla-central/source/docs/contributing/contribution_quickref.rst
https://searchfox.org/mozilla-central/source/docs/setup/linux_build.rst
https://searchfox.org/mozilla-central/source/docs/setup/macos_build.rst
https://searchfox.org/mozilla-central/source/docs/setup/windows_build.rst
My build is now successful! Thanks again Alex!
Glad I could help!
I'll be closing this bug as INVALID since there is no here issue that needs fixing.
Description
•