1732177 - non-existent virtualenv common used with "virtx ./mach build" on Linux

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Steven Newbury]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0

Steps to reproduce:

Using a live firefox ebuild:

After successful configuration
The compile stage called "virtx ./mach build --verbose"

Actual results:

Error running mach:

['build', '--verbose']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file build| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/portage/www-client/firefox-9999/work/firefox_build/instrumented/_virtualenvs/common/bin/python'

Expected results:

Successful build

Comment 1

[Steven Newbury]

Attachment: config.log

Comment 2

[Steven Newbury]

Attachment: instrumented/config.log

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::General' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

Thanks for the separate report 👍
The configure logs look good, and create the build virtualenv as expected.

Two questions for you:

• Are there additional log lines above the "Error running mach" line, or below the "FileNotFoundError: [Errno 2]" line? If so, please share them here.
• Can you reproduce this issue outside of Gentoo's "ebuild" environment? You may need to follow the quickstart instructions to set up the local build.

Comment 5

[Steven Newbury]

There are many lines above the error. It does manage to complete the instrumented build but fails after creating the langpack xpi file.

I did try to attach the full build.log, but it was too big. Even compressed it didn't work.

I'll attach the last 100 lines.

Comment 6

[Steven Newbury]

Attachment: last 100 lines of build.log

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

Looks like it's caused by PGO builds, since the profileserver.py script is invoked without specifying the build virtualenv.
Thanks for the build log, the part that made this possible to diagnose is:

  File "/var/tmp/portage/www-client/firefox-9999/work/firefox-9999/python/mozbuild/mozbuild/build_commands.py", line 219, in build
    subprocess.check_call(pgo_cmd, cwd=instr.topobjdir, env=pgo_env)
  File "/usr/lib/python3.10/subprocess.py", line 364, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 345, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.10/subprocess.py", line 965, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1841, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1732177: Fix virtualenv misuse during PGO builds

Build-related work should happen in the "build" virtualenv, but the
BuildDriver was associated with the default "common" venv instead.
Manually associate BuildDriver with the "build" venv.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7fdeacd6e97f
Fix virtualenv misuse during PGO builds r=ahal

Comment 10

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/7fdeacd6e97f