1865993 - firefox-120.0 MOZ_PGO fails with python-3.12

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement, P2)

Description

[Joonas Niilola]

Attachment: firefox-120.0:20231122-072927.log.xz

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

Try to build Firefox-120.0 with MOZ_PGO=1 and using python-3.12.

Actual results:

/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/testing/mozbase/mozinfo/mozinfo/mozinfo.py:198: SyntaxWarning: invalid escape sequence '\d'
  bits = re.search("(\d+)bit", bits).group(1)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/shellutil.py:38: SyntaxWarning: invalid escape sequence '\$'
  special="\$",
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozversioncontrol/mozversioncontrol/__init__.py:127: SyntaxWarning: invalid escape sequence '\+'
  match = re.search("version ([^\+\)]+)", info)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:4: SyntaxWarning: invalid escape sequence '\w'
  """
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:148: SyntaxWarning: invalid escape sequence '\s'
  not_ws = re.match("!\s*", self.content)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:158: SyntaxWarning: invalid escape sequence '\w'
  """
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:165: SyntaxWarning: invalid escape sequence '\s'
  m = re.match("defined\s*\(\s*(\w+)\s*\)", self.content)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:175: SyntaxWarning: invalid escape sequence '\w'
  word_len = re.match("\w*", self.content).end()
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:185: SyntaxWarning: invalid escape sequence '\s'
  ws_len = re.match("\s*", self.content).end()
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:355: SyntaxWarning: invalid escape sequence '\w'
  self.varsubst = re.compile("@(?P<VAR>\w+)@", re.U)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:387: SyntaxWarning: invalid escape sequence '\s'
  instruction_prefix = "\s*{0}"
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:388: SyntaxWarning: invalid escape sequence '\s'
  instruction_cmd = "(?P<cmd>[a-z]+)(?:\s+(?P<args>.*?))?\s*$"
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:390: SyntaxWarning: invalid escape sequence '\s'
  ambiguous_fmt = instruction_prefix + "\s+" + instruction_cmd
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:554: SyntaxWarning: invalid escape sequence '\d'
  numberValue = re.compile("\d+$")
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:678: SyntaxWarning: invalid escape sequence '\w'
  m = re.match("(?P<name>\w+)(?:\s(?P<value>.*))?", args, re.U)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:691: SyntaxWarning: invalid escape sequence '\w'
  m = re.match("(?P<name>\w+)$", args, re.U)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:731: SyntaxWarning: invalid escape sequence '\W'
  if re.search("\W", args, re.U):
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:746: SyntaxWarning: invalid escape sequence '\W'
  if re.search("\W", args, re.U):
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:795: SyntaxWarning: invalid escape sequence '\w'
  lst = re.split("__(\w+)__", args, re.U)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:840: SyntaxWarning: invalid escape sequence '\s'
  return re.sub("^\s*//.*", "", aLine)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/preprocessor.py:882: SyntaxWarning: invalid escape sequence '\.'
  re.search("\.(js|jsm|java|webidl)(?:\.in)?$", args.name)
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/util.py:1262: SyntaxWarning: invalid escape sequence '\$'
  VARIABLES_RE = re.compile("\$\((\w+)\)")
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/configure/__init__.py:935: SyntaxWarning: invalid escape sequence '\.'
  RE_MODULE = re.compile("^[a-zA-Z0-9_\.]+$")
/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/mozconfig.py:69: SyntaxWarning: invalid escape sequence '\s'
  """
Traceback (most recent call last):
  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/build/pgo/profileserver.py", line 19, in <module>
    from mozrunner import CLI, FirefoxRunner
  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/testing/mozbase/mozrunner/mozrunner/__init__.py", line 6, in <module>
    import mozrunner.base
  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/testing/mozbase/mozrunner/mozrunner/base/__init__.py", line 6, in <module>
    from .browser import BlinkRuntimeRunner, GeckoRuntimeRunner
  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/testing/mozbase/mozrunner/mozrunner/base/browser.py", line 11, in <module>
    from ..application import DefaultContext, FirefoxContext
  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/testing/mozbase/mozrunner/mozrunner/application.py", line 8, in <module>
    from distutils.spawn import find_executable
ModuleNotFoundError: No module named 'distutils'
Error running mach:

    mach build --verbose

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke ``./mach busted`` to check if this issue is already on file. If it
isn't, please use ``./mach busted file build`` to report it. If ``./mach busted`` is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['/var/tmp/portage/www-client/firefox-120.0/work/firefox_build/srcdirs/firefox-120.0-9f4e39aeab17/_virtualenvs/build/bin/python', '/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/build/pgo/profileserver.py']' returned non-zero exit status 1.

  File "/var/tmp/portage/www-client/firefox-120.0/work/firefox-120.0/python/mozbuild/mozbuild/build_commands.py", line 215, in build
    subprocess.check_call(pgo_cmd, cwd=instr.topobjdir, env=pgo_env)
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
 * ERROR: www-client/firefox-120.0::gentoo failed (compile phase):
 *   Failed to run './mach build --verbose'

Please find the compressed full build log as attachment.

Expected results:

I couldn't find a similar bug blocking bug 1857515. Without PGO build seems to pass, but there are lots of python-related warnings.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::General' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Alex Hochheiden [:ahochheiden]]

This is a symptom of the distutils refactoring still needing to be completed.

Comment 3

[Alex Hochheiden [:ahochheiden]]

It looks like this was fixed in bug 1866829 by serge. I'll close this now.