1826281 - After updating via `mach bootstrap`, runaway clang-15 process uses all 32 GB of RAM memory

Status: RESOLVED INVALID

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Jonathan Watt [:jwatt]]

On macOS, I ran ./mach bootstrap to update the tools. The build is now no longer able to complete because a single clang-15 process is left pegged at 100% on a single core, chewing through RAM until it runs out and the machine grinds to a halt.

Not much info on this currently:

25:22.95 In file included from /Users/jwatt/mozilla/media/ffvpx/config.h:50:
25:22.95 /Users/jwatt/mozilla/media/ffvpx/config_darwin64.h:672:9: warning: 'CONFIG_NULL_BSF' macro redefined [-Wmacro-redefined]
25:22.95 #define CONFIG_NULL_BSF 1
25:22.95         ^
25:22.95 /Users/jwatt/mozilla/media/ffvpx/config_components_audio_video.h:29:9: note: previous definition is here
25:22.95 #define CONFIG_NULL_BSF 0
25:22.95         ^
25:23.00 security/manager/ssl/tests/unit/tlsserver/cmd/FaultyServer
25:24.24 1 warning generated.
25:26.07 1 warning generated.
25:28.92 1 warning generated.
25:29.10 security/nss/lib/freebl/libfreebl3.dylib
25:34.37 media/ffvpx/libavcodec/libmozavcodec.dylib
26:10.36 56 warnings generated.
26:10.57 gfx/ots/src
26:10.58 parser/htmlparser
26:10.59 extensions/spellcheck/hunspell/glue
26:10.59 dom/media/ogg
26:10.72 gfx/thebes
26:13.29 In file included from Unified_cpp_gfx_ots_src0.cpp:56:
26:13.29 /Users/jwatt/mozilla/gfx/ots/src/cpal.cc:197:9: warning: unused variable 'start' [-Wunused-variable]
26:13.29   off_t start = out->Tell();
26:13.29         ^
26:25.84 1 warning generated.
^C^CProcess Process-1:t compile misc libs tools
Traceback (most recent call last):
  File "/Users/jwatt/mozilla/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 137, in _collect
    while not _poll(pipe, poll_interval=sleep_interval):
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jwatt/mozilla/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 101, in _poll
    return pipe.poll(poll_interval)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 256, in poll
    return self._poll(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 423, in _poll
    r = wait([self], timeout)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 930, in wait
    ready = selector.select(timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jwatt/mozilla/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 183, in _collect
    pipe.send(entry)
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 205, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 410, in _send_bytes
    self._send(header + buf)
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 367, in _send
    n = write(self._handle, buf)
        ^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
34:12.76 965 compiler warnings present.
34:13.12 /usr/local/bin/terminal-notifier -title Mozilla Build System -group mozbuild -message Build failed

Comment 1

[Jonathan Watt [:jwatt]]

I tried unlinking the python provided by brew so that the system python is used instead, but the same thing happens.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Bootstrap Configuration' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Jonathan Watt [:jwatt]]

Removing /Users/jwatt/.mozbuild and rerunning ./mach bootstrap does not help.

Comment 4

[Jonathan Watt [:jwatt]]

This seems to be a compiler bug that was triggered by a change I made to the code! I'll attach details once I figure out which part of the change is responsible.