Interleaved output from multiple pymake processes make it hard to debug build errors

RESOLVED WONTFIX

Status

()

Core
Build Config
RESOLVED WONTFIX
5 years ago
7 months ago

People

(Reporter: glandium, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
With pymake, we can use -jn, which is great for build speed, but the result is that logs are interleaved, and when there are build failures, it's painful to find the related logs. It's already bad on unix with make, but on windows with pymake, it's much worse.
I was going to close this, but I found this bug while looking to see if we had a bug on summarizing build failures. Even without pymake, given how parallel the build is nowadays when you hit a compile error it's usually way back in the log and hard to find. It would be great if we could capture these and summarize them at the end of the build. I don't care if it's mach or mozbuild or what, but it sucks right now.

Comment 2

3 years ago
As I mentioned in IRC the other day, we already have a mechanism for wrapping the compiler. I think we should extend this so our compiler wrapper captures output and saves it to a file in case of failure. We can then read files from a directory after the build to get full output from the failed commands. And, if we're wrapping the compiler, we can invoke the compiler process via psutil's APIs so we can record per-process CPU, memory, and I/O statistics. I know people have been interested in which compiler invocations consume the most resources...
(Reporter)

Comment 3

3 years ago
We're only wrapping the compiler on windows. Wrapping on other platforms might add unwanted overhead, especially if it's python needing to import our world of modules. OTOH, we already have *output* wrapping, and we track warnings from there. There's no reason we can't add errors to that.
Also, we can now leverage make -O target.

Comment 4

7 months ago
Mass close of pymake-related bugs.
Status: NEW → RESOLVED
Last Resolved: 7 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.