Closed
Bug 915535
Opened 11 years ago
Closed 10 years ago
BUILDSTATUS messages may be interleaved with other output, confusing mach
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla31
People
(Reporter: glandium, Assigned: glandium)
References
Details
Attachments
(2 files, 2 obsolete files)
18.73 KB,
patch
|
gps
:
review+
|
Details | Diff | Splinter Review |
15.24 KB,
patch
|
gps
:
review+
|
Details | Diff | Splinter Review |
Filing in build config, but this is a joint bug with mach. Currently, it's not very likely to happen, because there aren't too many directories for which we print TIERDIR_START. With my wip for bug 907365, I made it such that every directory (or almost every) prints it, because, really, in a world where everything can be built in parallel, there's no reason not to. The problem is that some of those TIERDIR_START/TIERDIR_STOP messages are interleaved with build logs from other make processes, most usually with compiler warnings. And mach doesn't catch them. So it ends up failing with a backtrace like: Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 761, in run self.__target(*self.__args, **self.__kwargs) File "/home/mh/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 698, in _processOutput self.processOutputLine(line.rstrip()) File "/home/mh/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 663, in processOutputLine handler(line) File "/home/mh/mozilla-central/python/mach/mach/mixin/process.py", line 86, in handleLine line_handler(line) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 261, in on_line warning, state_changed, relevant = self.monitor.on_line(line) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/controller/building.py", line 350, in on_line self.tiers.finish_dir(tier, subtier, d) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/controller/building.py", line 152, in finish_dir self.active_dirs[subtier].remove(d) KeyError: u'netwerk/streamconv/converters' I think the best thing to do is to change BUILDSTATUS to do echo >&$(MACH_FD) and have mach set up a pipe on that MACH_FD and pass its number. That would, however make pymake unhappy, so we may want a native command for pymake.
Assignee | ||
Comment 1•11 years ago
|
||
Using a separate fd, while the most definitive way to avoid the issue, requires fiddling deep inside mozprocess.processhandler :(
Assignee | ||
Comment 2•11 years ago
|
||
So, in fact, it looks like redirecting BUILDSTATUS messages to stderr works around the issue, without using a third file handle.
Assignee | ||
Comment 3•11 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #2) > So, in fact, it looks like redirecting BUILDSTATUS messages to stderr works > around the issue, without using a third file handle. Interestingly, the resource monitor is still confused...
Assignee | ||
Comment 4•11 years ago
|
||
So, in fact, that doesn't really pan out. I was just lucky that it worked once.
Comment 5•11 years ago
|
||
Perhaps pymake should be coordinating writes to stdout/stderr via a lock or having all line-buffered output piped to a common process.
Assignee | ||
Comment 6•11 years ago
|
||
Absolutely none of the last dozen builds i've done have working resource-usage info. All end with some variant of: Exception when writing resource usage file: u'compile_nspr' with the key varying.
Assignee | ||
Comment 7•10 years ago
|
||
It's not used except with pymake, anyways.
Attachment #8396876 -
Flags: review?(gps)
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
Assignee | ||
Comment 8•10 years ago
|
||
It's currently inaccurate, and soon, the very notion of subtier is going to blow away in the compile/binaries targets.
Attachment #8396877 -
Flags: review?(gps)
Updated•10 years ago
|
Attachment #8396876 -
Flags: review?(gps) → review+
Comment 9•10 years ago
|
||
Comment on attachment 8396877 [details] [diff] [review] Remove subtier tracking from build system Review of attachment 8396877 [details] [diff] [review]: ----------------------------------------------------------------- Please verify `mach resource-usage` still works with dirs and subtiers removed.
Attachment #8396877 -
Flags: review?(gps) → review+
Assignee | ||
Comment 10•10 years ago
|
||
With resource-usage fixed.
Attachment #8396891 -
Flags: review?(gps)
Assignee | ||
Updated•10 years ago
|
Attachment #8396876 -
Attachment is obsolete: true
Assignee | ||
Comment 11•10 years ago
|
||
With resource-usage fixed.
Attachment #8396893 -
Flags: review?(gps)
Assignee | ||
Updated•10 years ago
|
Attachment #8396877 -
Attachment is obsolete: true
Updated•10 years ago
|
Attachment #8396891 -
Flags: review?(gps) → review+
Updated•10 years ago
|
Attachment #8396893 -
Flags: review?(gps) → review+
Assignee | ||
Comment 12•10 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/870f0aa64918 https://hg.mozilla.org/integration/mozilla-inbound/rev/f204394cac0c
Assignee | ||
Comment 13•10 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/0102998866ac
Comment 14•10 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/870f0aa64918 https://hg.mozilla.org/mozilla-central/rev/f204394cac0c https://hg.mozilla.org/mozilla-central/rev/0102998866ac
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla31
Updated•6 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•