Filing in build config, but this is a joint bug with mach. Currently, it's not very likely to happen, because there aren't too many directories for which we print TIERDIR_START. With my wip for bug 907365, I made it such that every directory (or almost every) prints it, because, really, in a world where everything can be built in parallel, there's no reason not to. The problem is that some of those TIERDIR_START/TIERDIR_STOP messages are interleaved with build logs from other make processes, most usually with compiler warnings. And mach doesn't catch them. So it ends up failing with a backtrace like: Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 761, in run self.__target(*self.__args, **self.__kwargs) File "/home/mh/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 698, in _processOutput self.processOutputLine(line.rstrip()) File "/home/mh/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 663, in processOutputLine handler(line) File "/home/mh/mozilla-central/python/mach/mach/mixin/process.py", line 86, in handleLine line_handler(line) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 261, in on_line warning, state_changed, relevant = self.monitor.on_line(line) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/controller/building.py", line 350, in on_line self.tiers.finish_dir(tier, subtier, d) File "/home/mh/mozilla-central/python/mozbuild/mozbuild/controller/building.py", line 152, in finish_dir self.active_dirs[subtier].remove(d) KeyError: u'netwerk/streamconv/converters' I think the best thing to do is to change BUILDSTATUS to do echo >&$(MACH_FD) and have mach set up a pipe on that MACH_FD and pass its number. That would, however make pymake unhappy, so we may want a native command for pymake.
Using a separate fd, while the most definitive way to avoid the issue, requires fiddling deep inside mozprocess.processhandler :(
So, in fact, it looks like redirecting BUILDSTATUS messages to stderr works around the issue, without using a third file handle.
(In reply to Mike Hommey [:glandium] from comment #2) > So, in fact, it looks like redirecting BUILDSTATUS messages to stderr works > around the issue, without using a third file handle. Interestingly, the resource monitor is still confused...
So, in fact, that doesn't really pan out. I was just lucky that it worked once.
Perhaps pymake should be coordinating writes to stdout/stderr via a lock or having all line-buffered output piped to a common process.
Absolutely none of the last dozen builds i've done have working resource-usage info. All end with some variant of: Exception when writing resource usage file: u'compile_nspr' with the key varying.
It's not used except with pymake, anyways.
Attachment #8396876 - Flags: review?(gps)
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
It's currently inaccurate, and soon, the very notion of subtier is going to blow away in the compile/binaries targets.
Attachment #8396877 - Flags: review?(gps)
Comment on attachment 8396877 [details] [diff] [review] Remove subtier tracking from build system Review of attachment 8396877 [details] [diff] [review]: ----------------------------------------------------------------- Please verify `mach resource-usage` still works with dirs and subtiers removed.
Attachment #8396877 - Flags: review?(gps) → review+
With resource-usage fixed.
Attachment #8396891 - Flags: review?(gps)
Attachment #8396876 - Attachment is obsolete: true
With resource-usage fixed.
Attachment #8396893 - Flags: review?(gps)
Attachment #8396877 - Attachment is obsolete: true
https://hg.mozilla.org/mozilla-central/rev/870f0aa64918 https://hg.mozilla.org/mozilla-central/rev/f204394cac0c https://hg.mozilla.org/mozilla-central/rev/0102998866ac
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla31
You need to log in before you can comment on or make changes to this bug.