907849 - Tier monitor gets confused when performing partial builds (KeyError: u'nspr')

Status: RESOLVED DUPLICATE of bug 911957

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

$ mach build export
 0:00.22 /usr/bin/make -j8 -s export
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/gps/src/firefox/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 698, in _processOutput
    self.processOutputLine(line.rstrip())
  File "/Users/gps/src/firefox/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 663, in processOutputLine
    handler(line)
  File "/Users/gps/src/firefox/python/mach/mach/mixin/process.py", line 86, in handleLine
    line_handler(line)
  File "/Users/gps/src/firefox/python/mozbuild/mozbuild/mach_commands.py", line 259, in on_line
    warning, state_changed, relevant = self.monitor.on_line(line)
  File "/Users/gps/src/firefox/python/mozbuild/mozbuild/controller/building.py", line 333, in on_line
    self.tiers.begin_tier(tier, subtiers)
  File "/Users/gps/src/firefox/python/mozbuild/mozbuild/controller/building.py", line 74, in begin_tier
    t = self.tiers[tier]
KeyError: u'nspr'

Comment 1

[Nick Alexander :nalexander [he/him]]

This was a build immediately after a top-level ./mach build (so not a partial build), but it seems similar:

archo:mcgit ncalexan$ ./mach build -v
 0:01.66 /usr/bin/make -f client.mk
 0:02.81 /usr/bin/make -f /Users/ncalexan/Mozilla/mcgit/client.mk realbuild
 0:02.92 Adding client.mk options from /Users/ncalexan/Mozilla/mcgit/.mozconfig:
 0:02.92     MOZ_OBJDIR=./objdir-droid
 0:02.92     AUTOCLOBBER=1
 0:03.68 TEST-PASS | check-sync-dirs.py | /Users/ncalexan/Mozilla/mcgit/js/src/config <= /Users/ncalexan/Mozilla/mcgit/config
 0:03.76 /usr/bin/make -j8 -C ./objdir-droid
 0:09.20 Build configuration changed. Regenerating backend.
 0:09.81 Reticulating splines...
 0:32.26 Finished reading 1058 moz.build files into 3200 descriptors in 9.37s
 0:32.26 Backend executed in 12.28s
 0:32.26 2115 total backend files. 0 created; 0 updated; 2123 unchanged
 0:32.26 Total wall time: 22.45s; CPU time: 3.41s; Efficiency: 15%
 0:32.43 MOZBUILD_BACKEND_CHECKED= /usr/bin/make -C js/src backend.RecursiveMakeBackend.built
 0:32.50 if test -d dist/bin ; then touch dist/bin/.purgecaches ; fi
 0:33.28 make[3]: `backend.RecursiveMakeBackend.built' is up to date.
 0:33.28 /Users/ncalexan/Mozilla/mcgit/objdir-droid/_virtualenv/bin/python -m mozbuild.action.purge_manifests -d _build_manifests/purge .
 0:33.77 Purging unaccounted files from object directory...
 0:33.77 Deleted 23 files and 1 directories from dist/bin.
 0:33.77 Deleted 47 files and 75 directories from dist/include.
 0:33.77 Deleted 0 files and 0 directories from dist/private.
 0:33.77 Deleted 0 files and 0 directories from dist/public.
 0:33.77 Deleted 2 files and 2 directories from dist/sdk.
 0:33.77 Deleted 25 files and 1 directories from _tests.
 0:33.77 Deleted 0 files and 3 directories from config/makefiles/xpidl.
 0:33.77 Finished purging.
 0:33.84 /usr/bin/make  tier_base
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/ncalexan/Mozilla/mcgit/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 698, in _processOutput
    self.processOutputLine(line.rstrip())
  File "/Users/ncalexan/Mozilla/mcgit/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 663, in processOutputLine
    handler(line)
  File "/Users/ncalexan/Mozilla/mcgit/python/mach/mach/mixin/process.py", line 86, in handleLine
    line_handler(line)
  File "/Users/ncalexan/Mozilla/mcgit/python/mozbuild/mozbuild/mach_commands.py", line 266, in on_line
    self.log(logging.INFO, 'build_output', {'line': line}, '{line}')
  File "/Users/ncalexan/Mozilla/mcgit/python/mach/mach/mixin/logging.py", line 54, in log
    extra={'action': action, 'params': params})
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1213, in log
    self._log(level, msg, args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1268, in _log
    self.handle(record)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1278, in handle
    self.callHandlers(record)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 1318, in callHandlers
    hdlr.handle(record)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 749, in handle
    self.emit(record)
  File "/Users/ncalexan/Mozilla/mcgit/python/mozbuild/mozbuild/mach_commands.py", line 84, in emit
    self.footer.draw()
  File "/Users/ncalexan/Mozilla/mcgit/python/mozbuild/mozbuild/mach_commands.py", line 138, in draw
    for subtier, active, finished in tiers.current_subtier_status():
  File "/Users/ncalexan/Mozilla/mcgit/python/mozbuild/mozbuild/controller/building.py", line 166, in current_subtier_status
    for subtier, state in self.tiers[self.active_tier]['subtiers'].items():
KeyError: None

Comment 2

[Nick Alexander :nalexander [he/him]]

I have some patches pushed, but I can reproduce this with |./mach clobber && ./mach build && ./mach build -v|.

Comment 3

[Gregory Szorc [:gps]]

Patch in bug 911957, so this older bug gets duped.