909147 - Build error during "make check": [run-selftest.py] Error 1 ("assert current_type is not None")

Status: RESOLVED FIXED

Description

[Frank Wein [:mcsmurf]]

Starting with this checkin https://hg.mozilla.org/comm-central/rev/51d7c81f566c (certainly not related to the build problem) the Thunderbird build boxen fail while running "make check"/run-selftest.py looks like:
[...]
make -C ssltunnel check
make[3]: Nothing to be done for `check'.
make -C testing/xpcshell check
Traceback (most recent call last):
  File "/builds/slave/tb-c-cen-osx64-d-0000000000000/build/mozilla/testing/xpcshell/selftest.py", line 13, in <module>
    build_obj = MozbuildObject.from_environment()
  File "/builds/slave/tb-c-cen-osx64-d-0000000000000/build/mozilla/python/mozbuild/mozbuild/base.py", line 156, in from_environment
    config = loader.read_mozconfig(mozconfig)
  File "/builds/slave/tb-c-cen-osx64-d-0000000000000/build/mozilla/python/mozbuild/mozbuild/mozconfig.py", line 198, in read_mozconfig
    parsed = self._parse_loader_output(output)
  File "/builds/slave/tb-c-cen-osx64-d-0000000000000/build/mozilla/python/mozbuild/mozbuild/mozconfig.py", line 306, in _parse_loader_output
    assert current_type is not None
AssertionError
make[2]: *** [run-selftest.py] Error 1
make[2]: Target `check' not remade because of errors.
make -C example check
make[3]: Nothing to be done for `check'.
make[1]: *** [check] Error 2
make -C js/src check
make -C config check
[...]
(from https://tbpl.mozilla.org/php/getParsedLog.php?id=26959400&tree=Thunderbird-Trunk)

Linux and OS X are affected, not sure about Windows (hard-to-find error message, but it's also orange). The affected builds are (free-space) clobber builds according to tbpl.

Comment 1

[Joshua Cranmer [:jcranmer]]

Attachment: Resolve the objdir properly in comm-central

Ah, the twin-objdir pain of comm-central. This makes the test pass for me locally, and Firefox is still passing, so this should fix the bustage.

Comment 2

[Frank Wein [:mcsmurf]]

This patch might also fix Bug 907434 then.

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 797068 [details] [diff] [review]
Resolve the objdir properly in comm-central

Review of attachment 797068 [details] [diff] [review]:
-----------------------------------------------------------------

Test coverage would be nice. But meh.

Comment 4

[Joshua Cranmer [:jcranmer]]

   https://hg.mozilla.org/integration/mozilla-inbound/rev/fca5ac9f6329

Comment 5

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/fca5ac9f6329

Comment 6

[Joshua Cranmer [:jcranmer]]

Sigh. Don't you hate it when the error you get locally is not the same error you get on tinderbox?

Comment 7

[Joshua Cranmer [:jcranmer]]

Okay, the problem is reproduced if you add this to the mozconfig:
mk_add_options CLIENT_PY_ARGS="$(cat $topsrcdir/build/client.py-args)"

In-depth investigation reveals that $topsrcdir is pointing to mozilla-central where the mozconfig expects comm-central. Ain't our build system fun?

Comment 8

[Hiroyuki Ikezoe (:hiro)]

Attachment: A fix

Ugly, but I have no other idea.

Comment 9

[Hiroyuki Ikezoe (:hiro)]

try server results:

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=92de55f4496f

Comment 10

[Mark Banner (:standard8)]

Comment on attachment 804162 [details] [diff] [review]
A fix

Hmm, I'd like Joshua's feedback here as he's been looking at this as well.

Comment 11

[Joshua Cranmer [:jcranmer]]

Comment on attachment 804162 [details] [diff] [review]
A fix

This isn't a fix, it's wallpaper. The problem is that I don't know what the proper fix is. mozinfo.json has the wrong topsrcdir for setting $topsrcdir in .mozconfig, but it's probably the right topsrcdir for a lot of things too. Ultimately, the people most affected by topsrcdir issues are SeaMonkey's mochitests, apparently, but I honestly don't know what's up there or what "properly" fixing topsrcdir issues in mozbuild places would cause.

The best fix ultimately is ccrework. I guess the main question is if this wallpaper is sufficient to last us until ccrework makes all these problems go away or if we need to actually fix this for real. Considering that this is basically the patch I was going to write anyways to fix his bug (Plan B being plug my ears and ignoring this altogether), if Standard8 is asking me to review this, and this is passing try, then I guess there's no reason to reject it.

Comment 12

[Frank Wein [:mcsmurf]]

Just for reference: The first patch (Attachment 797068 [details] [diff]) did not resolve the mochitest problems from Bug 907434 (as noted in Bug 907434 Comment 4). Which is a bit surprising, it looks like msys/mozbuild is not able to properly resolve relative dirs (NTFS symbol links) in the comm-central case. Seems to work fine though with mozilla-central, not really sure what's different there regarding resolving path.

Comment 13

[Joshua Cranmer [:jcranmer]]

https://hg.mozilla.org/comm-central/rev/e64bd128bee3