1212587 - Intermittent WebIDL.WebIDLError: error: Unresolved type '<unresolved scope>::External'., Window.webidl line 69:43

Status: NEW

(Firefox Build System :: General, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://treeherder.mozilla.org/logviewer.html#?job_id=15300100&repo=mozilla-inbound

Comment 1

[Boris Zbarsky [:bzbarsky]]

"interface External" is right there in External.webidl.

Now that webidl file is only looked at if CONFIG['MOZ_BUILD_APP'] in ['browser', 'mobile/android', 'xulrunner'] in dom/webidl/moz.build and the interface is used ifdef HAVE_SIDEBAR.  So maybe those got out of sync on this machine somehow?  HAVE_SIDEBAR is set in dom/base/moz.build like so:

467 if CONFIG['MOZ_BUILD_APP'] in ['browser', 'mobile/android', 'xulrunner']:
468     DEFINES['HAVE_SIDEBAR'] = True

Is it possible this box went from building one app to building another without clobbering or some such insanity?

Comment 2

[Intermittent Failures Robot]

12 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 7
* b2g-inbound: 5

Platform breakdown:
* android-4-0-armv7-api11: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1212587&startday=2015-10-05&endday=2015-10-11&tree=all

Comment 3

[Boris Zbarsky [:bzbarsky]]

OK, so some more info from IRC:

These taskcluster jobs are in fact switching back and forth between b2g-inbound and mozilla-inbound in a single objdir without clobbering.  If this is a supported mode of operation, then we need to teach the build system to rebuild more stuff here somehow, presumably.  If it's not, then the taskcluster setup needs to get changed to not do that...

Comment 4

[Gregory Szorc [:gps]]

Ideally we could switch between trees and reuse the object directory. While switching between trees is behaviorally no different than backing out dozens of changesets and applying dozens other ones, "going backwards" does compound the risk for build bustage and needed clobbers. I would strongly prefer we didn't do that.

That being said, this is only theoretically supported if the *same* source directory is used. The reason is file mtimes. If you are using multiple source directories, the mtime in the objdir may be newer than both source directories, even if there are diverging changes between srcdir content. This could lead to build rules not being evaluated and bugs like this.

If TaskCluster is using a single objdir and multiple source checkouts (i.e. not performing a `hg up` on the exact checkout that was used to produce the objdir), I consider this a P1 stop-the-world bug for TaskCluster because it will only confuse make and result in excessive build failures and incorrect builds.

Jonas: what is the exact behavior of TC w.r.t. source and object directory management?

Comment 5

[Jonas Finnemann Jensen (:jonasfj)]

> Jonas: what is the exact behavior of TC w.r.t. source and object directory management?
From the task definition we have 3 cache folders:
  "cache": {
    "tooltool-cache": "/home/worker/tooltool-cache",
    "tc-vcs": "/home/worker/.tc-vcs",
    "build-android-api-11-c6-workspace": "/home/worker/workspace"
  },
afaik "/home/worker/workspace" is the place that both source tree and object dir is located.
So this should be okay.

But garndt might know more on this subject...

Comment 6

[Greg Arndt [:garndt]]

As of this morning, a push was made to b2g-inbound [1]  that makes the workspaces for the build referenced in this bug branch specific.  The workspace in this type of build is both the source checkout along with object directory.

For b2g-desktop, android, and firefox workers, the workspaces should not be branch specific.  

Emulator and device image workers are a little different just because of the size of the workspaces and the available diskspace on the workers since the workers are shared amongst all branches.  The workspace only contains the source checkout, and that is *not* branch specific.  The objdir resides outside of the cached workspace and is cached per branch. This sample task [2] displays this behavior.

Are there issues with sharing a common source worksspace for all branches, but having branch specific object directories? If so, we will work at creating branch specific worker types immediately.


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1213878#c3
[2] https://tools.taskcluster.net/task-inspector/#ZuN7XntQTW-JjldgfWDTvA/

Comment 7

[Greg Arndt [:garndt]]

Sorry, I mean:

For b2g-desktop, android, and firefox workers, the workspaces (objdir+source together) *should* be branch specific.

Comment 8

[Gregory Szorc [:gps]]

[2] from comment #6 implies that the source directory and object directories are managed independently. This will only create bustage. Because build systems rely on file mtimes for checking what needs to be built, there needs to be a 1:1 relationship between source and object directories or you are going to have a bad time.

What you can do is pull all the repo remotes into the same local repo and `hg up` or `git checkout`. These commands will update mtimes so build times don't get confused. However, this will inevitably lead to a lot of clobbers, especially if you are going between e.g. mozilla-central and mozilla-aurora. That's why it's best to have one srcdir+objdir for every head/branch and for that "workspace" to always fast forward.

Comment 9

[Mats Palmgren (inactive)]

Uhm, I get this error in my mozilla-central Firefox Linux debug build now... :-(
"rm -rf $OBJDIR" does not help.

Comment 10

[Mats Palmgren (inactive)]

My error might be different actually, here it is in all its glory:

 0:34.72 nsCSSPropsGenerated.inc
 0:34.82 Traceback (most recent call last):
 0:34.82   File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
 0:34.82     "__main__", fname, loader, pkg_name)
 0:34.82   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
 0:34.82     exec code in run_globals
 0:34.82   File "/md1/src/mc-asan/python/mozbuild/mozbuild/action/webidl.py", line 19, in <module>
 0:34.82     sys.exit(main(sys.argv[1:]))
 0:34.82   File "/md1/src/mc-asan/python/mozbuild/mozbuild/action/webidl.py", line 15, in main
 0:34.82     manager.generate_build_files()
 0:34.82   File "/md1/src/mc-asan/dom/bindings/mozwebidlcodegen/__init__.py", line 248, in generate_build_files
 0:34.82     self._parse_webidl()
 0:34.82   File "/md1/src/mc-asan/dom/bindings/mozwebidlcodegen/__init__.py", line 335, in _parse_webidl
 0:34.82     self._parser_results = parser.finish()
 0:34.82   File "/md1/src/mc-asan/dom/bindings/parser/WebIDL.py", line 6833, in finish
 0:34.82     production.finish(self.globalScope())
 0:34.82   File "/md1/src/mc-asan/dom/bindings/parser/WebIDL.py", line 929, in finish
 0:34.82     member.finish(scope)
 0:34.82   File "/md1/src/mc-asan/dom/bindings/parser/WebIDL.py", line 4736, in finish
 0:34.82     argument.complete(scope)
 0:34.82   File "/md1/src/mc-asan/dom/bindings/parser/WebIDL.py", line 4348, in complete
 0:34.82     type = self.type.complete(scope)
 0:34.82   File "/md1/src/mc-asan/dom/bindings/parser/WebIDL.py", line 2168, in complete
 0:34.82     [self.location])
 0:34.83 WebIDL.WebIDLError: error: Unresolved type '<unresolved scope>::VoidFunction'., /md1/src/mc-asan/dom/webidl/AudioWorkletGlobalScope.webidl line 12:44
 0:34.83     void registerProcessor (DOMString name, VoidFunction processorCtor);
 0:34.83                                             ^
 0:34.83 unixcharset.properties.h
 0:35.00 Makefile:55: recipe for target 'codegen.pp' failed
 0:35.00 make[5]: *** [codegen.pp] Error 1

Comment 11

[Boris Zbarsky [:bzbarsky]]

Mats, that doesn't seem obviously related to this bug, if it's not intermittent.

That said, what's your build config?  VoidFunction is used in AudioWorkletGlobalScope.webidl as of a few days ago, which is built unconditionally, but it's _defined_ in RTCPeerConnection.webidl, which is only built if MOZ_WEBRTC is defined.  So if you're disabling webrtc, you would absolutely hit that error.

Comment 12

[Boris Zbarsky [:bzbarsky]]

I filed bug 1324169 on that.

Comment 13

[Mats Palmgren (inactive)]

Ah, yes, I have --disable-webrtc in my mozconfig.  Sorry about the noise.