1603561 - IOError: Missing files when running `mach try`

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P1)

Description

[Andrew Halberstadt [:ahal]]

Smaug was having issues generating the taskgraph for ./mach try chooser (note this will also affect ./mach try fuzzy):
https://paste.mozilla.org/t71UhKQp

The traceback shows the issue is happening during test resolving, so this is fallout from the most recent landing in bug 1583353. I don't seem to be able to reproduce, but I'll take a deeper look.

Comment 1

[Andrew Halberstadt [:ahal]]

I know what's happening:

1. As of bug 1583353, taskgraph generation invokes reader.all_mozbuild_paths:
   https://searchfox.org/mozilla-central/rev/923eec8d2fb8078ebc7a33a9e1ce73eac01f7446/python/mozbuild/mozbuild/frontend/reader.py#896
2. This function in turn uses a FileFinder that tries to exclude object directories that might appear in you srcdir:
   https://searchfox.org/mozilla-central/rev/923eec8d2fb8078ebc7a33a9e1ce73eac01f7446/python/mozbuild/mozbuild/frontend/reader.py#846
3. However, your object directory doesn't start with obj*, so the FileFinder is erroneously searching it for moz.build files.
4. The objdir contains a <objdir>/dist/bin/browser/features/[e-mail]/test/browser/moz.build file which registers the browser.ini file that the traceback is complaining about.

While I figure out a way to solve this, you have two short term workarounds:
A) Add a line to the list in 2) that matches your objdirs.
B) Rename/move your objdirs so they start with obj* (e.g, move them to <srcdir>/objdirs/<objdir>)

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1603561 - [mozbuild] Ignore all potential objdirs in BuildReader._relevant_mozbuild_finder, r?#firefox-build-system-reviewers

Currently BuildReader._relevant_mozbuild_finder attempted to exclude objdirs by
ignoring the pattern 'obj*'. However this can cause problems for developers who
don't prefix their objdirs with the string 'obj'. This attempts to ignore all
objdirs by looking for 'config.status' files in the topsrcdir.

This wasn't a huge issue before because BuildReader._relevant_mozbuild_finder
was only used in a handful of obscure edge cases. But now it's being invoked by
the taskgraph and consequently |mach try|. So we'll see this issue pop up for
people more frequently.

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

FWIW, I just used the patch so that I could push something to tryserver. Seemed to work.
Thanks :)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/00ed15d902df
[mozbuild] Ignore all potential objdirs in BuildReader._relevant_mozbuild_finder, r=firefox-build-system-reviewers,chmanchester

Comment 5

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/00ed15d902df