1601777 - mach build faster breaks after filename change on android

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Brendan Dahl [:bdahl]]

STR:

1. ./mach build non-artifact android build of mozilla/central e.g. 778b6b11194c072d2603e58118aaf6959e98902f
2. apply patch that renames the file geckoview.xul to geckoview.html
3. ./mach build again
4. ./mach build faster

I get the following error:

 0:00.82 Traceback (most recent call last):
 0:00.82   File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
 0:00.82     "__main__", fname, loader, pkg_name)
 0:00.82   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
 0:00.82     exec code in run_globals
 0:00.82   File "/home/bdahl/projects/gecko/python/mozbuild/mozbuild/action/process_install_manifest.py", line 114, in <module>
 0:00.82     main(sys.argv[1:])
 0:00.82   File "/home/bdahl/projects/gecko/python/mozbuild/mozbuild/action/process_install_manifest.py", line 100, in main
 0:00.82     defines=args.defines)
 0:00.82   File "/home/bdahl/projects/gecko/python/mozbuild/mozbuild/action/process_install_manifest.py", line 69, in process_manifest
 0:00.82     remove_empty_directories=remove_empty_directories)
 0:00.82   File "/home/bdahl/projects/gecko/python/mozbuild/mozpack/copier.py", line 432, in copy
 0:00.82     copy_results.append((destfile, f.copy(destfile, skip_if_older)))
 0:00.82   File "/home/bdahl/projects/gecko/python/mozbuild/mozpack/files.py", line 364, in copy
 0:00.82     raise ErrorMessage('Symlink target path does not exist: %s' % self.path)
 0:00.82 mozpack.errors.ErrorMessage: Symlink target path does not exist: /home/bdahl/projects/gecko/mobile/android/chrome/geckoview/geckoview.xul
 0:00.83 /home/bdahl/projects/gecko/config/faster/rules.mk:86: recipe for target 'install-dist/bin' failed

Comment 1

[Chris Manchester (limited bugmail, email directly)]

I can reproduce on desktop by renaming a random file in jar.mn. The issue seems to be that while the build system knows the build backend depends on jar.mn, the faster target in particular (vs. ./mach build with the backend selected) doesn't go through the backend out of date check correctly because the backend used to build isn't the same as the one we think is configured.

Comment 2

[Nico Grunbaum [:ng, @chew:mozilla.org]]

A regular desktop build on linux (non-artifact) is failing for me this evening with this trace. All I have to do is run mach mochitest test_peerConnection_stats and subsequent builds yield that error. My patch set does add a file, and it has been added to the relevant moz.build file. Any clues why this may be happening?

Comment 3

[Michael Shal [:mshal]]

Attachment: Bug 1601777 - Check if any build backend is out of date; r?#firefox-build-system-reviewers

By default we create both the RecursiveMake and FasterMake backends, but
building.py is only checking if the RecursiveMake backend is out of date
to decide if the backend needs to be updated. Since the FasterMake
backend may have different input files, we also need to check if that's
out of date, otherwise renaming certain files can cause './mach build
faster' to break.

Comment 4

[Michael Shal [:mshal]]

I believe this patch fixes the problem for the Android case above. It seems we weren't checking if the FasterMake backend was out of date, which meant the install manifest had stale entries. :ng - can you verify that it also works for you? If not, it might be a separate issue.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c21846170569
Check if any build backend is out of date; r=froydnj

Comment 6

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/c21846170569

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Now I am getting something different:

 0:00.41 /usr/bin/make -f client.mk MOZ_PARALLEL_BUILD=15 -s
 0:00.52 Traceback (most recent call last):
 0:00.52   File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
 0:00.52     "__main__", fname, loader, pkg_name)
 0:00.52   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
 0:00.52     exec code in run_globals
 0:00.52   File "/home/ng/sync/hg/mozilla-unified/python/mozbuild/mozbuild/action/process_install_manifest.py", line 114, in <module>
 0:00.52     main(sys.argv[1:])
 0:00.52   File "/home/ng/sync/hg/mozilla-unified/python/mozbuild/mozbuild/action/process_install_manifest.py", line 100, in main
 0:00.52     defines=args.defines)
 0:00.52   File "/home/ng/sync/hg/mozilla-unified/python/mozbuild/mozbuild/action/process_install_manifest.py", line 74, in process_manifest
 0:00.52     manifest.write(path=track, expand_pattern=True)
 0:00.52   File "/home/ng/sync/hg/mozilla-unified/python/mozbuild/mozpack/manifests.py", line 252, in write
 0:00.52     six.ensure_text(p) for p in parts))
 0:00.52   File "/home/ng/sync/hg/mozilla-unified/python/mozbuild/mozpack/manifests.py", line 252, in <genexpr>
 0:00.52     six.ensure_text(p) for p in parts))
 0:00.52 AttributeError: 'module' object has no attribute 'ensure_text'

Comment 8

[Ricky Stewart]

That's a different thing; see bug 1605128. You should be able to fix it by deleting the _virtualenvs directory in your objdir, or, if that fails, deleting the entire objdir.