1428409 - [Mac] "mach build package" Often fails, requiring clobber to get a package build

Status: NEW

(Firefox Build System :: General, defect)

Description

[Haik Aftandilian [:haik]]

I often run into a situation where "mach build package" fails with an error like what is pasted below. I most recently hit this in my repo based on 397556:351c75ab74c9. I have my own changes, but I don't think they're related to the problem.

$ hg log -r "max(not outgoing())"
changeset:   397556:351c75ab74c9
...

$ MOZCONFIG=./mozconfig.opt ./mach build
<completes successfully>

$ MOZCONFIG=./mozconfig.opt ./mach build package
 0:00.53 /usr/bin/make -C /Users/haftandilian/r/mozilla-central/obj-opt.noindex -j8 -s backend
 0:00.77 /usr/bin/make -j8 -s package
 0:05.18 Traceback (most recent call last):
 0:05.18   File "/Users/haftandilian/r/mozilla-central/toolkit/mozapps/installer/packager.py", line 342, in <module>
 0:05.18     main()
 0:05.18   File "/Users/haftandilian/r/mozilla-central/toolkit/mozapps/installer/packager.py", line 336, in main
 0:05.18     copier.copy(args.destination)
 0:05.18   File "/Users/haftandilian/r/mozilla-central/python/mozbuild/mozpack/copier.py", line 431, in copy
 0:05.19     copy_results.append((destfile, f.copy(destfile, skip_if_older)))
 0:05.19   File "/Users/haftandilian/r/mozilla-central/python/mozbuild/mozpack/copier.py", line 604, in copy
 0:05.19     file.copy(deflater, skip_if_older)
 0:05.19   File "/Users/haftandilian/r/mozilla-central/python/mozbuild/mozpack/files.py", line 189, in copy
 0:05.19     shutil.copyfileobj(self.open(), dest)
 0:05.19   File "/Users/haftandilian/r/mozilla-central/python/mozbuild/mozpack/files.py", line 217, in open
 0:05.19     return open(self.path, 'rb')
 0:05.19 IOError: [Errno 2] No such file or directory: '../../dist/Nightly.app/Contents/Resources/chrome/toolkit/skin/classic/global/arrow/panelarrow-horizontal.png'
 0:05.26 make[3]: *** [stage-package] Error 1
 0:05.26 make[2]: *** [make-package] Error 2
 0:05.26 make[1]: *** [default] Error 2
 0:05.26 make: *** [package] Error 2

$ grep -v "^#" mozconfig.opt
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-opt.noindex
mk_add_options AUTOCLOBBER=1
mk_add_options AUTOCONF=/usr/local/Cellar/autoconf213/2.13/bin/autoconf213
export MOZ_TELEMETRY_REPORTING=1
ac_add_options --enable-optimize
ac_add_options --with-ccache
ac_add_options --enable-dtrace
ac_add_options --enable-tests
ac_add_options --enable-eme=widevine

Comment 1

[(not currently active) Ted Mielczarek]

I assume this is due to dangling symlinks winding up in the app bundle. Is the file referenced in the error message a dangling symlink? If not there's something stranger happening.

The Makefile that generates the app bundle is a bunch of hacks with rsync, it's not very smart:
https://dxr.mozilla.org/mozilla-central/rev/351c75ab74c9a83db5c0662ba271b49479adb1f1/browser/app/Makefile.in#79

Comment 2

[Haik Aftandilian [:haik]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #1)
> I assume this is due to dangling symlinks winding up in the app bundle. Is
> the file referenced in the error message a dangling symlink? If not there's
> something stranger happening.
> 
> The Makefile that generates the app bundle is a bunch of hacks with rsync,
> it's not very smart:
> https://dxr.mozilla.org/mozilla-central/rev/
> 351c75ab74c9a83db5c0662ba271b49479adb1f1/browser/app/Makefile.in#79

Thanks! Yes, panelarrow-horizontal.png was a dangling symlink and after removing all dangling symlinks from the object dir "build package" finished successfully.

There are a couple of rsync commands there that don't use -L (--copy-links) which appears to result in skipping broken symlinks instead of failing. I don't have my failing state anymore, but could try adding that next time.