780835 - Intermittent build/macosx/universal/unify failure: "copyIfIdentical: files differ"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Mike Hommey [:glandium]]

Mac universal builds sometime fail with errors like this:
/builds/slave/m-in-osx64/build/build/macosx/universal/unify: copyIfIdentical: files differ:
  obj-firefox/i386/dist/test-package-stage/mochitest/extensions/specialpowers/chrome.manifest,
  obj-firefox/x86_64/dist/test-package-stage/mochitest/extensions/specialpowers/chrome.manifest

Comment 1

[(not currently active) Ted Mielczarek]

This is probably an line-ordering issue. Can we sort the contents? I added that feature to unify a while ago.

Comment 2

[Ed Morley [:emorley]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=14190297&tree=Mozilla-Inbound

Comment 3

[Mike Hommey [:glandium]]

(In reply to Ted Mielczarek [:ted] from comment #1)
> This is probably an line-ordering issue. Can we sort the contents? I added
> that feature to unify a while ago.

The full log is:
           /builds/slave/m-in-osx64/build/build/macosx/universal/unify                 \
             --unify-with-sort "all-test-dirs\.list$"               \
             obj-firefox/i386/dist/test-package-stage                          \
             obj-firefox/x86_64/dist/test-package-stage                          \
             obj-firefox/i386/dist/universal/test-package-stage; fi
/builds/slave/m-in-osx64/build/build/macosx/universal/unify: copyIfIdentical: files differ:
  obj-firefox/i386/dist/test-package-stage/mochitest/extensions/specialpowers/chrome.manifest,
  obj-firefox/x86_64/dist/test-package-stage/mochitest/extensions/specialpowers/chrome.manifest

The sorting option *is* used. So either it doesn't work, or there *are* differences in the files. Can someone check on the failing buildbots?

Comment 4

[Mike Hommey [:glandium]]

Ah, the argument that follows the option is a regexp for the file names to use it on.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Unify test manifest files with sort

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 649677 [details] [diff] [review]
Unify test manifest files with sort

Review of attachment 649677 [details] [diff] [review]:
-----------------------------------------------------------------

Sorting manifests doesn't screw us up with regards to the content, does it?

Comment 7

[Mike Hommey [:glandium]]

(In reply to Ted Mielczarek [:ted] from comment #6)
> Comment on attachment 649677 [details] [diff] [review]
> Unify test manifest files with sort
> 
> Review of attachment 649677 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Sorting manifests doesn't screw us up with regards to the content, does it?

We already sort them on non-test unification, see http://mxr.mozilla.org/mozilla-central/source/build/macosx/universal/flight.mk#77

Comment 8

[Mike Hommey [:glandium]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ce9504d8a2a2

Comment 9

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=14211783&tree=Mozilla-Inbound

/builds/slave/m-in-osx64/build/build/macosx/universal/unify: copyIfIdentical: files differ:
  obj-firefox/i386/dist/firefox/FirefoxNightly.app/Contents/MacOS/application.ini,
  obj-firefox/x86_64/dist/firefox/FirefoxNightly.app/Contents/MacOS/application.ini
make[2]: *** [postflight_all] Error 1


which is rather quite a lot later than that push.

Comment 10

[Phil Ringnalda (:philor)]

Hmm, and https://tbpl.mozilla.org/php/getParsedLog.php?id=14212839&tree=Mozilla-Inbound is the same slave, which probably means I need to clobber the lot of them, to make sure I get anybody who managed to hit the bustage.

Comment 11

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=14214222&tree=Mozilla-Inbound before I got to the clobberer.

Comment 12

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/ce9504d8a2a2