On this push: https://tbpl.mozilla.org/?tree=Fx-Team&rev=410f509e9913&jobname=gaia-unit B2G desktop gaia-unit tests were run twice (the second being a retrigger). On the first run, the tests runs prior to ftu/test/unit/tutorial_test.js (the failing test under investigation) were: https://tbpl.mozilla.org/php/getParsedLog.php?id=41416128&tree=Fx-Team ... communications/contacts/test/unit/contacts_merger_test.js communications/contacts/test/unit/navigation_test.js ftu/test/unit/wifi_test.js ftu/test/unit/tutorial_test.js For the second run: https://tbpl.mozilla.org/php/getParsedLog.php?id=41427395&tree=Fx-Team ... ftu/test/unit/operatorVariant_test.js ftu/test/unit/wifi_test.js ftu/test/unit/ui_manager_test.js ftu/test/unit/tutorial_test.js Given that both of these jobs are on the same gaia and gecko revisions, the list of enabled tests should be the same. We appear to walk the filesystem here: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-unit-tests/gaia_unit_test/main.py#L223 I would have thought this would be deterministic - though I guess we could be mangling it later?
Ah, on: https://blog.torproject.org/category/tags/deterministic-builds > Reordering due to inode ordering differences (exposed via Python's os.walk()) > Several places in the Firefox build process use python scripts to repackage both compiled > library archives and zip files. In particular, they tend to obtain directory listings > using os.walk(), which is dependent upon the inode ordering of the filesystem. > We fix this by sorting those file lists in the applicable places.
It appears that elsewhere we do sort after the os.walk(): http://mxr.mozilla.org/mozilla-central/source/js/src/tests/lib/jittests.py#196 Though will likely be worth filing a meta bug once this is fixed, to vet other uses throughout the tree.
Created attachment 8439507 [details] [review] Sort Gu tests after os.walk
Comment on attachment 8439507 [details] [review] Sort Gu tests after os.walk ty!