1378663 - symbolstore.py produces indeterministic output

Status: RESOLVED WORKSFORME

(Thunderbird :: Build Config, defect)

Description

[Bernhard M. Wiedemann]

Attachment: patch to fix this

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20170504000000

Steps to reproduce:

building the thunderbird-52.2.1 package in the openSUSE Tumbleweed Linux distribution with disposable VMs using
osc build --vm-type=kvm


Actual results:

/usr/share/mozilla/thunderbird-52.2.1-0.0.x86_64-1330-symbols.zip
contains entries in partially random order (because filesystems return directory listings in random order) and the current 'sort' does not care about files with same sizes.

e.g. diff shows
 thunderbird-bin/507557B62B810AB2B9526CC41AC9EF540/thunderbird-bin.sym
-libclearkey.so/14347486AF1FB39B7ECB10C3BB64860A0/libclearkey.so.sym
 crashreporter/EF17DDE9475012F900E432384A5F8C9E0/crashreporter.sym
+libclearkey.so/14347486AF1FB39B7ECB10C3BB64860A0/libclearkey.so.sym
 plugin-container/C92A8E2333A589AF6D9DA6F1F14259FA0/plugin-container.sym
 xpcshell/F6F4DB4DB50E4B23A9685CA63AC1ED8A0/xpcshell.sym
-libfake.so/EFD23600F1348490A99C9DF270CEA3550/libfake.so.sym
 libmozavutil.so/C70DADE5B5F6D8B4E9E2EC6B146C88A40/libmozavutil.so.sym
+libfake.so/EFD23600F1348490A99C9DF270CEA3550/libfake.so.sym
 liblgpllibs.so/6CBD5E394B991B4E19A9FE60E70FC0D80/liblgpllibs.so.sym



Expected results:

symbols.zip content should not differ between builds.
See https://reproducible-builds.org/ for why this matters.

Comment 1

[Bernhard M. Wiedemann]

actually, the patch seems not to help. There are still such diffs in
thunderbird-52.2.1-0.0.x86_64-1330-symbols.txt
I'm testing some other patches...

Comment 2

[Bernhard M. Wiedemann]

Attachment: patch to fix this

Comment 3

[Tom Prince [:tomprince]]

This patch no longer applies to recent versions. It also looks like recent versions do sort the output. See https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozpack/files.py#907-909 called from https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozpack/files.py#907-909 from https://dxr.mozilla.org/mozilla-central/source/obj-x86_64-pc-linux-gnu/Makefile#290

This should be fixed in Thunderbird 55 when it gets released.