Closed Bug 1526744 Opened 7 years ago Closed 6 years ago

Build of Thunderbird ESR 60.x is failing on mips and mipsel

Categories

(Thunderbird :: Build Config, defect)

Product:
Thunderbird
Component:
Build Config
Platform:
Other
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 70.0

People

(Reporter: c.schoenert, Assigned: rjl)

References

Details

Attachments

(3 files)

md5_by_chunk.patch
2.04 KB, patch
Details | Diff | Splinter Review
md5_chunk.py
857 bytes, text/plain
Details
Bug 1526744 - Read files in chunks when calculating md5 in find_dupes.
47 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0

Steps to reproduce:

Current versions of Thunderbird ESR releases are not build able on 32bit architectures anymore in Debian without some special handling. Thanks to Mike Homey and his additions to the build configuration for Firefox which I've cherry picked into the build configuration for Thunderbird the build is successful again for most of the 32bit RC (release critical) platforms in Debian. So Thunderbird can get build on armel, armhf, i386 but not on mips and mipsel.

https://buildd.debian.org/status/package.php?p=thunderbird

The build on mipsel will be the same as on mips, I tested this before on a Debian porterbox.

Actual results:

If I look into the output of the build process it's visible the main build of the binaries is successful, but the reducing of duplicated files is failing.

https://buildd.debian.org/status/fetch.php?pkg=thunderbird&arch=mips&ver=1%3A60.5.0-3&stamp=1549416769&raw=0

This is happen due the call of

content = f.open().read()

which is ending in a memory error as the file is probably to big to load completely into the memory. The complete error message is

install thunderbird into temp install folder

./mach -v install
0:01.24 /usr/bin/make -C . -j4 -s -w install
0:01.24 Environment: {'CPPFLAGS': '-Wdate-time -D_FORTIFY_SOURCE=2', 'BUILD_DATE': 'Tue, 05 Feb 2019 17:11:25 +0100', 'DEB_BUILD_GNU_SYSTEM': 'linux-gnu', 'DEB_TARGET_ARCH_OS': 'linux', 'DEB_TARGET_MULTIARCH': 'mips-linux-gnu', 'SCHROOT_UID': '2952', 'DEB_BUILD_MULTIARCH': 'mips-linux-gnu', 'MOZCONFIG': '/<<PKGBUILDDIR>>/mozconfig.thunderbird', 'PERL_USE_UNSAFE_INC': '1', 'DEB_BUILD_ARCH_CPU': 'mips', 'DEB_HOST_ARCH_OS': 'linux', 'CXXFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -fno-delete-null-pointer-checks -fno-schedule-insns2 -fno-lifetime-dse', 'SCHROOT_CHROOT_NAME': 'sid-mips-sbuild', 'FAKEROOTKEY': '1739468122', 'LOGNAME': 'buildd', 'USER': 'buildd', 'OBJCFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security', 'DEB_TARGET_GNU_CPU': 'mips', 'DH_INTERNAL_OPTIONS': '-a', 'HOME': '/sbuild-nonexistent', 'LD_LIBRARY_PATH': '/usr/lib/mips-linux-gnu/libfakeroot:/usr/lib64/libfakeroot:/usr/lib32/libfakeroot', 'MAKEFLAGS': 'w', 'DEB_HOST_ARCH_ENDIAN': 'big', 'DEB_BUILD_ARCH_BITS': '32', 'SCHROOT_SESSION_ID': 'sid-mips-sbuild-3c313439-768f-4231-9976-a83ed47ba530', 'MOZ_BUILD_DATE': '1549383085', 'DEB_TARGET_ARCH_LIBC': 'gnu', 'DEB_BUILD_ARCH_ENDIAN': 'big', 'MAKELEVEL': '2', 'SHLVL': '0', 'DEB_HOST_GNU_TYPE': 'mips-linux-gnu', 'DEB_HOST_GNU_CPU': 'mips', 'DEB_TARGET_ARCH_BITS': '32', 'SCHROOT_ALIAS_NAME': 'sid-mips-sbuild', 'SHELL': '/bin/bash', 'SCHROOT_USER': 'buildd', 'LDFLAGS': '-Wl,-z,relro -Wl,--reduce-memory-overheads -Wl,--no-keep-memory -Wl,--as-needed -Wl,--stats', 'DEB_BUILD_ARCH_OS': 'linux', 'SCHROOT_GID': '1009', 'DEB_TARGET_GNU_TYPE': 'mips-linux-gnu', 'SCHROOT_GROUP': 'buildd', 'MFLAGS': '-w', 'SOURCE_DATE_EPOCH': '1549383085', 'DEB_HOST_ARCH_LIBC': 'gnu', 'DEB_HOST_GNU_SYSTEM': 'linux-gnu', 'XDG_RUNTIME_DIR': '/run/user/2952', 'SCHROOT_COMMAND': 'dpkg-buildpackage -us -uc -mmips Build Daemon (mips-sil-01) <buildd_mips-mips-sil-01@buildd.debian.org> -B -rfakeroot', 'APT_CONFIG': '/var/lib/sbuild/apt.conf', 'FFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong', 'DEB_HOST_ARCH_BITS': '32', 'DH_INTERNAL_OVERRIDE': 'dh_auto_install', 'DEB_HOST_ARCH': 'mips', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games', 'FAKED_MODE': 'unknown-is-root', 'MOZILLA_OFFICIAL': '1', 'DEB_BUILD_ARCH_ABI': 'base', 'DEB_BUILD_GNU_TYPE': 'mips-linux-gnu', 'DEB_BUILD_ARCH': 'mips', 'LC_ALL': 'POSIX', 'XDG_SESSION_ID': 'c68714', 'DEB_BUILD_OPTIONS': 'parallel=4', '_': './mach', 'DH_INTERNAL_BUILDFLAGS': '1', 'LD_PRELOAD': 'libfakeroot-sysv.so', 'DEB_HOST_MULTIARCH': 'mips-linux-gnu', 'DEB_HOST_ARCH_ABI': 'base', 'MACH': u'1', 'MOZ_LANGPACK_ID': 'thunderbird.mozilla.org', 'FCFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong', 'DEB_HOST_ARCH_CPU': 'mips', 'DEB_RULES_REQUIRES_ROOT': 'binary-targets', 'GCJFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong', 'PWD': '/<<PKGBUILDDIR>>', 'DEB_BUILD_GNU_CPU': 'mips', 'DEB_TARGET_ARCH_ENDIAN': 'big', 'CFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -fno-delete-null-pointer-checks -fno-schedule-insns2 -fno-lifetime-dse', 'OBJCXXFLAGS': '-g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security', 'DEB_TARGET_ARCH_CPU': 'mips', 'DEB_TARGET_GNU_SYSTEM': 'linux-gnu', 'DEB_TARGET_ARCH': 'mips', 'DEB_TARGET_ARCH_ABI': 'base', 'DEB_BUILD_ARCH_LIBC': 'gnu'}
0:01.26 make[2]: Entering directory '/<<PKGBUILDDIR>>/obj-thunderbird'
0:01.56 make[3]: Entering directory '/<<PKGBUILDDIR>>/obj-thunderbird/comm/mail/installer'
1:04.31 Traceback (most recent call last):
1:04.31 File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/find-dupes.py", line 120, in <module>
1:04.31 main()
1:04.31 File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/find-dupes.py", line 117, in main
1:04.31 find_dupes(args.directory, bail=not args.warning, allowed_dupes=allowed_dupes)
1:04.31 File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/find-dupes.py", line 47, in find_dupes
1:04.31 content = f.open().read()
1:04.32 MemoryError
1:04.37 make[3]: *** [/<<PKGBUILDDIR>>/toolkit/mozapps/installer/packager.mk:23: stage-package] Error 1
1:04.37 make[3]: Leaving directory '/<<PKGBUILDDIR>>/obj-thunderbird/comm/mail/installer'
1:04.37 make[2]: *** [/<<PKGBUILDDIR>>/comm/mail/build.mk:16: install] Error 2
1:04.37 make[2]: Leaving directory '/<<PKGBUILDDIR>>/obj-thunderbird'

Expected results:

I've no idea about the build process in detail here, but if it would be possible to not load the whole file and just load some line (or chunks of lines) I guess this memory bottleneck could be bypassed an builds on mips 32bit architectures can be successful too.

OS: Unspecified → Linux
Hardware: Unspecified → Other

Since this is likely in shared code (with Firefox), doesn't Firefox have the same problem?

I guess yes, but FF-ESR 60.5.1 isn't reaching this part of the build process yet in Debian. Maintainer in Debian is Mike Hommey who is also working for Mozilla as you may now. ;) Maybe his has already worked in this corner?

https://buildd.debian.org/status/fetch.php?pkg=firefox-esr&arch=mips&ver=60.5.1esr-1&stamp=1550150119&raw=0
https://buildd.debian.org/status/fetch.php?pkg=firefox-esr&arch=mipsel&ver=60.5.1esr-1&stamp=1550321497&raw=0

For FF 65 is basically the same, but the build is failing on other parts. Currently I'm unable start any useful build process with TB 66.0b1 on mips so I can say anything to the situation of Thunderbird in the recent beta. The build stops early within the configuration run because of a not found libbz2 library (which is of course installed). I haven't found time to investigate any reason of this failure.

https://buildd.debian.org/status/fetch.php?pkg=thunderbird&arch=mips&ver=1%3A66.0%7Eb1-1&stamp=1550412098&raw=0

Carsten,

Can you see if this patch helps? What's happening there is an md5 checksum is being calculated, and to do so the entire file is being read into RAM at once. This just does the same hash calculation but reads the file in chunks so it won't take up nearly as much RAM for large files.

You may have to adjust the chunk_size, the value I picked is used elsewhere in the build code.

This is a new function rather than using an existing implementation because we also need the size of the file later on and due to idiosyncrasies in the various mozpack.files classes that could turn up, this was the cleanest way of getting the size.

Assignee: nobody → rob

Hello Rob,

yes will do a test build including your patch. Hope I'm able to start and monitor this on the weekend.

Rob,
I did a testing build in a mipsel chroot (native mips64el) and the build looks good on this specific part.

...
Packing ../debian/thunderbird-l10n-build/langpack-zh-TW@thunderbird.mozilla.org.xpi
Packed XPI file. It is located in /home/tijuca/thunderbird-60.5.1/debian/thunderbird-l10n-build
make[1]: Leaving directory '/home/tijuca/thunderbird-60.5.1'
   dh_auto_test
   create-stamp debian/debhelper-build-stamp
 fakeroot debian/rules binary
dh binary
   dh_testroot
   dh_prep
   dh_installdirs
   debian/rules override_dh_auto_install
make[1]: Entering directory '/home/tijuca/thunderbird-60.5.1'
# install thunderbird into temp install folder
./mach -v install
 0:00.96 /usr/bin/make -C . -j4 -s -w install
 0:00.96 Environment: {'CPPFLAGS': '-Wdate-time -D_FORTIFY_SOURCE=2', 'BUILD_DATE': 'Thu, 14 Feb 2019 20:01:03 +0100', 'DEB_BUILD_GNU_SYSTEM': 'linux-gnu', 'DEB_TARGET_ARCH_OS': 'linux', 'DEB_TARGET_MULTIARCH': 'mipsel-linux-gnu', 'SCHROOT_UID': '3488', 'DEB_BUILD_MULTIARCH': 'mipsel-linux-gnu', 'MOZCONFIG': '/home/tijuca/thunderbird-60.5.1/mozconfig.thunderbird', 'PERL_USE_UNSAFE_INC': '1', 'DEB_BUILD_ARCH_CPU': 'mipsel', 'DEB_HOST_ARCH_OS': 'linux', 'CXXFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong -Wformat -Werror=format-security -fno-delete-null-pointer-checks -fno-schedule-insns2 -fno-lifetime-dse', 'SCHROOT_CHROOT_NAME': 'sid_mipsel-dchroot', 'FAKEROOTKEY': '837806447', 'LOGNAME': 'tijuca', 'USER': 'tijuca', 'OBJCFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong -Wformat -Werror=format-security', 'DEB_TARGET_GNU_CPU': 'mipsel', 'DH_INTERNAL_OPTIONS': '', 'HOME': '/home/tijuca', 'LD_LIBRARY_PATH': '/usr/lib/mipsel-linux-gnu/libfakeroot:/usr/lib64/libfakeroot:/usr/lib32/libfakeroot', 'MAKEFLAGS': 'w -j4 --jobserver-auth=3,4', 'DEB_HOST_ARCH_ENDIAN': 'little', 'DEB_BUILD_ARCH_BITS': '32', 'TERM': 'xterm-256color', 'SCHROOT_SESSION_ID': 'sid_mipsel-dchroot-67a29f94-43b2-49a6-9e6f-dba8f3604741', 'MOZ_BUILD_DATE': '1550170863', 'DEB_TARGET_ARCH_LIBC': 'gnu', 'DEB_BUILD_ARCH_ENDIAN': 'little', 'MAKELEVEL': '2', 'MAKE_TERMERR': '/dev/pts/1', 'DEB_HOST_GNU_TYPE': 'mipsel-linux-gnu', 'DEB_HOST_GNU_CPU': 'mipsel', 'DEB_TARGET_ARCH_BITS': '32', 'SCHROOT_ALIAS_NAME': 'sid_mipsel-dchroot', 'SHELL': '/bin/bash', 'SCHROOT_USER': 'tijuca', 'LDFLAGS': '-Wl,-z,relro -Wl,--reduce-memory-overheads -Wl,--no-keep-memory -Wl,--as-needed -Wl,--stats', 'DEB_BUILD_ARCH_OS': 'linux', 'SCHROOT_GID': '3488', 'DEB_TARGET_GNU_TYPE': 'mipsel-linux-gnu', 'SCHROOT_GROUP': 'tijuca', 'MFLAGS': '-w -j4 --jobserver-auth=3,4', 'SOURCE_DATE_EPOCH': '1550170863', 'DEB_HOST_ARCH_LIBC': 'gnu', 'DH_INTERNAL_OVERRIDE': 'dh_auto_install', 'DEB_HOST_GNU_SYSTEM': 'linux-gnu', 'DEB_HOST_MULTIARCH': 'mipsel-linux-gnu', 'SCHROOT_COMMAND': '-bash', 'FFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong', 'DEB_HOST_ARCH_BITS': '32', 'SHLVL': '1', 'OBJCXXFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong -Wformat -Werror=format-security', 'DEB_HOST_ARCH': 'mipsel', 'PATH': '/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games', 'FAKED_MODE': 'unknown-is-root', 'MOZILLA_OFFICIAL': '1', 'DEB_BUILD_ARCH_ABI': 'base', 'DEB_BUILD_GNU_TYPE': 'mipsel-linux-gnu', 'DEB_BUILD_ARCH': 'mipsel', 'FCFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong', 'MACH_STDOUT_ISATTY': '1', 'DEB_TARGET_GNU_SYSTEM': 'linux-gnu', 'DEB_BUILD_OPTIONS': 'parallel=4', '_': './mach', 'DH_INTERNAL_BUILDFLAGS': '1', 'LD_PRELOAD': 'libfakeroot-sysv.so', 'DEB_HOST_ARCH_ABI': 'base', 'MACH': u'1', 'MOZ_LANGPACK_ID': 'thunderbird.mozilla.org', 'OLDPWD': '/home/tijuca', 'DEB_HOST_ARCH_CPU': 'mipsel', 'DEB_RULES_REQUIRES_ROOT': 'binary-targets', 'GCJFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong', 'PWD': '/home/tijuca/thunderbird-60.5.1', 'DEB_BUILD_GNU_CPU': 'mipsel', 'DEB_TARGET_ARCH_ENDIAN': 'little', 'CFLAGS': '-g -O2 -fdebug-prefix-map=/home/tijuca/thunderbird-60.5.1=. -fstack-protector-strong -Wformat -Werror=format-security -fno-delete-null-pointer-checks -fno-schedule-insns2 -fno-lifetime-dse', 'MAKE_TERMOUT': '/dev/pts/1', 'DEB_TARGET_ARCH_CPU': 'mipsel', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'DEB_TARGET_ARCH': 'mipsel', 'DEB_TARGET_ARCH_ABI': 'base', 'DEB_BUILD_ARCH_LIBC': 'gnu'}
 0:00.98 make[2]: Entering directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird'
 0:00.98 make[2]: warning: -j4 forced in submake: resetting jobserver mode.
 0:01.23 make[3]: Entering directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/comm/mail/installer'
 1:02.21 Duplicates 0 bytes:
 1:02.21   chrome/messenger/skin/classic/messenger/messages/simple/Variants/Normal.css
 1:02.22   chrome/chat/content/chat/conv.html
 1:02.22
 1:02.22 Duplicates 140 bytes:
 1:02.22   chrome/messenger/skin/classic/messenger/messages/simple/Incoming/Context.html
 1:02.22   chrome/messenger/skin/classic/messenger/messages/simple/Incoming/NextContext.html
 1:02.22
 1:02.22 Duplicates 260 bytes:
 1:02.22   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/lightning/imip.css
 1:02.22   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/lightning/imip.css
1:02.23
 1:02.23 Duplicates 273 bytes:
 1:02.23   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/lightning/lightning-toolbar.css
 1:02.23   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/osx/lightning/lightning-toolbar.css
 1:02.23
 1:02.23 Duplicates 290 bytes:
 1:02.23   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/calendar/calendar-invitations-dialog.css
 1:02.23   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/calendar/calendar-invitations-dialog.css
 1:02.23
 1:02.23 Duplicates 359 bytes (2 times):
 1:02.24   chrome/en-US/locale/en-US/global-platform/mac/intl.properties
 1:02.24   chrome/en-US/locale/en-US/global-platform/unix/intl.properties
 1:02.24   chrome/en-US/locale/en-US/global-platform/win/intl.properties
 1:02.24
 1:02.24 Duplicates 361 bytes:
 1:02.24   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/lightning/lightning-widgets.css
 1:02.24   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/lightning/lightning-widgets.css
 1:02.24
 1:02.24 Duplicates 389 bytes:
 1:02.24   features/wetransfer@extensions.thunderbird.net/webextension/_locales/en/messages.json
 1:02.25   features/wetransfer@extensions.thunderbird.net/webextension/_locales/en_GB/messages.json
 1:02.25
 1:02.25 Duplicates 439 bytes:
 1:02.25   chrome/messenger/skin/classic/messenger/messages/mail/Incoming/buddy_icon.png
 1:02.25   chrome/messenger/skin/classic/messenger/messages/mail/Outgoing/buddy_icon.png
 1:02.25
 1:02.25 Duplicates 440 bytes:
 1:02.25   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/lightning/accountCentral.css
 1:02.25   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/lightning/accountCentral.css
 1:02.25
 1:02.25 Duplicates 495 bytes:
 1:02.26   chrome/en-US/locale/en-US/global-platform/unix/accessible.properties
 1:02.26   chrome/en-US/locale/en-US/global-platform/win/accessible.properties
 1:02.26
 1:02.26 Duplicates 505 bytes:
 1:02.26   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/calendar/calendar-daypicker.css
 1:02.26   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/calendar/calendar-daypicker.css
 1:02.26
 1:02.26 Duplicates 515 bytes:
 1:02.26   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/attach.svg
 1:02.26   chrome/classic/skin/classic/messenger/icons/attach.svg
 1:02.27
 1:02.27 Duplicates 542 bytes (2 times):
 1:02.27   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/edit.svg
 1:02.27   chrome/classic/skin/classic/messenger/icons/newmsg.svg
 1:02.27   chrome/classic/skin/classic/messenger/shared/in-content/compose.svg
 1:02.27
 1:02.27 Duplicates 569 bytes:
 1:02.27   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/save.svg
 1:02.27   chrome/classic/skin/classic/messenger/icons/save.svg
 1:02.28
 1:02.28 Duplicates 580 bytes:
 1:02.28   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/calendar/calendar-alarm-dialog.css
 1:02.28   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/calendar/calendar-alarm-dialog.css
 1:02.29
 1:02.29 Duplicates 614 bytes:
 1:02.29   chrome/messenger/skin/classic/messenger/messages/bubbles/Bitmaps/plus.png
 1:02.29   chrome/messenger/skin/classic/messenger/messages/mail/Bitmaps/plus.png
 1:02.29
 1:02.29 Duplicates 615 bytes:
 1:02.29   chrome/messenger/skin/classic/messenger/messages/bubbles/Bitmaps/plus-hover.png
 1:02.29   chrome/messenger/skin/classic/messenger/messages/mail/Bitmaps/plus-hover.png
 1:02.29
 1:02.29 Duplicates 619 bytes:
 1:02.30   chrome/messenger/skin/classic/messenger/messages/bubbles/Bitmaps/minus.png
 1:02.30   chrome/messenger/skin/classic/messenger/messages/mail/Bitmaps/minus.png
 1:02.30
 1:02.30 Duplicates 620 bytes:
 1:02.30   chrome/messenger/skin/classic/messenger/messages/bubbles/Bitmaps/minus-hover.png
 1:02.30   chrome/messenger/skin/classic/messenger/messages/mail/Bitmaps/minus-hover.png
 1:02.30
 1:02.30 Duplicates 638 bytes:
 1:02.30   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/print.svg
 1:02.30   chrome/classic/skin/classic/messenger/icons/print.svg
 1:02.30
 1:02.31 Duplicates 656 bytes:
 1:02.31   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/category.svg
 1:02.31   chrome/classic/skin/classic/messenger/icons/tag.svg
 1:02.31
 1:02.31 Duplicates 696 bytes:
 1:02.31   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/address.svg
 1:02.31   chrome/classic/skin/classic/messenger/icons/address.svg
 1:02.31
 1:02.31 Duplicates 714 bytes:
 1:02.31   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/calendar-tab.svg
 1:02.32   chrome/classic/skin/classic/messenger/shared/in-content/calendar.svg
 1:02.32
 1:02.32 Duplicates 728 bytes:
 1:02.32   chrome/en-US/locale/en-US/global/viewSource.properties
 1:02.32   chrome/en-US/locale/en-US/messenger/viewSource.properties
 1:02.32
 1:02.32 Duplicates 741 bytes:
 1:02.32   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/security.svg
 1:02.32   chrome/classic/skin/classic/messenger/icons/security.svg
 1:02.32
 1:02.32 Duplicates 766 bytes:
 1:02.33   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/common/icons/delete.svg
 1:02.33   chrome/classic/skin/classic/messenger/icons/delete.svg
 1:02.33
 1:02.33 Duplicates 835 bytes:
 1:02.33   res/table-remove-column-active.gif
 1:02.33   res/table-remove-row-active.gif
 1:02.33
 1:02.33 Duplicates 841 bytes:
 1:02.33   res/table-remove-column-hover.gif
 1:02.33   res/table-remove-row-active.gif
 1:02.33
 1:02.33 Duplicates 841 bytes:
 1:02.33   res/table-remove-column-hover.gif
 1:02.33   res/table-remove-row-hover.gif
 1:02.33
 1:02.34 Duplicates 841 bytes:
 1:02.34   res/table-remove-column.gif
 1:02.34   res/table-remove-row.gif
 1:02.34
 1:02.34 Duplicates 849 bytes:
 1:02.34   chrome/classic/skin/classic/messenger/icons/accounts.svg
 1:02.34   chrome/classic/skin/classic/messenger/shared/in-content/account.svg
 1:02.34
 1:02.34 Duplicates 980 bytes:
 1:02.34   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/linux/calendar/calendar-management.css
 1:02.35   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/skin/windows/calendar/calendar-management.css
 1:02.35
 1:02.35 Duplicates 1640 bytes:
 1:02.35   chrome/messenger/content/messenger/cloudfile/Box/management.js
 1:02.35   chrome/messenger/content/messenger/cloudfile/Hightail/management.js
 1:02.35
 1:02.35 Duplicates 3433 bytes (2 times):
 1:02.35   chrome/icons/default/default48.png
 1:02.35   chrome/messenger/content/branding/icon48.png
 1:02.35   chrome/classic/skin/classic/messenger/icons/new-mail-alert.png
 1:02.35
 1:02.36 Duplicates 4858 bytes:
 1:02.36   chrome/toolkit/skin/classic/mozapps/downloads/downloadButtons.png
 1:02.36   chrome/toolkit/skin/classic/mozapps/update/downloadButtons.png
 1:02.36
 1:02.36 Duplicates 4914 bytes:
 1:02.36   chrome/icons/default/default64.png
 1:02.36   chrome/messenger/content/branding/icon64.png
 1:02.36
 1:02.36 Duplicates 5618 bytes:
 1:02.36   chrome/devtools/modules/devtools/client/themes/toolbars.css
 1:02.37   chrome/devtools/skin/toolbars.css
 1:02.37
 1:02.37 Duplicates 37477 bytes:
 1:02.37   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/calendar-js/calFilter.js
 1:02.37   distribution/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/chrome/calendar/content/calendar/calFilter.js
 1:02.37
 1:02.37 Duplicates 2422700 bytes:
 1:02.37   thunderbird
 1:02.37   thunderbird-bin
 1:02.37
 1:02.37 WARNING: Found 39 duplicated files taking 2502884 bytes (uncompressed)
 1:04.76 Wrote 472 files in 2323363 bytes to mozharness.zip in 1.91s
 1:04.80 Packing stylo binding files...
 1:04.94   adding: structs.rs (deflated 88%)
 1:05.02   adding: atom_macro.rs (deflated 93%)
 1:05.03   adding: pseudo_element_definition.rs (deflated 90%)
 1:05.04   adding: bindings.rs (deflated 87%)
 1:16.45 make[3]: Leaving directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/comm/mail/installer'
 1:16.45 make[2]: Leaving directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird'
 1:16.46 Notification center failed: Install notify-send (usually part of the libnotify package) to get a notification when the build finishes.

Unfortunately the build is failing later probably because the linker has still memory problems that happens earlier to the above output. But from my POV the patch is working. I already did a test build yesterday on amd64 to see if the build here is still successful and it was.

365:09.46 /home/tijuca/thunderbird-60.5.1/obj-thunderbird/dist/include/nsISupportsImpl.h:988:44: note: in expansion of macro 'NS_IMPL_QUERY_BODY'
365:09.47  #define NS_INTERFACE_MAP_ENTRY(_interface) NS_IMPL_QUERY_BODY(_interface)
365:09.47                                             ^~~~~~~~~~~~~~~~~~
365:09.47 /home/tijuca/thunderbird-60.5.1/dom/canvas/OffscreenCanvas.cpp:311:3: note: in expansion of macro 'NS_INTERFACE_MAP_ENTRY'
365:09.47    NS_INTERFACE_MAP_ENTRY(nsISupports)
365:09.47    ^~~~~~~~~~~~~~~~~~~~~~
368:01.01 libdom_canvas.a.desc
368:01.37 make[6]: Leaving directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/dom/canvas'
368:01.41 make[6]: Entering directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/toolkit/library'
368:01.41 symverscript
368:02.75 libxul_s.a.desc
368:02.75 libxul.so
375:42.76 /usr/bin/ld: error: libxul.so(.debug_info) is too large (0x36d2ef07 bytes)   <---
375:42.77 /usr/bin/ld: error: libxul.so(.debug_str) is too large (0x898e9de bytes)     <---
375:42.77 /usr/bin/ld: error: libxul.so(.debug_loc) is too large (0xf7a8051 bytes)     <---
375:42.77 /usr/bin/ld: total time in link: 447.720000
375:48.49 make[6]: Leaving directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/toolkit/library'
375:48.50 make[6]: Entering directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/ipc/app'
375:48.51 make[6]: Entering directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/js/xpconnect/shell'
375:52.73 plugin-container
375:54.52 /usr/bin/ld: total time in link: 1.312000
375:55.01 make[6]: Leaving directory '/home/tijuca/thunderbird-60.5.1/obj-thunderbird/ipc/app'
375:55.85 xpcshell
375:57.64 /usr/bin/ld: total time in link: 1.380000
... [snip]
if [ "unstable" != "UNRELEASED" ]; then \
        dh_strip --automatic-dbgsym ;\
else \  
        dh_strip --no-automatic-dbgsym ;\
fi
objcopy: debian/thunderbird/usr/lib/thunderbird/libxul.so: unable to initialize compress status for section .debug_info
objcopy: debian/thunderbird/usr/lib/thunderbird/libxul.so: unknown type [0x7000001e] section `.debug_info'
objcopy:debian/thunderbird/usr/lib/thunderbird/libxul.so: memory exhausted
dh_strip: objcopy --only-keep-debug --compress-debug-sections debian/thunderbird/usr/lib/thunderbird/libxul.so debian/.debhelper/thunderbird/dbgsym-root/usr/lib/debug/.build-id/9b/ea6cfe72b2fa10b00499eca53470c08952a51a.debug returned exit code 1
dh_strip: Aborting due to earlier error
Attached file md5_chunk.py —

Before submitting this patch to Mozilla, I wanted to see what the impact might be on build times since find_dupes gets executed many times during a build.

The test is just comparing the chunked vs whole file method of calculating the checksum, each run 100 times on the same 350MB iso file.

I kind of thought these results would be the other way around, so I ran it three times to be sure. Then did a fourth run with the order of the tests reversed in case there was some caching going on that I wasn't accounting for.

Based on this, reading in chunks looks like a win? I'm thinking because memory only needs to be allocated once when chunking a large file, but I really don't know.

$ python md5_chunk.py
Testing whole file
57.6181268692
Testing chunked file
53.3301968575

$ python md5_chunk.py
Testing whole file
58.3839740753
Testing chunked file
53.4132239819

$ python md5_chunk.py
Testing whole file
58.4137079716
Testing chunked file
53.4710948467

# Reversed test order on this run
$ python md5_chunk.py
Testing chunked file
52.8899769783
Testing whole file
58.2163009644

I added this patch to the source within the ESR versions for the build of TB in Debian unstable and and also (old-)stable since several months. So I've discovered no build issues on any platform related to this modification. The builds on mips(el) are still failing later while linking libxul.so due running out of memory as visible in the previous post above.

That's good to know (about the md5sum patch).

As far as your other error, I really don't know. Firefox-esr seems to be building okay on mips since 60.6 or so. I looked real quick at the list of patches you have on Thunderbird vs Firefox, and didn't see any differences as far as the MIPS based ones go. (I didn't look too closely, because your overall patch list is much longer).

Yes, I wasn't able to fully figure out what tricks Mike is doing, must be at least something to decrease the symbols size on these platforms. I've tried some options but doesn't succeeded until now. And Mike is much more in the source than me as here probably looking every day on it. ;)
I think currently it doesn't make much sense to try to solve this for 60.x given 68.x is already in the door. But I wanted to give at least the feedback that this approach of reading chunks is working so far.

This fixes a problem when building on a machine with lower than suggested
available RAM. When calculating the hash of large files (libxul.so in this case)
the machine ran out of RAM and the build failed. Reading in chunks, you get the
same hash value at the end, but use less RAM.

Wait a second. Why are you running find-dupes? It only runs when MOZ_AUTOMATION is set and you should not really set that downstream.

Also note that libxul.so is only large at the point find-dupes runs because you disable strip.

(In reply to Mike Hommey [:glandium] from comment #12)

Also note that libxul.so is only large at the point find-dupes runs because you disable strip.

And FWIW, I solved this in Debian by passing -Wl,--compress-debug-sections=zlib in LDFLAGS.

Pushed by thunderbird@calypsoblue.org: https://hg.mozilla.org/integration/autoland/rev/9b45b5d3aa3c Read files in chunks when calculating md5 in find_dupes. r=glandium

https://hg.mozilla.org/mozilla-central/rev/9b45b5d3aa3c

Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 70.0
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: