1836369 - Fine grain control over symbol visibility

Status: RESOLVED FIXED

(Toolkit :: General, enhancement)

Description

[[:sergesanspaille]]

libxul exports hundreds of symbols. There may be room for optimization there: less symbols gives more room to the linker gc to remove unused code.

At worse the symbol is just used internally and we reduce the number of dynamic symbols.
At best some symbol are actually unused and we shave some bytes.

This only requires crafting a beautiful linker script, and there's already one for libxul.

This linker feature requires -ffunction-sections and --gc-sections that we're already using.

Comment 1

[[:sergesanspaille]]

Attachment: Bug 1836369 - Only export XRE_GetBootstrap symbol from libxul r=glandium

Better control of the ABI has its own merit, but in that case, we
significantly shrink the amount of dynamic symbol, which in turns
reduces the size of the binary.

On linux64 opt build, this saves ~2.5MB on the installer size libxul.so

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2766c50b0ce5
Only export XRE_GetBootstrap symbol from libxul r=glandium

Comment 3

[Norisz Fay [:noriszfay]]

Backed out for causing android build bustages

Backout link

Push with failures

Failure log

Comment 4

[[:sergesanspaille]]

No big deal - just a relative path, I'll fix this.

Comment 5

[[:sergesanspaille]]

Turns out it was also filtering symbol for the libxul version used for testing. I disabled that.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5e5ece1f117a
Only export XRE_GetBootstrap symbol from libxul r=glandium

Comment 7

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/5e5ece1f117a

Comment 8

[:gerard-majax]

Unfortunately, it looks like this badly broke Fenix: https://bugzilla.mozilla.org/show_bug.cgi?id=1837869#c8

Comment 9

[Cristian Tuns]

Backed out for causing Bug 1837869
Backout link: https://hg.mozilla.org/mozilla-central/rev/176bd422292eca8f35fdb09e12e34e731c4d42db

Comment 10

[[:sergesanspaille]]

I've restricted the patch to non-Android platform. We'll follow-up with the android team on a special version of the symbol file for Android.

Comment 11

[Jan-Erik Rediger [:janerik]]

Please do ping me if there are changes around this for Android. I'm on the Glean team, Glean is the one that needs a bunch of extra symbols exported on Android.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fe77912afbd3
Only export XRE_GetBootstrap symbol from libxul r=glandium

Comment 13

[Iulian Moraru]

Backed out for causing valgrind bustages on moz-automation.mk.

• Push with failures

• Failure log

• Backout link

[task 2023-06-15T07:45:12.766Z] 07:45:12     INFO -  package-generated-sources> gmake[2]: Entering directory '/builds/worker/workspace/obj-build'
[task 2023-06-15T07:45:12.766Z] 07:45:12     INFO -  package-generated-sources> /builds/worker/workspace/obj-build/_virtualenvs/build/bin/python -m mozbuild.action.package_generated_sources 'dist/target.generated-files.tar.gz'
[task 2023-06-15T07:45:12.766Z] 07:45:12    ERROR -  package-generated-sources> Traceback (most recent call last):
[task 2023-06-15T07:45:12.766Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/fetches/python/lib/python3.8/runpy.py", line 194, in _run_module_as_main
[task 2023-06-15T07:45:12.766Z] 07:45:12     INFO -  package-generated-sources>     return _run_code(code, main_globals, None,
[task 2023-06-15T07:45:12.767Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/fetches/python/lib/python3.8/runpy.py", line 87, in _run_code
[task 2023-06-15T07:45:12.767Z] 07:45:12     INFO -  package-generated-sources>     exec(code, run_globals)
[task 2023-06-15T07:45:12.767Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/action/package_generated_sources.py", line 42, in <module>
[task 2023-06-15T07:45:12.767Z] 07:45:12     INFO -  package-generated-sources>     sys.exit(main(sys.argv[1:]))
[task 2023-06-15T07:45:12.767Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/action/package_generated_sources.py", line 38, in main
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>     create_tar_gz_from_files(fh, files, compresslevel=5)
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/checkouts/gecko/python/mozbuild/mozpack/archive.py", line 118, in create_tar_gz_from_files
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>     create_tar_from_files(gf, files)
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/checkouts/gecko/python/mozbuild/mozpack/archive.py", line 71, in create_tar_from_files
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>     ti.mode = f.mode or 0o0644
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>   File "/builds/worker/checkouts/gecko/python/mozbuild/mozpack/files.py", line 278, in mode
[task 2023-06-15T07:45:12.768Z] 07:45:12     INFO -  package-generated-sources>     mode = os.stat(self.path).st_mode
[task 2023-06-15T07:45:12.769Z] 07:45:12     INFO -  package-generated-sources> FileNotFoundError: [Errno 2] No such file or directory: '/builds/worker/workspace/obj-build/toolkit/library/gtest/libxul.so.symbols'
[task 2023-06-15T07:45:12.769Z] 07:45:12    ERROR -  package-generated-sources> gmake[2]: *** [Makefile:215: package-generated-sources] Error 1
[task 2023-06-15T07:45:12.769Z] 07:45:12     INFO -  package-generated-sources> gmake[2]: Leaving directory '/builds/worker/workspace/obj-build'
[task 2023-06-15T07:45:12.770Z] 07:45:12    ERROR -  gmake[1]: *** [/builds/worker/checkouts/gecko/build/moz-automation.mk:110: automation/package-generated-sources] Error 2
[task 2023-06-15T07:45:34.815Z] 07:45:34     INFO -  buildsymbols> gmake[2]: Entering directory '/builds/worker/workspace/obj-build'

Comment 14

[Alex Finder|PTO, returning April 28th]

(In reply to Cristian Tuns from comment #9)

Backed out for causing Bug 1837869
Backout link: https://hg.mozilla.org/mozilla-central/rev/176bd422292eca8f35fdb09e12e34e731c4d42db

== Change summary for alert #38665 (as of Mon, 12 Jun 2023 11:31:31 GMT) ==

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
5%	installer size	osx-aarch64-shippable	aarch64 nightly	80,996,052.33 -> 84,919,571.08
5%	installer size	osx-nightlyasrelease	nightly nightly-as-release	83,260,703.17 -> 87,006,940.00
4%	installer size	osx-shippable	nightly	85,016,244.33 -> 88,775,568.67
0.30%	installer size	osx-shippable	instrumented	124,622,196.50 -> 124,989,996.75
0.22%	installer size	osx-cross		84,745,436.46 -> 84,932,397.50

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=38665

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2f9f01f9fe93
Only export XRE_GetBootstrap symbol from libxul r=glandium

Comment 16

[Noemi Erli[:noemi_erli]]

Backed out changeset 2f9f01f9fe93 (Bug 1836369) for causing build bustages

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&selectedTaskRun=MenODrt-ReyLZVO4JXC57Q.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=windows%2C2012%2Cdebug%2Cbuild-win32%2Fdebug%2Cb&revision=2f9f01f9fe93f61a7bc9ae749c6d3599c5bffb87

Log: https://treeherder.mozilla.org/logviewer?job_id=419945674&repo=autoland&lineNumber=4581

Backout: https://hg.mozilla.org/integration/autoland/rev/2f9f01f9fe93f61a7bc9ae749c6d3599c5bffb87

Comment 17

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d63ff7cd70b5
Backed out changeset 2f9f01f9fe93 for causing build bustages

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4b9eda653a64
Only export XRE_GetBootstrap symbol from libxul r=glandium

Comment 19

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/4b9eda653a64

Comment 20

[Andrej (:aglavic)]

== Change summary for alert #38889 (as of Mon, 26 Jun 2023 22:44:37 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
7%	cpstartup content-process-startup	linux1804-64-shippable-qr	e10s fission stylo webrender	82.83 -> 76.67

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=38889