1492121 - Startup profiling should start before loading libxul

Status: RESOLVED FIXED

(Core :: Gecko Profiler, enhancement, P2)

Description

[Florian Quèze [:florian]]

According to bug 1437245, the gecko profiler platform code doesn't depend on libxul, so it should be possible to load it before libxul when the startup profiling environment variables are set.


Here is an example case where this would be very valuable: I profiled a cold startup of a current Nightly on the 2018 quantum reference hardware.

https://perfht.ml/2MIZnr5

The 'main' marker here is shown as being 138ms after the begining of the profile.


The data visible at about:telemetry#simple-measurements-tab for the same browser session shows a different picture:

"start" is after 801ms (https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/browser/app/nsBrowserApp.cpp#264)

"main" is after 17951ms (https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/toolkit/xre/nsAppRunner.cpp#3215)

"firstPaint" is after 26041ms


So out of the 26s it takes to show the browser window, we miss information about the first 17.8s.

Comment 1

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Hi Jim, I asked about removing profiler code from libxul to Markus and he told me that you talked about this issue before and you could have some pointers for me. It could be great if you can tell some more about the possible solutions or approaches you have.
(I asked you on Slack but it was probably better to ask you here.)

Comment 2

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Sorry, this needinfo request stayed here for a while. We talked with Jim and he suggested that we should move the profiler to `mozglue` as a standalone library and call the `profiler_init`[1] as soon as possible(Currently it’s inside of `XRE_XPCShellMain`[2]). But there are some libxul dependent parts in profiler, they should still stay there and once the libxul is initialized, we should let profiler know that libxul is now initialized and can do some additional things that normally can’t without the libxul.

There were two paths in my mind that we can select while doing this. But I wasn’t sure which one to select so I asked this. That was, either removing only the `profiler_init` and friends from the libxul and leaving the other parts inside libxul or removing everything that we could from libxul and making the profiler a standalone library. We could handle the libxul only parts by making some abstraction for the things that we need to implement on the libxul side. Jim suggested that if we are gonna do this, we should go with the second path, which is removing everything we could and making it a standalone library. 
The reason doing it a standalone library is to make it more easy to figure out what the library depends on and which level. So people who look at the profiler header, should understand what  they are doing precisely.
For example marker payloads[3] and serialization code[4] is libxul dependent right now. We should create some abstract classes for them in the profiler library and implement them inside libxul. 

He also suggested that, after moving out the profiler, we can test it with implementing profiler for the JS shell[5] which is pretty straightforward and doesn’t depend on many things. That way we can really see if the profiler is completely independent or not.

Currently, I’m doing some experiments to see how it’s gonna be but I don’t have so many updates about it yet.

[1]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/core/platform.cpp#2672
[2]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/js/xpconnect/src/XPCShellImpl.cpp#1135
[3]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/public/ProfilerMarkerPayload.h
[4]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/core/platform.cpp#1962
[5]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/js/src/shell/js.cpp#11021

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thank you Nazim for starting on this and discussing options.

I'm now working on it. Still early days, doing preparatory work locally: Fixing #includes, moving away from XUL data structs, etc.; bugs to be filed...
I'll probably head towards Jim's suggested 2nd path: A standalone minimal profiler on which we can add more functionality through abstract interfaces and/or code injection. (Keeping in mind other wanted optimizations and features in case they can be better handled in a separate library, e.g.: bug 1518949.)

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Configuration option --enable-base-profiler, false by default for now. - r?njn

MozProfiler will not be built by default (yet).
Add 'ac_add_options --enable-mozprofiler' in mozconfig to enable it on Linux,
Mac, and Windows.

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Copy most of Gecko Profiler code to mozglue/baseprofiler - r?njn

Almost-straight copy of a subset of files from tools/profiler to
mozglue/mozprofiler.
Some minor changes first:

• MOZ_GECKO_PROFILER -> MOZ_MOZPROFILER; so thanks to the previous patch we
  won't even try to build this not-yet-buildable module.
• Reduced moz.build to only mention actually-copied files.
• Headers in 'public' prefixed with "Moz" (to distinguish them from their
  originals, in case they later get #included from the same units).
• Also copied profiling categories from js/src/vm/GeckoProfiler.cpp to
  ProfilingCategory.cpp, and copied js/src/vm/ProfilingStack.cpp, and their
  respective headers -- as they are needed for a significant part of
  the profiler API, and are not strictly js-specific.

Depends on D31922

Comment 6

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - All necessary changes to make baseprofiler build - r?njn

Almost-mechanical changes include:

• Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
  related).
• Use std::string for strings and nsIDs.
• Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
  mfbt/RefCounted.h because of bug 1536656.
• Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
• Only record the main thread by default.

This now builds (with --enable-mozprofiler), but is not usable yet.

Depends on D31923

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Make memory allocations compatible between mozglue and non-mozglue calls - r?njn

By default, mozglue code will use OS-provided allocators, but this can lead to
issues if/when objects are allocated one way but destroyed another way (e.g.,
because they were transferred to/from outside code.)

To fix this, a new header core/CppMemory.h must be #included by every
compilation unit, so that all local built objects will use Mozilla allocators.

MozProfiler should now be usable; tests and more improvements to follow.

Depends on D31924

Comment 8

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - TestBaseProfiler - r?njn

Simple test program that exercises the most important APIs of MozProfiler.
(Including checking that macros work even when MozProfiler is not enabled.)

Depends on D31925

Comment 9

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Enable Base Profiler by default on Linux and Mac - r?njn

Depends on D31926

Comment 10

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r?njn

Notice the extra 'MOZ' in the env-var names.
This is to control MozProfiler separately from the Gecko Profiler.

Depends on D31927

Comment 11

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Added "BASE" to all public macros - r?njn

E.g., AUTO_PROFILER_INIT -> AUTO_MOZPROFILER_INIT.
This will allow #including MozProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.

Depends on D31928

Comment 12

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Enclosing BaseProfiler in namespace mozilla::baseprofiler - r?njn

This prevents potential name clashes between the two profilers.

Depends on D31929

Comment 13

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r?njn

Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.

Depends on D31930

Comment 14

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Gecko Profiler integrates BaseProfiler startup profiled threads - r?njn

If MOZ_MOZPROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads, which will appear as a new track under the
main process.

Only adding threads from MozProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.

Depends on D31931

Comment 15

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Use BaseProfiler in nsBrowserApp.cpp's main() - r?njn

Start using MozProfiler in Firefox main(), before&after XPCOM runs.

Also added a MozProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).

Main process name changed to "MozGeckoMain", so as not to confuse the
front-end.

Depends on D31932

Comment 16

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Use BaseProfiler in XRE_Init{Parent,Child}Process() - r?njn

Depends on D31933

Comment 17

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn

Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on MOZ_BASE_PROFILER, which is currently
not #defined by default (in public/BaseProfiler.h).

Depends on D31923

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5e7817b385fc
Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/0689c1b8f4f7
MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/de1c3ae8df14
All necessary changes to make baseprofiler build - r=njn
https://hg.mozilla.org/integration/autoland/rev/5f30af0d0f63
TestBaseProfiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/3800ef0a0e37
Enable Base Profiler by default on Linux and Mac - r=njn
https://hg.mozilla.org/integration/autoland/rev/ac8da816859d
Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn
https://hg.mozilla.org/integration/autoland/rev/0cddbdfdfae2
Added "BASE" to all public macros - r=njn
https://hg.mozilla.org/integration/autoland/rev/0c6cf7bc131f
Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/bdf3b865c947
Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/da72675c0d37
Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn
https://hg.mozilla.org/integration/autoland/rev/1bf52d547eb7
Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn
https://hg.mozilla.org/integration/autoland/rev/9d768006784a
Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn

Comment 19

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out 12 changesets (Bug 1492121) for platform.cpp and TestBaseProfiler.cpp related bustages
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=26c117fc572d8cace9694a3b08979b5963354234&tochange=4494c0e5d35d3b297842c45d11dcc8f02c210944&selectedJob=249882229

Backout link: https://hg.mozilla.org/integration/autoland/rev/4494c0e5d35d3b297842c45d11dcc8f02c210944

Failures logs:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882229&repo=autoland&lineNumber=6092

[task 2019-06-04T08:51:41.483Z] 08:51:41 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler'
[task 2019-06-04T08:51:41.486Z] 08:51:41 INFO - /builds/worker/workspace/build/src/sccache/sccache /builds/worker/workspace/build/src/gcc/bin/g++ -o Unified_cpp_mozglue_baseprofiler0.o -c -I/builds/worker/workspace/build/src/obj-firefox/dist/stl_wrappers -I/builds/worker/workspace/build/src/obj-firefox/dist/system_wrappers -include /builds/worker/workspace/build/src/config/gcc_hidden.h -DDEBUG=1 -DIMPL_MFBT -DMOZ_VTUNE_INSTRUMENTATION -DMOZ_HAS_MOZGLUE -DIMPL_MFBT -I/builds/worker/workspace/build/src/mozglue/baseprofiler -I/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler -I/builds/worker/workspace/build/src/mozglue/baseprofiler/core -I/builds/worker/workspace/build/src/mozglue/linker -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wduplicated-cond -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object -Wformat -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -freorder-blocks -Os -fno-omit-frame-pointer -funwind-tables -Werror -Wno-error=shadow -Wno-ignored-qualifiers -MD -MP -MF .deps/Unified_cpp_mozglue_baseprofiler0.o.pp /builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler/Unified_cpp_mozglue_baseprofiler0.cpp
[task 2019-06-04T08:51:41.486Z] 08:51:41 INFO - In file included from /builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler/Unified_cpp_mozglue_baseprofiler0.cpp:101:0:
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/mozglue/baseprofiler/core/platform.cpp: In function 'void mozilla::baseprofiler::profiler_ensure_started(uint32_t, double, uint32_t, const char**, uint32_t, const mozilla::Maybe<double>&)':
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - /builds/worker/workspace/build/src/mozglue/baseprofiler/core/platform.cpp:2682:8: error: variable 'startedProfiler' set but not used [-Werror=unused-but-set-variable]
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - bool startedProfiler = false;
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - ^~~~~~~~~~~~~~~
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - cc1plus: all warnings being treated as errors
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/config/rules.mk:810: recipe for target 'Unified_cpp_mozglue_baseprofiler0.o' failed
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - make[4]: *** [Unified_cpp_mozglue_baseprofiler0.o] Error 1
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler'
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'mozglue/baseprofiler/target' failed
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - make[3]: *** [mozglue/baseprofiler/target] Error 2
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - make[3]: *** Waiting for unfinished jobs....
[task 2019-06-04T08:51:41.488Z] 08:51:41 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/media/libjpeg'
[task 2019-06-04T08:51:41.491Z] 08:51:41 INFO - media/libjpeg/jcapimin.o

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882141&repo=autoland&lineNumber=36379

[task 2019-06-04T08:51:48.500Z] 08:51:48 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests/interceptor'
[task 2019-06-04T08:51:48.500Z] 08:51:48 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests'
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - /builds/worker/workspace/build/src/sccache/sccache /builds/worker/workspace/build/src/clang/bin/x86_64-w64-mingw32-clang++ -mwindows -o TestBaseProfiler.o -c -DDEBUG=1 -DMOZ_HAS_MOZGLUE -I/builds/worker/workspace/build/src/mozglue/tests -I/builds/worker/workspace/build/src/obj-firefox/mozglue/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/testing -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++1z-compat -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-unknown-pragmas -Wno-unused-function -Wno-conversion-null -Wno-switch -Wno-enum-compare -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fms-extensions -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -mms-bitfields -Wno-incompatible-ms-struct -fno-rtti -ffunction-sections -fdata-sections -Wa,-mbig-obj -fno-exceptions -fno-math-errno -pipe -g -gcodeview -O1 -fno-omit-frame-pointer -funwind-tables -MD -MP -MF .deps/TestBaseProfiler.o.pp /builds/worker/workspace/build/src/mozglue/tests/TestBaseProfiler.cpp
[task 2019-06-04T08:51:48.501Z] 08:51:48 ERROR - /builds/worker/workspace/build/src/mozglue/tests/TestBaseProfiler.cpp:20:4: error:
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - # error
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - ^
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - 1 error generated.
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - /builds/worker/workspace/build/src/config/rules.mk:810: recipe for target 'TestBaseProfiler.o' failed
[task 2019-06-04T08:51:48.501Z] 08:51:48 ERROR - make[4]: *** [TestBaseProfiler.o] Error 1
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests'
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - make[4]: *** Waiting for unfinished jobs....

Comment 20

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Looks like I didn't pick enough build jobs in my Try... Thanks Bodgan, will follow up.

Comment 21

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Another failure in 'pkg':
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882250&repo=autoland&lineNumber=87673

[task 2019-06-04T08:51:52.994Z] The error occurred while processing the following file:
[task 2019-06-04T08:51:52.994Z]
[task 2019-06-04T08:51:52.995Z] /builds/worker/workspace/sm-package/mozjs-69.0a1.0/mozglue/moz.build
[task 2019-06-04T08:51:52.995Z]
[task 2019-06-04T08:51:52.995Z] The underlying problem is we referenced a path that does not exist. That path is:
[task 2019-06-04T08:51:52.995Z]
[task 2019-06-04T08:51:52.995Z] /builds/worker/workspace/sm-package/mozjs-69.0a1.0/mozglue/baseprofiler/moz.build

So I'll need to add mozglue/baseprofiler in there. (Can't see a way not to, as mozglue/moz.build now unconditionally refers to it.)

Comment 22

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Copy mozglue/baseprofiler in make-source-package.sh - r?sfink

Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build will refer to it unconditionally.
Note that if MOZ_GECKO_PROFILER and MOZ_BASE_PROFILER are not defined, no
actual code will be generated.

Depends on D31923

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/72c4026e9455
Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/db1506f94d0d
Copy mozglue/baseprofiler in make-source-package.sh - r=sfink
https://hg.mozilla.org/integration/autoland/rev/1f10b50f758f
MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/77e7b13c6237
All necessary changes to make baseprofiler build - r=njn
https://hg.mozilla.org/integration/autoland/rev/bf1731627e07
TestBaseProfiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/39c486afacec
Enable Base Profiler by default on Linux and Mac - r=njn
https://hg.mozilla.org/integration/autoland/rev/a3fe26927b31
Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn
https://hg.mozilla.org/integration/autoland/rev/46f57504c087
Added "BASE" to all public macros - r=njn
https://hg.mozilla.org/integration/autoland/rev/b54b813c4c6c
Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/4215fefb6ef3
Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/2ffb6ccca437
Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn
https://hg.mozilla.org/integration/autoland/rev/90aeaad4a4de
Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn
https://hg.mozilla.org/integration/autoland/rev/e707f1890820
Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn

Comment 24

[Noemi Erli[:noemi_erli]]

Backed out 13 changesets (Bug 1492121) for valgrind bustage

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=e707f1890820c707eb43e7195f26e3ef2fb7571b&searchStr=valgrind&selectedJob=250270901

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=250270901&repo=autoland&lineNumber=40901

Backout: https://hg.mozilla.org/integration/autoland/rev/d57670df353e6d6ca69d88f7415a011c2db43b5d

Comment 25

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Thanks Noemi. I thought I had done an "everything" try from ./mach try chooser but it seems it doesn't actually do everything 😒

Comment 26

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1492121 - Suppress dlopen leak triggered by next patch - r?njn

Valgrind report:
TEST-UNEXPECTED-FAIL | valgrind-test | 192 bytes in 3 blocks are definitely lost at malloc / dl_open_worker / _dl_catch_error / _dl_open
==2561== 192 bytes in 3 blocks are definitely lost in loss record 348 of 399
==2561== at 0x4C2B240: malloc+112 (vg_replace_malloc.c:298)
==2561== by 0x4012919: dl_open_worker+1977 (dl-open.c:457)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x4011CC9: _dl_open+185 (dl-open.c:633)
==2561== by 0x5051F65: dlopen_doit+101 (dlopen.c:67)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x50522EB: _dlerror_run+123 (dlerror.c:164)
==2561== by 0x5051EE0: dlopen@@GLIBC_2.2.5+48 (dlopen.c:88)
==2561== by 0x1148FC: GetLibHandle (xpcom/glue/standalone/nsXPCOMGlue.cpp:86)
==2561== by 0x1148FC: ReadDependentCB (xpcom/glue/standalone/nsXPCOMGlue.cpp:136)
==2561== by 0x1148FC: XPCOMGlueLoad (xpcom/glue/standalone/nsXPCOMGlue.cpp:306)
==2561== by 0x1148FC: mozilla::GetBootstrap(char const*, mozilla::LibLoadingStrategy)+572 (xpcom/glue/standalone/nsXPCOMGlue.cpp:374)
==2561== by 0x114213: InitXPCOMGlue(mozilla::LibLoadingStrategy)+131 (browser/app/nsBrowserApp.cpp:223)
==2561== by 0x113E6B: main+219 (browser/app/nsBrowserApp.cpp:284)

Nothing due directly to this patch, so it is likely a dlopen issue.

Depends on D31932

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/64fe00416593
Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/1a63bc8f0afd
Copy mozglue/baseprofiler in make-source-package.sh - r=sfink
https://hg.mozilla.org/integration/autoland/rev/838a84ef85f8
MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/d0be75b5f773
All necessary changes to make baseprofiler build - r=njn
https://hg.mozilla.org/integration/autoland/rev/3a7f3ab482b1
TestBaseProfiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/4d72e64503c8
Enable Base Profiler by default on Linux and Mac - r=njn
https://hg.mozilla.org/integration/autoland/rev/842723642da6
Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn
https://hg.mozilla.org/integration/autoland/rev/b4c77fd9b97d
Added "BASE" to all public macros - r=njn
https://hg.mozilla.org/integration/autoland/rev/a4c102433761
Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/a679e252817a
Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn
https://hg.mozilla.org/integration/autoland/rev/ebe23063def2
Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn
https://hg.mozilla.org/integration/autoland/rev/0cbffc5bb535
Suppress dlopen leak triggered by next patch - r=njn
https://hg.mozilla.org/integration/autoland/rev/ebfca0243df2
Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn
https://hg.mozilla.org/integration/autoland/rev/0b97aaf0efce
Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn

Comment 28

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/64fe00416593
https://hg.mozilla.org/mozilla-central/rev/1a63bc8f0afd
https://hg.mozilla.org/mozilla-central/rev/838a84ef85f8
https://hg.mozilla.org/mozilla-central/rev/d0be75b5f773
https://hg.mozilla.org/mozilla-central/rev/3a7f3ab482b1
https://hg.mozilla.org/mozilla-central/rev/4d72e64503c8
https://hg.mozilla.org/mozilla-central/rev/842723642da6
https://hg.mozilla.org/mozilla-central/rev/b4c77fd9b97d
https://hg.mozilla.org/mozilla-central/rev/a4c102433761
https://hg.mozilla.org/mozilla-central/rev/a679e252817a
https://hg.mozilla.org/mozilla-central/rev/ebe23063def2
https://hg.mozilla.org/mozilla-central/rev/0cbffc5bb535
https://hg.mozilla.org/mozilla-central/rev/ebfca0243df2
https://hg.mozilla.org/mozilla-central/rev/0b97aaf0efce