Closed Bug 1492121 Opened 7 years ago Closed 7 years ago

Startup profiling should start before loading libxul

Categories

(Core :: Gecko Profiler, enhancement, P2)

Product:
Core
Component:
Gecko Profiler
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla69
Tracking Flags:
Tracking Status
firefox69 --- fixed

People

(Reporter: florian, Assigned: mozbugz)

References

(Blocks 3 open bugs)

Details

Attachments

(14 files, 2 obsolete files)

Bug 1492121 - Copy most of Gecko Profiler code to mozglue/baseprofiler - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - All necessary changes to make baseprofiler build - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - TestBaseProfiler - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Enable Base Profiler by default on Linux and Mac - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Added "BASE" to all public macros - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Enclosing BaseProfiler in namespace mozilla::baseprofiler - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Gecko Profiler integrates BaseProfiler startup profiled threads - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Use BaseProfiler in nsBrowserApp.cpp's main() - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Use BaseProfiler in XRE_Init{Parent,Child}Process() - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Copy mozglue/baseprofiler in make-source-package.sh - r?sfink
47 bytes, text/x-phabricator-request
Details | Review
Bug 1492121 - Suppress dlopen leak triggered by next patch - r?njn
47 bytes, text/x-phabricator-request
Details | Review
According to bug 1437245, the gecko profiler platform code doesn't depend on libxul, so it should be possible to load it before libxul when the startup profiling environment variables are set. Here is an example case where this would be very valuable: I profiled a cold startup of a current Nightly on the 2018 quantum reference hardware. https://perfht.ml/2MIZnr5 The 'main' marker here is shown as being 138ms after the begining of the profile. The data visible at about:telemetry#simple-measurements-tab for the same browser session shows a different picture: "start" is after 801ms (https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/browser/app/nsBrowserApp.cpp#264) "main" is after 17951ms (https://searchfox.org/mozilla-central/rev/6c82481caa506a240a626bb44a2b8cbe0eedb3a0/toolkit/xre/nsAppRunner.cpp#3215) "firstPaint" is after 26041ms So out of the 26s it takes to show the browser window, we miss information about the first 17.8s.
Assignee: nobody → canaltinova
Status: NEW → ASSIGNED
Hi Jim, I asked about removing profiler code from libxul to Markus and he told me that you talked about this issue before and you could have some pointers for me. It could be great if you can tell some more about the possible solutions or approaches you have. (I asked you on Slack but it was probably better to ask you here.)
Flags: needinfo?(jimb)
Sorry, this needinfo request stayed here for a while. We talked with Jim and he suggested that we should move the profiler to `mozglue` as a standalone library and call the `profiler_init`[1] as soon as possible(Currently it’s inside of `XRE_XPCShellMain`[2]). But there are some libxul dependent parts in profiler, they should still stay there and once the libxul is initialized, we should let profiler know that libxul is now initialized and can do some additional things that normally can’t without the libxul. There were two paths in my mind that we can select while doing this. But I wasn’t sure which one to select so I asked this. That was, either removing only the `profiler_init` and friends from the libxul and leaving the other parts inside libxul or removing everything that we could from libxul and making the profiler a standalone library. We could handle the libxul only parts by making some abstraction for the things that we need to implement on the libxul side. Jim suggested that if we are gonna do this, we should go with the second path, which is removing everything we could and making it a standalone library. The reason doing it a standalone library is to make it more easy to figure out what the library depends on and which level. So people who look at the profiler header, should understand what they are doing precisely. For example marker payloads[3] and serialization code[4] is libxul dependent right now. We should create some abstract classes for them in the profiler library and implement them inside libxul. He also suggested that, after moving out the profiler, we can test it with implementing profiler for the JS shell[5] which is pretty straightforward and doesn’t depend on many things. That way we can really see if the profiler is completely independent or not. Currently, I’m doing some experiments to see how it’s gonna be but I don’t have so many updates about it yet. [1]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/core/platform.cpp#2672 [2]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/js/xpconnect/src/XPCShellImpl.cpp#1135 [3]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/public/ProfilerMarkerPayload.h [4]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/tools/profiler/core/platform.cpp#1962 [5]: https://searchfox.org/mozilla-central/rev/0859e6b10fb901875c80de8f8fc33cbb77b2505e/js/src/shell/js.cpp#11021
Flags: needinfo?(jimb)
Priority: -- → P2

Thank you Nazim for starting on this and discussing options.

I'm now working on it. Still early days, doing preparatory work locally: Fixing #includes, moving away from XUL data structs, etc.; bugs to be filed...
I'll probably head towards Jim's suggested 2nd path: A standalone minimal profiler on which we can add more functionality through abstract interfaces and/or code injection. (Keeping in mind other wanted optimizations and features in case they can be better handled in a separate library, e.g.: bug 1518949.)

Assignee: canaltinova → gsquelart

MozProfiler will not be built by default (yet).
Add 'ac_add_options --enable-mozprofiler' in mozconfig to enable it on Linux,
Mac, and Windows.

Almost-straight copy of a subset of files from tools/profiler to
mozglue/mozprofiler.
Some minor changes first:

  • MOZ_GECKO_PROFILER -> MOZ_MOZPROFILER; so thanks to the previous patch we
    won't even try to build this not-yet-buildable module.
  • Reduced moz.build to only mention actually-copied files.
  • Headers in 'public' prefixed with "Moz" (to distinguish them from their
    originals, in case they later get #included from the same units).
  • Also copied profiling categories from js/src/vm/GeckoProfiler.cpp to
    ProfilingCategory.cpp, and copied js/src/vm/ProfilingStack.cpp, and their
    respective headers -- as they are needed for a significant part of
    the profiler API, and are not strictly js-specific.

Depends on D31922

Almost-mechanical changes include:

  • Removed unneeded/incompatible #includes and functions (any JS- or XPCOM-
    related).
  • Use std::string for strings and nsIDs.
  • Use hand-rolled AddRef&Release's for ref-counted classes -- could not use
    mfbt/RefCounted.h because of bug 1536656.
  • Added some platform-specific polyfills, e.g.: MicrosecondsSince1970().
  • Only record the main thread by default.

This now builds (with --enable-mozprofiler), but is not usable yet.

Depends on D31923

By default, mozglue code will use OS-provided allocators, but this can lead to
issues if/when objects are allocated one way but destroyed another way (e.g.,
because they were transferred to/from outside code.)

To fix this, a new header core/CppMemory.h must be #included by every
compilation unit, so that all local built objects will use Mozilla allocators.

MozProfiler should now be usable; tests and more improvements to follow.

Depends on D31924

Simple test program that exercises the most important APIs of MozProfiler.
(Including checking that macros work even when MozProfiler is not enabled.)

Depends on D31925

Notice the extra 'MOZ' in the env-var names.
This is to control MozProfiler separately from the Gecko Profiler.

Depends on D31927

E.g., AUTO_PROFILER_INIT -> AUTO_MOZPROFILER_INIT.
This will allow #including MozProfiler.h anywhere as needed, without clashing
with Gecko Profiler macros.

Depends on D31928

This prevents potential name clashes between the two profilers.

Depends on D31929

Running identical (but separate) InitializeWin64ProfilerHooks in both profilers
confuses the DLL interceptor and the 2nd one crashes because of unexpected
opcodes introduced by the 1st one.

Depends on D31930

If MOZ_MOZPROFILER_STARTUP and MOZ_PROFILER_STARTUP are set, this will integrate
a pre-XPCOM startup profile into the main profile.
It is stored as separate threads, which will appear as a new track under the
main process.

Only adding threads from MozProfiler means a better integration with Gecko
Profiler profiles, and is more efficient: Less code, and a smaller memory
footprint.

Depends on D31931

Start using MozProfiler in Firefox main(), before&after XPCOM runs.

Also added a MozProfiler label around Gecko Profiler init/shutdown (so that
samples may be ignored if user is only interested in non-XPCOM profiling).

Main process name changed to "MozGeckoMain", so as not to confuse the
front-end.

Depends on D31932

Attachment #9066284 - Attachment description: Bug 1492121 - Use MozProfiler in XRE_InitParentChildProcess() - → Bug 1492121 - Use MozProfiler in XRE_Init{Parent,Child}Process() -
No longer depends on: 1535227
Attachment #9066272 - Attachment description: Bug 1492121 - Configuration option --enable-mozprofiler, false by default for now. - → Bug 1492121 - Configuration option --enable-mozprofiler, false by default for now. - r?njn
Attachment #9066273 - Attachment description: Bug 1492121 - Copy most of Gecko Profiler code to mozglue/mozprofiler - → Bug 1492121 - Copy most of Gecko Profiler code to mozglue/mozprofiler - r?njn
Attachment #9066274 - Attachment description: Bug 1492121 - All necessary changes to make mozprofiler build - → Bug 1492121 - All necessary changes to make mozprofiler build - r?njn
Attachment #9066275 - Attachment description: Bug 1492121 - Make memory allocations compatible between mozglue and non-mozglue calls - → Bug 1492121 - Make memory allocations compatible between mozglue and non-mozglue calls - r?njn
Attachment #9066276 - Attachment description: Bug 1492121 - TestMozProfiler - → Bug 1492121 - TestMozProfiler - r?njn
Attachment #9066277 - Attachment description: Bug 1492121 - --enable-mozprofiler by default on Linux and Mac - → Bug 1492121 - --enable-mozprofiler by default on Linux and Mac - r?njn
Attachment #9066278 - Attachment description: Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_MOZPROFILER_* - → Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_MOZPROFILER_* - r?njn
Attachment #9066279 - Attachment description: Bug 1492121 - Added "MOZ" to all public macros - → Bug 1492121 - Added "MOZ" to all public macros - r?njn
Attachment #9066280 - Attachment description: Bug 1492121 - Enclosing MozProfiler in namespace mozilla::mozprofiler - → Bug 1492121 - Enclosing MozProfiler in namespace mozilla::mozprofiler - r?njn
Attachment #9066281 - Attachment description: Bug 1492121 - Run MozProfiler's InitializeWin64ProfilerHooks once from either profiler - → Bug 1492121 - Run MozProfiler's InitializeWin64ProfilerHooks once from either profiler - r?njn
Attachment #9066282 - Attachment description: Bug 1492121 - Gecko Profiler integrates MozProfiler startup profiled threads - → Bug 1492121 - Gecko Profiler integrates MozProfiler startup profiled threads - r?njn
Attachment #9066283 - Attachment description: Bug 1492121 - Use MozProfiler in nsBrowserApp.cpp's main() - → Bug 1492121 - Use MozProfiler in nsBrowserApp.cpp's main() - r?njn
Attachment #9066284 - Attachment description: Bug 1492121 - Use MozProfiler in XRE_Init{Parent,Child}Process() - → Bug 1492121 - Use MozProfiler in XRE_Init{Parent,Child}Process() - r?njn
Attachment #9066272 - Attachment description: Bug 1492121 - Configuration option --enable-mozprofiler, false by default for now. - r?njn → Bug 1492121 - Configuration option --enable-base-profiler, false by default for now. - r?njn
Attachment #9066273 - Attachment description: Bug 1492121 - Copy most of Gecko Profiler code to mozglue/mozprofiler - r?njn → Bug 1492121 - Copy most of Gecko Profiler code to mozglue/baseprofiler - r?njn
Attachment #9066274 - Attachment description: Bug 1492121 - All necessary changes to make mozprofiler build - r?njn → Bug 1492121 - All necessary changes to make baseprofiler build - r?njn
Attachment #9066276 - Attachment description: Bug 1492121 - TestMozProfiler - r?njn → Bug 1492121 - TestBaseProfiler - r?njn
Attachment #9066277 - Attachment description: Bug 1492121 - --enable-mozprofiler by default on Linux and Mac - r?njn → Bug 1492121 - --enable-base-profiler by default on Linux and Mac - r?njn
Attachment #9066278 - Attachment description: Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_MOZPROFILER_* - r?njn → Bug 1492121 - Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r?njn
Attachment #9066280 - Attachment description: Bug 1492121 - Enclosing MozProfiler in namespace mozilla::mozprofiler - r?njn → Bug 1492121 - Enclosing BaseProfiler in namespace mozilla::baseprofiler - r?njn
Attachment #9066281 - Attachment description: Bug 1492121 - Run MozProfiler's InitializeWin64ProfilerHooks once from either profiler - r?njn → Bug 1492121 - Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r?njn
Attachment #9066282 - Attachment description: Bug 1492121 - Gecko Profiler integrates MozProfiler startup profiled threads - r?njn → Bug 1492121 - Gecko Profiler integrates BaseProfiler startup profiled threads - r?njn
Attachment #9066283 - Attachment description: Bug 1492121 - Use MozProfiler in nsBrowserApp.cpp's main() - r?njn → Bug 1492121 - Use BaseProfiler in nsBrowserApp.cpp's main() - r?njn
Attachment #9066284 - Attachment description: Bug 1492121 - Use MozProfiler in XRE_Init{Parent,Child}Process() - r?njn → Bug 1492121 - Use BaseProfiler in XRE_Init{Parent,Child}Process() - r?njn
Attachment #9066279 - Attachment description: Bug 1492121 - Added "MOZ" to all public macros - r?njn → Bug 1492121 - Added "BASE" to all public macros - r?njn

Added baseprofiler to mozglue/moz.build, so it will be built.
However all cpp files are dependent on MOZ_BASE_PROFILER, which is currently
not #defined by default (in public/BaseProfiler.h).

Depends on D31923

Attachment #9066277 - Attachment description: Bug 1492121 - --enable-base-profiler by default on Linux and Mac - r?njn → Bug 1492121 - Enable Base Profiler by default on Linux and Mac - r?njn
Attachment #9066272 - Attachment is obsolete: true
Attachment #9066275 - Attachment is obsolete: true
Pushed by gsquelart@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5e7817b385fc Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/0689c1b8f4f7 MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/de1c3ae8df14 All necessary changes to make baseprofiler build - r=njn https://hg.mozilla.org/integration/autoland/rev/5f30af0d0f63 TestBaseProfiler - r=njn https://hg.mozilla.org/integration/autoland/rev/3800ef0a0e37 Enable Base Profiler by default on Linux and Mac - r=njn https://hg.mozilla.org/integration/autoland/rev/ac8da816859d Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn https://hg.mozilla.org/integration/autoland/rev/0cddbdfdfae2 Added "BASE" to all public macros - r=njn https://hg.mozilla.org/integration/autoland/rev/0c6cf7bc131f Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/bdf3b865c947 Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/da72675c0d37 Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn https://hg.mozilla.org/integration/autoland/rev/1bf52d547eb7 Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn https://hg.mozilla.org/integration/autoland/rev/9d768006784a Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn

Backed out 12 changesets (Bug 1492121) for platform.cpp and TestBaseProfiler.cpp related bustages
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=26c117fc572d8cace9694a3b08979b5963354234&tochange=4494c0e5d35d3b297842c45d11dcc8f02c210944&selectedJob=249882229

Backout link: https://hg.mozilla.org/integration/autoland/rev/4494c0e5d35d3b297842c45d11dcc8f02c210944

Failures logs:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882229&repo=autoland&lineNumber=6092

[task 2019-06-04T08:51:41.483Z] 08:51:41 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler'
[task 2019-06-04T08:51:41.486Z] 08:51:41 INFO - /builds/worker/workspace/build/src/sccache/sccache /builds/worker/workspace/build/src/gcc/bin/g++ -o Unified_cpp_mozglue_baseprofiler0.o -c -I/builds/worker/workspace/build/src/obj-firefox/dist/stl_wrappers -I/builds/worker/workspace/build/src/obj-firefox/dist/system_wrappers -include /builds/worker/workspace/build/src/config/gcc_hidden.h -DDEBUG=1 -DIMPL_MFBT -DMOZ_VTUNE_INSTRUMENTATION -DMOZ_HAS_MOZGLUE -DIMPL_MFBT -I/builds/worker/workspace/build/src/mozglue/baseprofiler -I/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler -I/builds/worker/workspace/build/src/mozglue/baseprofiler/core -I/builds/worker/workspace/build/src/mozglue/linker -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wduplicated-cond -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object -Wformat -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -freorder-blocks -Os -fno-omit-frame-pointer -funwind-tables -Werror -Wno-error=shadow -Wno-ignored-qualifiers -MD -MP -MF .deps/Unified_cpp_mozglue_baseprofiler0.o.pp /builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler/Unified_cpp_mozglue_baseprofiler0.cpp
[task 2019-06-04T08:51:41.486Z] 08:51:41 INFO - In file included from /builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler/Unified_cpp_mozglue_baseprofiler0.cpp:101:0:
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/mozglue/baseprofiler/core/platform.cpp: In function 'void mozilla::baseprofiler::profiler_ensure_started(uint32_t, double, uint32_t, const char**, uint32_t, const mozilla::Maybe<double>&)':
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - /builds/worker/workspace/build/src/mozglue/baseprofiler/core/platform.cpp:2682:8: error: variable 'startedProfiler' set but not used [-Werror=unused-but-set-variable]
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - bool startedProfiler = false;
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - ^~~~~~~~~~~~~~~
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - cc1plus: all warnings being treated as errors
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/config/rules.mk:810: recipe for target 'Unified_cpp_mozglue_baseprofiler0.o' failed
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - make[4]: *** [Unified_cpp_mozglue_baseprofiler0.o] Error 1
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/baseprofiler'
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'mozglue/baseprofiler/target' failed
[task 2019-06-04T08:51:41.487Z] 08:51:41 ERROR - make[3]: *** [mozglue/baseprofiler/target] Error 2
[task 2019-06-04T08:51:41.487Z] 08:51:41 INFO - make[3]: *** Waiting for unfinished jobs....
[task 2019-06-04T08:51:41.488Z] 08:51:41 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/media/libjpeg'
[task 2019-06-04T08:51:41.491Z] 08:51:41 INFO - media/libjpeg/jcapimin.o

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882141&repo=autoland&lineNumber=36379

[task 2019-06-04T08:51:48.500Z] 08:51:48 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests/interceptor'
[task 2019-06-04T08:51:48.500Z] 08:51:48 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests'
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - /builds/worker/workspace/build/src/sccache/sccache /builds/worker/workspace/build/src/clang/bin/x86_64-w64-mingw32-clang++ -mwindows -o TestBaseProfiler.o -c -DDEBUG=1 -DMOZ_HAS_MOZGLUE -I/builds/worker/workspace/build/src/mozglue/tests -I/builds/worker/workspace/build/src/obj-firefox/mozglue/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/testing -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++1z-compat -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-unknown-pragmas -Wno-unused-function -Wno-conversion-null -Wno-switch -Wno-enum-compare -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fms-extensions -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -mms-bitfields -Wno-incompatible-ms-struct -fno-rtti -ffunction-sections -fdata-sections -Wa,-mbig-obj -fno-exceptions -fno-math-errno -pipe -g -gcodeview -O1 -fno-omit-frame-pointer -funwind-tables -MD -MP -MF .deps/TestBaseProfiler.o.pp /builds/worker/workspace/build/src/mozglue/tests/TestBaseProfiler.cpp
[task 2019-06-04T08:51:48.501Z] 08:51:48 ERROR - /builds/worker/workspace/build/src/mozglue/tests/TestBaseProfiler.cpp:20:4: error:
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - # error
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - ^
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - 1 error generated.
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - /builds/worker/workspace/build/src/config/rules.mk:810: recipe for target 'TestBaseProfiler.o' failed
[task 2019-06-04T08:51:48.501Z] 08:51:48 ERROR - make[4]: *** [TestBaseProfiler.o] Error 1
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mozglue/tests'
[task 2019-06-04T08:51:48.501Z] 08:51:48 INFO - make[4]: *** Waiting for unfinished jobs....

Flags: needinfo?(gsquelart)

Looks like I didn't pick enough build jobs in my Try... Thanks Bodgan, will follow up.

Flags: needinfo?(gsquelart)

Another failure in 'pkg':
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249882250&repo=autoland&lineNumber=87673

[task 2019-06-04T08:51:52.994Z] The error occurred while processing the following file:
[task 2019-06-04T08:51:52.994Z]
[task 2019-06-04T08:51:52.995Z] /builds/worker/workspace/sm-package/mozjs-69.0a1.0/mozglue/moz.build
[task 2019-06-04T08:51:52.995Z]
[task 2019-06-04T08:51:52.995Z] The underlying problem is we referenced a path that does not exist. That path is:
[task 2019-06-04T08:51:52.995Z]
[task 2019-06-04T08:51:52.995Z] /builds/worker/workspace/sm-package/mozjs-69.0a1.0/mozglue/baseprofiler/moz.build

So I'll need to add mozglue/baseprofiler in there. (Can't see a way not to, as mozglue/moz.build now unconditionally refers to it.)

Added mozglue/mozprofiler to js/src/make-source-package.sh, because
mozglue/moz.build will refer to it unconditionally.
Note that if MOZ_GECKO_PROFILER and MOZ_BASE_PROFILER are not defined, no
actual code will be generated.

Depends on D31923

Attachment #9068931 - Attachment description: Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn → Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn,sfink
Attachment #9068931 - Attachment description: Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn,sfink → Bug 1492121 - MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r?njn
Pushed by gsquelart@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/72c4026e9455 Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/db1506f94d0d Copy mozglue/baseprofiler in make-source-package.sh - r=sfink https://hg.mozilla.org/integration/autoland/rev/1f10b50f758f MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/77e7b13c6237 All necessary changes to make baseprofiler build - r=njn https://hg.mozilla.org/integration/autoland/rev/bf1731627e07 TestBaseProfiler - r=njn https://hg.mozilla.org/integration/autoland/rev/39c486afacec Enable Base Profiler by default on Linux and Mac - r=njn https://hg.mozilla.org/integration/autoland/rev/a3fe26927b31 Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn https://hg.mozilla.org/integration/autoland/rev/46f57504c087 Added "BASE" to all public macros - r=njn https://hg.mozilla.org/integration/autoland/rev/b54b813c4c6c Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/4215fefb6ef3 Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/2ffb6ccca437 Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn https://hg.mozilla.org/integration/autoland/rev/90aeaad4a4de Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn https://hg.mozilla.org/integration/autoland/rev/e707f1890820 Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn

Thanks Noemi. I thought I had done an "everything" try from ./mach try chooser but it seems it doesn't actually do everything 😒

Flags: needinfo?(gsquelart)

Valgrind report:
TEST-UNEXPECTED-FAIL | valgrind-test | 192 bytes in 3 blocks are definitely lost at malloc / dl_open_worker / _dl_catch_error / _dl_open
==2561== 192 bytes in 3 blocks are definitely lost in loss record 348 of 399
==2561== at 0x4C2B240: malloc+112 (vg_replace_malloc.c:298)
==2561== by 0x4012919: dl_open_worker+1977 (dl-open.c:457)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x4011CC9: _dl_open+185 (dl-open.c:633)
==2561== by 0x5051F65: dlopen_doit+101 (dlopen.c:67)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x50522EB: _dlerror_run+123 (dlerror.c:164)
==2561== by 0x5051EE0: dlopen@@GLIBC_2.2.5+48 (dlopen.c:88)
==2561== by 0x1148FC: GetLibHandle (xpcom/glue/standalone/nsXPCOMGlue.cpp:86)
==2561== by 0x1148FC: ReadDependentCB (xpcom/glue/standalone/nsXPCOMGlue.cpp:136)
==2561== by 0x1148FC: XPCOMGlueLoad (xpcom/glue/standalone/nsXPCOMGlue.cpp:306)
==2561== by 0x1148FC: mozilla::GetBootstrap(char const*, mozilla::LibLoadingStrategy)+572 (xpcom/glue/standalone/nsXPCOMGlue.cpp:374)
==2561== by 0x114213: InitXPCOMGlue(mozilla::LibLoadingStrategy)+131 (browser/app/nsBrowserApp.cpp:223)
==2561== by 0x113E6B: main+219 (browser/app/nsBrowserApp.cpp:284)

Nothing due directly to this patch, so it is likely a dlopen issue.

Depends on D31932

Pushed by gsquelart@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/64fe00416593 Copy most of Gecko Profiler code to mozglue/baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/1a63bc8f0afd Copy mozglue/baseprofiler in make-source-package.sh - r=sfink https://hg.mozilla.org/integration/autoland/rev/838a84ef85f8 MOZ_BASE_PROFILER may be defined in BaseProfiler.h to enable Base Profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/d0be75b5f773 All necessary changes to make baseprofiler build - r=njn https://hg.mozilla.org/integration/autoland/rev/3a7f3ab482b1 TestBaseProfiler - r=njn https://hg.mozilla.org/integration/autoland/rev/4d72e64503c8 Enable Base Profiler by default on Linux and Mac - r=njn https://hg.mozilla.org/integration/autoland/rev/842723642da6 Rename env-vars MOZ_PROFILER_* to MOZ_BASE_PROFILER_* - r=njn https://hg.mozilla.org/integration/autoland/rev/b4c77fd9b97d Added "BASE" to all public macros - r=njn https://hg.mozilla.org/integration/autoland/rev/a4c102433761 Enclosing BaseProfiler in namespace mozilla::baseprofiler - r=njn https://hg.mozilla.org/integration/autoland/rev/a679e252817a Run BaseProfiler's InitializeWin64ProfilerHooks once from either profiler - r=njn https://hg.mozilla.org/integration/autoland/rev/ebe23063def2 Gecko Profiler integrates BaseProfiler startup profiled threads - r=njn https://hg.mozilla.org/integration/autoland/rev/0cbffc5bb535 Suppress dlopen leak triggered by next patch - r=njn https://hg.mozilla.org/integration/autoland/rev/ebfca0243df2 Use BaseProfiler in nsBrowserApp.cpp's main() - r=njn https://hg.mozilla.org/integration/autoland/rev/0b97aaf0efce Use BaseProfiler in XRE_Init{Parent,Child}Process() - r=njn
Regressions: 1557841
Blocks: 2011833
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: