1693502 - Crash [@ mozilla::ExtensionPolicyService::GetSingleton ]

Status: RESOLVED FIXED

(Core :: Gecko Profiler, defect, P2)

Description

[Florian Quèze [:florian]]

I encountered this crash today: https://crash-stats.mozilla.org/report/index/3fdb4fad-17d2-477b-991c-f5c020210218

I thought it was during startup profiling, but seeing the content of the stack that might have been shutdown profiling (I thought I had captured profiles and stopped the profiler, but maybe not).

0 	xul.dll 	static mozilla::ExtensionPolicyService::GetSingleton() 	toolkit/components/extensions/ExtensionPolicyService.cpp:81 	context
1 	xul.dll 	StreamMetaJSCustomObject(PSAutoLock const&, mozilla::baseprofiler::SpliceableJSONWriter&, bool, PreRecordedMetaInformation const&) 	tools/profiler/core/platform.cpp:2641 	cfi
2 	xul.dll 	locked_profiler_stream_json_for_this_process(PSAutoLock const&, mozilla::baseprofiler::SpliceableJSONWriter&, double, PreRecordedMetaInformation const&, bool, ProfilerCodeAddressService*) 	tools/profiler/core/platform.cpp:2827 	cfi
3 	xul.dll 	locked_profiler_save_profile_to_file(PSAutoLock const&, char const*, PreRecordedMetaInformation const&, bool) 	tools/profiler/core/platform.cpp:4545 	cfi
4 	xul.dll 	profiler_shutdown(IsFastShutdown) 	tools/profiler/core/platform.cpp:4309 	cfi
5 	xul.dll 	XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) 	toolkit/xre/nsAppRunner.cpp:5494

The Crash Reason is MOZ_RELEASE_ASSERT(mObs). https://searchfox.org/mozilla-central/rev/8432d4fe31245ae9121231d7c0335aaa342675d2/toolkit/components/extensions/ExtensionPolicyService.cpp#89

I don't understand how it's possible given the gXPCOMShuttingDown check at https://hg.mozilla.org/mozilla-central/file/6d7590bfd8d37fd1088f8d184702238881fc048b/tools/profiler/core/platform.cpp#l2628

Comment 1

[Florian Quèze [:florian]]

I got the same crash stack a second time. Now I'm convinced it was really during startup. The crash annotations say "StartupCrash: 1", and the uptime of 8 or 9s is too small for a shutdown crash.

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Good clue about gXPCOMShuttingDown.

Hypothesis:
ScopedXPCOMStartup is instantiated there in XREMain::XREMain() : https://searchfox.org/mozilla-central/rev/281009d3b7e1e1496b9524d5478ff4f0b7369573/toolkit/xre/nsAppRunner.cpp#5435
(Its destructor calls ShutdownXPCOM(), which sets gXPCOMShuttingDown to true before destroying services.)
So if there is an "early return" before that line (but after AUTO_PROFILER_INIT), gXPCOMShuttingDown will stay at its default false, and before exiting the function, profiler_shutdown is still called... BOOM

So we'll need something better than testing gXPCOMShuttingDown. Maybe the profiler's StreamMetaJSCustomObject() should instead try services::GetObserverService();, which will give us the correct answer in all cases.

Note that this may not be the true cause of this particular crash, but I think it's a possible scenario that should be fixed anyway.

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1693502 - Check GetObserverService() instead of gXPCOMShuttingDown to handle pre/failed-XPCOM calls - r?florian

In XREMain(), if something went wrong between PROFILER_INIT and MakeUnique<ScopedXPCOMStartup>(),

• XPCOM would not be initialized -- Meaning ExtensionPolicyService::GetSingleton() would assert when not finding the Observer Service,
• But gXPCOMShuttingDown would still be false.
  So StreamMetaJSCustomObject (invoked by profiler_shutdown, called by the PROFILER_INIT object destructor at the end of XREMain()) would still attempt to use ExtensionPolicyService::GetSingleton().

In this patch, StreamMetaJSCustomObject now checks that the observer service exists, instead of only testing !gXPCOMShuttingDown. This should handle all situations where ExtensionPolicyService::GetSingleton() could assert, not just during normal shutdown.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cc927878ddbf
Check GetObserverService() instead of gXPCOMShuttingDown to handle pre/failed-XPCOM calls - r=florian

Comment 5

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/cc927878ddbf