Closed Bug 1797301 Opened 2 years ago Closed 2 years ago

Crash on preload failure of libraries

Categories

(Core :: Security: Process Sandboxing, enhancement, P3)

Product:
Core
Component:
Security: Process Sandboxing
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
108 Branch
Tracking Flags:
Tracking Status
firefox108 --- fixed

People

(Reporter: gerard-majax, Assigned: gerard-majax)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files, 1 obsolete file)

Bug 1797301 - Add lib with failure handling r?nika!
48 bytes, text/x-phabricator-request
Details | Review
Bug 1797301 - Move Utility process to check preloading failures r?nika!
48 bytes, text/x-phabricator-request
Details | Review
Bug 1797301 - Rework preloading for WMF r?alwu!
48 bytes, text/x-phabricator-request
Details | Review
Severity: -- → S4
Priority: -- → P3
Attachment #9300868 - Flags: data-review?(chutten)
Blocks: 1798069
Blocks: 1798070

Comment on attachment 9300868 [details]
data review for https://phabricator.services.mozilla.com/D160685

Is there truly no place where crash annotations are documented as data collections? That's one of the more firm requirements of data collection review.

Attachment #9300868 - Flags: data-review?(chutten) → data-review-

(In reply to Chris H-C :chutten from comment #4)

Comment on attachment 9300868 [details]
data review for https://phabricator.services.mozilla.com/D160685

Is there truly no place where crash annotations are documented as data collections? That's one of the more firm requirements of data collection review.

I am not sure what you are asking for there. If that is about "6) Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.", I'm afraid I dont have a good answer. We will just have a library name going into the crash reports.

Pushed by alissy@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/147777c37f57
Add lib with failure handling r=nika
https://hg.mozilla.org/integration/autoland/rev/2552990ed77e
Move Utility process to check preloading failures r=nika

(In reply to Alexandre LISSY :gerard-majax from comment #5)

(In reply to Chris H-C :chutten from comment #4)

Comment on attachment 9300868 [details]
data review for https://phabricator.services.mozilla.com/D160685

Is there truly no place where crash annotations are documented as data collections? That's one of the more firm requirements of data collection review.

I am not sure what you are asking for there. If that is about "6) Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.", I'm afraid I dont have a good answer. We will just have a library name going into the crash reports.

Yes, it is about that. For Histograms and friends there's the Probe Dictionary, for Legacy Telemetry Custom Pings each is expected to keep their source docs up to date, for everything in Glean you get the Glean Dictionary...

If I were a user who wanted to know about crash annotations in general and specific, where would I turn?

Comment on attachment 9300868 [details]
data review for https://phabricator.services.mozilla.com/D160685

retracting data-review, I've removed MOZ_CRASH_UNSAFE_PRINTF use

Attachment #9300868 - Flags: data-review-
Attachment #9300868 - Attachment is obsolete: true

(In reply to Chris H-C :chutten from comment #7)

If I were a user who wanted to know about crash annotations in general and specific, where would I turn?

I'm sorry but I dont know exactly, it would just have been exposed in MOZ_CRASH Reason (Sanitized) on any crash report like e.g., https://crash-stats.mozilla.org/report/index/23bcef23-db3b-458c-b805-180340221102 MOZ_CRASH Reason (Sanitized) Shutdown hanging at step XPCOMShutdownThreads. Something is blocking the main-thread.

Hrm, maybe this'd be a good question for WillKG: Where does the public documentation for crash annotations live? Or Crash Reports' contents in general?

Flags: needinfo?(willkg)

Crash annotations are documented in CrashAnnotations.yaml including MozCrashReason.

If I understand what I'm skimming correctly, the code changes in this bug don't create a new annotation--they either add a new or adjust an existing reason value to the MozCrashReason annotation, but that's fine and within the way the annotation is already used. I don't think this requires a data review.

Flags: needinfo?(willkg)

Backed out for causing Bp-Hybrid bustages.

Pushed by alissy@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4f523653d387
Add lib with failure handling r=nika
https://hg.mozilla.org/integration/autoland/rev/7268493e86bd
Move Utility process to check preloading failures r=nika

Backed out for causing bp-hybrid bustages in ipc/glue/test/gtest/TestUtils.h

Backout link: https://hg.mozilla.org/integration/autoland/rev/99af90df8af6f0175aa5407e9c7712e4ab4eb71e

Push with failures

Failure log

Failure log bustage win

/builds/worker/checkouts/gecko/ipc/glue/test/gtest/TestUtils.h:14:13: error: unused function 'DisableCrashReporter' [-Werror,-Wunused-function]
Flags: needinfo?(lissyx+mozillians)
Flags: needinfo?(lissyx+mozillians)

Silly mistake, I forgot that this one was only for windows

Pushed by alissy@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4bb66adab63a
Add lib with failure handling r=nika
https://hg.mozilla.org/integration/autoland/rev/2ad5f267bbf2
Move Utility process to check preloading failures r=nika

Backed out for causing windows build bustage in mozilla::wmf::MediaFoundationInitializer::HasInitialized()

Backout link: https://hg.mozilla.org/integration/autoland/rev/81af3953de3d19d670f4df0142505270c7642a52

Push with failures

Failure log

PROCESS-CRASH | Firefox exited successfully? | application crashed [@ mozilla::wmf::MediaFoundationInitializer::HasInitialized()]
Flags: needinfo?(lissyx+mozillians)
Flags: needinfo?(lissyx+mozillians)

This time we are failing on mfplat.dll: https://treeherder.mozilla.org/logviewer?job_id=395328482&repo=try&lineNumber=354
Alastor, is it something we would expect on Windows 2012 ?

Flags: needinfo?(alwu)

:jmaher, can you work with relops to get a loaner windows builder or check for mfplat.dll in the system directory. We can at least determine if this is on our builders (windows server 2012 - probably doesn't have all end user libraries)

Flags: needinfo?(jmaher)

It looks like the mfplat.dll is part of a Media Feature Pack. It is a safe assumption that the 2012 workers do not have the needed media pack installed or feature enabled. Though I will work on verifying that.

The Media Features are not enabled on 2012. However, mfplat.dll is on the workers in the directories listed below.

c:\>dir mfplat.dll /s
 Volume in drive C is os
 Volume Serial Number is E247-9190

 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.16384_none_e768f8b8276c5796
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17031_none_e79beafc2746a208
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17090_none_e75a0b2a27783057
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17238_none_e7a2f0ce27404db7
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17415_none_e7b593942732c41e
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17490_none_e75a12c2277824f3
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.16384_none_8b4a5d346f0ee660
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17031_none_8b7d4f786ee930d2
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17090_none_8b3b6fa66f1abf21
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17238_none_8b84554a6ee2dc81
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17415_none_8b96f8106ed552e8

If needed we could try enabling the feature and/or adding the Windows Media SDK, but i don't know how that would affect other things that happen on these workers.

(In reply to Mark Cornmesser [:markco] from comment #22)

The Media Features are not enabled on 2012. However, mfplat.dll is on the workers in the directories listed below.

c:\>dir mfplat.dll /s
 Volume in drive C is os
 Volume Serial Number is E247-9190

 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.16384_none_e768f8b8276c5796
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17031_none_e79beafc2746a208
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17090_none_e75a0b2a27783057
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17238_none_e7a2f0ce27404db7
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17415_none_e7b593942732c41e
 Directory of c:\Windows\WinSxS\amd64_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17490_none_e75a12c2277824f3
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.16384_none_8b4a5d346f0ee660
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17031_none_8b7d4f786ee930d2
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17090_none_8b3b6fa66f1abf21
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17238_none_8b84554a6ee2dc81
 Directory of c:\Windows\WinSxS\x86_microsoft-windows-mfplat_31bf3856ad364e35_6.3.9600.17415_none_8b96f8106ed552e8

If needed we could try enabling the feature and/or adding the Windows Media SDK, but i don't know how that would affect other things that happen on these workers.

Thanks, so maybe mfplat.dll is there but since Media feature is missing/not enabled then you can't load it. I dont think we should change anything, this just highlights that maybe we should be less strict than anticipated on those checks.

Flags: needinfo?(jmaher)

Depends on D160686

Flags: needinfo?(alwu)
Pushed by alissy@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f8406f212dc3
Add lib with failure handling r=nika
https://hg.mozilla.org/integration/autoland/rev/636226d53ee8
Move Utility process to check preloading failures r=nika
https://hg.mozilla.org/integration/autoland/rev/4a8b15ebc3a5
Rework preloading for WMF r=alwu

https://hg.mozilla.org/mozilla-central/rev/f8406f212dc3
https://hg.mozilla.org/mozilla-central/rev/636226d53ee8
https://hg.mozilla.org/mozilla-central/rev/4a8b15ebc3a5

Status: NEW → RESOLVED
Closed: 2 years ago
status-firefox108: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 108 Branch
Regressions: 1800226
Regressions: 1800033
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: