1724368 - Catch `EXC_RESOURCE` exceptions on macOS

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, enhancement)

Description

[Gabriele Svelto [:gsvelto] (PTO)]

We need to catch EXC_RESOURCE exceptions on macOS as they cover a variety of crashes that can happen when exceeding certain resource thresholds.

The various types of resources and how their data is in the exception code field are documented here.

To implement this we need to do the following:

• Add the EXC_RESOURCE exception to the list of exceptions we request to be notified of in the mac exception handler
• Modify the mac minidump writer to account for these exception and store the code field correctly in the minidump
• Adjust the minidump processor to properly decode the exception code field and print out its contents

Once we'll have applied these changes to mozilla-central I'll need to file another bug to import them in Socorro's stack-walker.

Comment 1

[Gabriele Svelto [:gsvelto] (PTO)]

Corresponding issue in rust-minidump

Comment 2

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Bug 1724368 - Add support for catching and processing EXC_RESOURCE exceptions on macOS r=KrisWright

This introduces a few changes to the crash reporting machinery:

• The macOS exception handler now registers itself for catching EXC_RESOURCE
  exceptions, those are thrown when the process exceeds a pre-set resource
  limit (memory, CPU usage, I/O, etc...)
• The minidump writer has been updated to correctly store the subcode from the
  EXC_RESOURCE exceptions, this involves widening to 64-bits the code and
  subcode passed to the writer. The upper 32 bits of the code are now set in
  the minidump's exception_flags field (vs the lower 32 bits for all other
  exceptions). Additionally the exception type, code and subcode are now
  stored in the exception_information array like Crashpad does. This preserves
  the entirety of the data that came with the exception.
• The stackwalker has been modified to print out these type of exceptions as
  well as the resource type and flavor.

This also fixes a couple of typos in the previous patch we applied to
breakpad.

Comment 3

[Gabriele Svelto [:gsvelto] (PTO)]

Attachment: Bug 1724368 - Add support for catching and processing EXC_GUARD exceptions on macOS r=KrisWright

Besides adding EXC_GUARD to the list of exceptions the exception handler
listens to and modifying the minidump processor to interpret the exception
correctly this patch introduces a unit-test that crashes with an EXC_RESOURCE
exception.

Information about the exception itself is available here:

https://github.com/apple/darwin-xnu/blob/main/osfmk/kern/exc_guard.h

Additional flavors for the exception are documented here:

https://github.com/apple/darwin-xnu/osfmk/mach/port.h
https://github.com/apple/darwin-xnu/osfmk/mach/vm_statistics.h

Last but not least the undocumented API that generates these exceptions and
which we use in the unit-test was inferred from this header:

https://github.com/apple/darwin-xnu/blob/main/bsd/sys/guarded.h

Depends on D122229

Comment 4

[Phabricator Automation]

Comment on attachment 9235838 [details]
Bug 1724368 - Add support for catching and processing EXC_GUARD exceptions on macOS r=KrisWright

Revision D122374 was moved to bug 1724388. Setting attachment 9235838 [details] to obsolete.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fd5d00822477
Add support for catching and processing EXC_RESOURCE exceptions on macOS r=KrisWright

Comment 6

[Alexandru Michis [:malexandru]]

Backed out 3 changesets (Bug 1724368, Bug 1725154, Bug 1724388) for causing bustages in minidump_processor.cc
Backout link: https://hg.mozilla.org/integration/autoland/rev/35aecc2db94d6be53e02aa88f1d54575b1782bc5
Push with failures, failure log.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b3cfc657b5fb
Add support for catching and processing EXC_RESOURCE exceptions on macOS r=KrisWright

Comment 8

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/b3cfc657b5fb

Comment 9

[Gabriele Svelto [:gsvelto] (PTO)]

Botched patch, re-landed fine.