1862279 - Crashes in [@ pthread_kill | abort | GeForceGLDriver@0x30736f] on macOS versions that don't support GeForce video hardware, installed by OCLP

Status: NEW

(Core :: Graphics, defect)

Description

[BugBot [:suhaib / :marco/ :calixte]]

Crash report: https://crash-stats.mozilla.org/report/index/18238543-cd42-44ca-b025-5530a0231028

Reason: EXC_SOFTWARE / SIGABRT

Top 10 frames of crashing thread:

0  libsystem_kernel.dylib  __pthread_kill  
1  libsystem_pthread.dylib  pthread_kill  
2  libsystem_c.dylib  abort  
3  libGPUSupportMercury.dylib  gpusGenerateCrashLog  
4  GeForceGLDriver  GeForceGLDriver@0x30736f  
5  libGPUSupportMercury.dylib  gpusSubmitDataBuffers  
6  GeForceGLDriver  GeForceGLDriver@0x30282d  
7  GeForceGLDriver  GeForceGLDriver@0x2c6847  
8  GeForceGLDriver  GeForceGLDriver@0x30a46f  
9  GeForceGLDriver  GeForceGLDriver@0x3239f0  

By querying Nightly crashes reported within the last 2 months, here are some insights about the signature:

• First crash report: 2023-08-24
• Process type: Parent
• Is startup crash: No
• Has user comments: No
• Is null crash: No

Comment 1

[Steven Michaud [:smichaud] (Retired)]

Apple dropped support for GeForce video hardware in macOS 12 (Monterey). But most of these crashes are on macOS 12 or higher. So I suspect many of the crashing machines are hackintoshes, or that something like OCLP is involved.

https://crash-stats.mozilla.org/search/?proto_signature=~GeForceGLDriver&platform_version=%5E12.&platform_version=%5E13.&platform_version=%5E14.&platform=Mac%20OS%20X&date=%3E%3D2023-10-01T15%3A25%3A00.000Z&date=%3C2023-11-01T15%3A25%3A00.000Z&_facets=signature&_facets=platform_version&_facets=proto_signature&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-platform_version

Comment 2

[Steven Michaud [:smichaud] (Retired)]

All of the crashes with this bug's particular signature are on macOS 12 or higher:

https://crash-stats.mozilla.org/search/?signature=~GeForceGLDriver%400x30736f&platform=Mac%20OS%20X&date=%3E%3D2023-08-01T15%3A34%3A00.000Z&date=%3C2023-11-01T15%3A34%3A00.000Z&_facets=signature&_facets=platform_version&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-platform_version

Comment 3

[Steven Michaud [:smichaud] (Retired)]

The fact that crashes on different versions of macOS all share the same unsymbolicated signature suggests that they happen in a GeForceGLDriver binary that was copied from a single version of macOS that still supports GeForce hardware, and that it was done "systemmatically", by something like OCLP.

So there probably aren't any hackintoshes here.

Comment 4

[Steven Michaud [:smichaud] (Retired)]

I grepped through my archives of manually scraped macOS symbols, and didn't find a GeForceGLDriver with UUID 0669F00AFD85383BBA83F8A42C2EB5270 (this module's UUID in all this bug's crash stacks). These archives go back to 2019, and cover fairly completely new versions of macOS that were released since then.

I'm not entirely sure what this means. One possibility is that this bug's GeForceGLDriver comes from a non-Apple source.

Comment 5

[Steven Michaud [:smichaud] (Retired)]

It'd be really nice to get bug 1835881 fixed. OCLP leaves traces in macOS's kernel boot args. And if the source of this bug's GeForceGLDriver isn't OCLP, it's also likely to have changed the kernel boot args.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

(In reply to Steven Michaud [:smichaud] (Retired) from comment #4)

I grepped through my archives of manually scraped macOS symbols, and didn't find a GeForceGLDriver with UUID 0669F00AFD85383BBA83F8A42C2EB5270 (this module's UUID in all this bug's crash stacks). These archives go back to 2019, and cover fairly completely new versions of macOS that were released since then.

I'm not entirely sure what this means. One possibility is that this bug's GeForceGLDriver comes from a non-Apple source.

From OCLP's source code, it looks like they're the ones who copied in this bug's GeForceGLDriver module, and that it comes from macOS "11.0 Beta 3", whose build id was "20A5323l" (as you can see from digging around in the the editing history of https://en.wikipedia.org/wiki/MacOS_Big_Sur#Release_history). This is one of the 11.0 beta releases I didn't scrape.

So yes, the source of the GeForceGLDriver module does seem to be OCLP, and it does seem to come from Apple (though it puzzles me that they copied it from a very early beta of macOS 11). The Wikipedia page's history doesn't reveal the UUID, so I can't check it. But this really does seem quite likely.

I can't be sure until bug 1835881 is fixed, though.

Comment 7

[Steven Michaud [:smichaud] (Retired)]

OK, I've now confirmed that OCLP's GeForceGLDriver module has UUID 0669F00A-FD85-383B-BA83-F8A42C2EB527. It's embedded in their OpenCore-Patcher.app, in OpenCore-Patcher.app/Contents/MacOS/Universal-Binaries.dmg. Opening this requires a password, but you can tell from the source code that the password is "password" :-)

Opening this DMG will mount a OpenCore Patcher Resources (Root Patching) volume, which contains GeForceGLDriver at /Volumes/OpenCore Patcher Resources (Root Patching)/11.0 Beta 3/System/Library/Extensions/GeForceGLDriver.bundle/Contents/MacOS/GeForceGLDriver. To get the UUID, do otool -l GeForceGLDriver | grep uuid.

Comment 8

[Steven Michaud [:smichaud] (Retired)]

All of the bug's crash reports have "mac_crash_info". Most look like this:

{
  "num_records": 2,
  "records": [
    {
      "abort_cause": null,
      "backtrace": null,
      "dialog_mode": null,
      "message": "abort() called",
      "message2": null,
      "module": "/usr/lib/system/libsystem_c.dylib",
      "signature_string": null,
      "thread": null
    },
    {
      "abort_cause": null,
      "backtrace": null,
      "dialog_mode": null,
      "message": null,
      "message2": null,
      "module": "/System/Library/PrivateFrameworks/GPUSupport.framework/Versions/A/Libraries/libGPUSupportMercury.dylib",
      "signature_string": "Graphics kernel error: 0xfffffffb\n",
      "thread": null
    }
  ]
}

Comment 9

[Steven Michaud [:smichaud] (Retired)]

Needless to say, these are probably an Apple bug (or bugs). They also probably effect other applications. But that's hard to know because the big guys (Apple and Alphabet) hide their crash reporting systems.

Comment 10

[Steven Michaud [:smichaud] (Retired)]

OCLP might want to try using a GeForceGLDriver.bundle (and other GeForce components) from a more recent version of macOS BigSur than "11.0 beta 3". But there's no guarantee that will fix the problem. These crashes on BigSur (and without OCLP) don't seem to tail off in the most recent BigSur releases:

https://crash-stats.mozilla.org/search/?modules_in_stack=~geforcegldriver&platform_version=%5E11.&platform=Mac%20OS%20X&date=%3E%3D2023-05-02T16%3A14%3A00.000Z&date=%3C2023-11-02T16%3A14%3A00.000Z&_facets=signature&_facets=proto_signature&_facets=platform_version&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-platform_version

Comment 11

[Gabriele Svelto [:gsvelto]]

FYI the vast majority of the crashes here have the following four flags set in the boot arguments:

• keepsyms=1
• debug=0x100
• ipc_control_port_options=0
• -nokcmismatchpanic

Comment 12

[Steven Michaud [:smichaud] (Retired)]

(In reply to Gabriele Svelto [:gsvelto] from comment #11)

FYI the vast majority of the crashes here have the following four flags set in the boot arguments:

• keepsyms=1
• debug=0x100
• ipc_control_port_options=0
• -nokcmismatchpanic

These are all standard settings for OCLP.

If there are no other mac_boot_args that appear in all (or almost all) of this bug's crash reports, then the crashes probably aren't triggered by any mac_boot_arg.