1516367 - The minidump-analyzer crashes on macOS with "Library not loaded: @executable_path/libmozglue.dylib"

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect, P1)

Description

[Stephen A Pohl [:spohl] (OOO until 3/25)]

The crash reporter crashes when a content process crashes. I have seen this several times over the past few days. Firefox 64.0 on macOS 10.13.6. I suspect that one of our build scripts failed to replace the "@executable_path", which results in a crash due to libmozglue.dylib not being loaded with reason "image not found". Here is the macOS crash report:

Process:               minidump-analyzer [6296]
Path:                  /Applications/Firefox.app/Contents/MacOS/crashreporter.app/Contents/MacOS/minidump-analyzer
Identifier:            minidump-analyzer
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        firefox [1954]
Responsible:           minidump-analyzer [6296]
User ID:               501

Date/Time:             2018-12-25 18:24:52.444 -0500
OS Version:            Mac OS X 10.13.6 (17G4015)
Report Version:        12
Bridge OS Version:     3.0 (14Y664)
Anonymous UUID:        DE2A43B1-5B3C-5752-9F3C-1EE42AEB2B33


Time Awake Since Boot: 730000 seconds

System Integrity Protection: enabled

Crashed Thread:        0

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    DYLD, [0x1] Library missing

Application Specific Information:
dyld: launch, loading dependent libraries

Dyld Error Message:
  Library not loaded: @executable_path/libmozglue.dylib
  Referenced from: /Applications/Firefox.app/Contents/MacOS/crashreporter.app/Contents/MacOS/minidump-analyzer
  Reason: image not found

Binary Images:
       0x109682000 -        0x1096f0fff +minidump-analyzer (0) <184D92AD-5416-3784-83B5-FDC5CF24E794> /Applications/Firefox.app/Contents/MacOS/crashreporter.app/Contents/MacOS/minidump-analyzer
       0x10fbf5000 -        0x10fc3facf  dyld (551.5) <30B355CB-35BA-3112-AA76-4E46CD45F699> /usr/lib/dyld
    0x7fff37a51000 -     0x7fff37cdfff7  com.apple.audio.toolbox.AudioToolbox (1.14 - 1.14) <E0B8B5D8-80A0-308B-ABD6-F8612102B5D8> /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
    0x7fff388c6000 -     0x7fff388c6fff  com.apple.Cocoa (6.11 - 22) <80C1AABB-FC32-3833-A53B-0E36E70EBABA> /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
    0x7fff3b1b4000 -     0x7fff3b1b6ff3  com.apple.ExceptionHandling (1.5 - 12) <66F613CE-62BE-31C0-9F52-DCC2DAB762CA> /System/Library/Frameworks/ExceptionHandling.framework/Versions/A/ExceptionHandling
    0x7fff60402000 -     0x7fff607f07e7  libobjc.A.dylib (723) <DD9E5EC5-B507-3249-B700-93433E2D5EDF> /usr/lib/libobjc.A.dylib

Model: MacBookPro13,3, BootROM 250.0.0.0.0, 4 processors, Intel Core i7, 2.9 GHz, 16 GB, SMC 2.38f7
Graphics: Intel HD Graphics 530, Intel HD Graphics 530, Built-In
Graphics: Radeon Pro 460, AMD Radeon Pro 460, PCIe
Memory Module: BANK 0/DIMM0, 8 GB, LPDDR3, 2133 MHz, 0x80CE, 0x4B3445424533303445422D45474347202020
Memory Module: BANK 1/DIMM0, 8 GB, LPDDR3, 2133 MHz, 0x80CE, 0x4B3445424533303445422D45474347202020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x15A), Broadcom BCM43xx 1.0 (7.77.37.31.1a9)
Bluetooth: Version 6.0.7f11, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
USB Device: USB 3.0 Bus
USB Device: USB2.0 Hub
USB Device: USB-C Digital AV Multiport Adapter
USB Device: iBridge
USB Device: USB2.0 Hub
USB Device: USB-C Digital AV Multiport Adapter
USB Device: USB3.0 Hub
USB Device: USB3.0 Hub
Thunderbolt Bus: MacBook Pro, Apple Inc., 33.1
Thunderbolt Bus: MacBook Pro, Apple Inc., 33.1

Comment 1

[Gabriele Svelto [:gsvelto]]

This should not prevent crash reporting from working but it will reduce the amount of useful information we end up with in crash pings. Moving it to the proper component.

Comment 2

[Stephen A Pohl [:spohl] (OOO until 3/25)]

Here is a crash report from this morning:
https://crash-stats.mozilla.com/report/index/403f7e86-1026-4296-999a-457e10190121#allthreads

Comment 3

[Gabriele Svelto [:gsvelto]]

I've got a fix for this. The issue boils down to where minidump-analyzer is stored vs where libmozglue. This prevents crash pings on Mac from having stacks for the crashed thread. This isn't a huge problem right now because (almost) nobody is looking at those yet, but they'll be used soon so leaving at P1.

Comment 4

[macterrassa]

Hi:
This is my first message and I do not know if I put it in the right place yet.
I have a problem that I think is similar: since some versions of Firefox, using Mac OS X 10.11.6, when opening a new tab from some link it is blocked and I get the message to retry load it standard. But although it ends up working, the mac jumps a crash message similar to the first one. Related to the impossibility of loading the libmozglue.dylib library, but as a different type of exception: EXC_BREAKPOINT (SIGTRAP).
I assumed that these messages in general, were problems with bookstores offered by Apple not fully compatible with "obsolete" systems such as 10.11, and in many cases only offered for current versions. Is it like that? Is the error possible and can it be solved or not?
By the way, I'm not a developer but I thought it was more correct to ask in the Firefox development forum, not in the user group directly. In other similar cases with other programs, they relate these problems to the development of the program and to a failure in my use of it.

"Process: minidump-analyzer [1208]
Path: /Applications/Firefox.app/Contents/MacOS/crashreporter.app/Contents/MacOS/minidump-analyzer
Identifier: minidump-analyzer
Version: ???
Code Type: X86-64 (Native)
Parent Process: firefox [827]
Responsible: minidump-analyzer [1208]
User ID: 501

Date/Time: 2019-06-28 12:46:18.639 +0200
OS Version: Mac OS X 10.11.6 (15G22010)
Report Version: 11
Anonymous UUID: 246543A0-FA28-E7C5-B22D-4E86A14F31CF

Time Awake Since Boot: 3200 seconds

System Integrity Protection: enabled

Crashed Thread: 0

Exception Type: EXC_BREAKPOINT (SIGTRAP)
Exception Codes: 0x0000000000000002, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY

Application Specific Information:
dyld: launch, loading dependent libraries

Dyld Error Message:
Library not loaded: @executable_path/libmozglue.dylib
Referenced from: /Applications/Firefox.app/Contents/MacOS/crashreporter.app/Contents/MacOS/minidump-analyzer
Reason: image not found

Binary Images:
0x7fff64a71000 - 0x7fff64aa8a47 dyld (360.22) <6B9CD3BA-2D60-3629-A0CB-AB810F55E8DD> /usr/lib/dyld

"
Regards,

Jose Luis

P.S. I do not speak English, mechanically translated message, I apologize if I do not express myself correctly.

Comment 5

[Stephen A Pohl [:spohl] (OOO until 3/25)]

(In reply to Gabriele Svelto [:gsvelto] from comment #3)

I've got a fix for this. The issue boils down to where minidump-analyzer is stored vs where libmozglue. This prevents crash pings on Mac from having stacks for the crashed thread. This isn't a huge problem right now because (almost) nobody is looking at those yet, but they'll be used soon so leaving at P1.

We are starting to get more reports of this issue. Could we land your fix?

Comment 7

[Gabriele Svelto [:gsvelto]]

I'll try to land it ASAP but I'm caught up in a bunch of other urgent stuff. Leaving the NI? flag so I don't forget.

Comment 8

[Gabriele Svelto [:gsvelto]]

Attachment: Bug 1516367 - Move the minidump-analyzer out of the crash reporter application bundle r=spohl

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5df0e903d13f
Move the minidump-analyzer out of the crash reporter application bundle r=spohl,dmajor

Comment 10

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/5df0e903d13f

Comment 11

[phil.statham]

Attachment: Problem report

Just tried the beta of 72 and this is still not working.