1610213 - Crash in [@ __CFRunLoopServiceMachPort]

Status: NEW

(Core :: General, defect)

Description

[Asif Youssuff]

This bug is for crash report bp-45b65755-176b-40db-a8b2-ecd3b0200119.

Top 10 frames of crashing thread:

0 libsystem_kernel.dylib mach_msg_trap 
1 CoreFoundation __CFRunLoopServiceMachPort 
2 CoreFoundation __CFRunLoopRun 
3 CoreFoundation CFRunLoopRunSpecific 
4 HIToolbox RunCurrentEventLoopInMode 
5 HIToolbox ReceiveNextEventCommon 
6 HIToolbox _BlockUntilNextEventMatchingListInModeWithFilter 
7 AppKit _DPSNextEvent 
8 AppKit -[NSApplication _nextEventMatchingEventMask:untilDate:inMode:dequeue:] 
9 AppKit GCC_except_table7 

Forced Firefox to crash due to multiple hung content processes. This has been happening for a week or two now. I was unable to capture a profile because the profiler would not start after installation.

If I see this again (the hung content processes), I will try to capture a profile and post here. Happy to provide any further information.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:ajones, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[Stephen A Pohl [:spohl]]

Since this has to do with hung content processes I'm going to move this over to IPC for now.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:jld, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 4

[Jed Davis [:jld] ⟨⏰|UTC-8⟩ ⟦he/him⟧]

No evidence that it's IPC, and nothing actionable in comment #0 because the crash report is for the wrong process. (Also the summary is misleading — it was a deliberate crash of an idle process, so the fact that it hit the place where an event loop waits for events is of little informational value.) Resolving as incomplete to stop the bot nagging; reopen if there's more information.

Comment 5

[Steven Michaud [:smichaud] (Retired)]

This is a Cocoa Widgets bug. Or at least some of these crashes are.

Bug 1577886 landed support for mac_crash_info data in Mozilla crash reports. Now one has appeared with this bug's signature and very interesting mac_crash_info data:

    {
      "num_records": 1,
      "records": [
        {
          "message": "Calling windowShouldClose: on the delegate for the ToolbarWindow 0x115acc800",
          "module": "/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit"
        }
      ]
    }

bp-f56a9f79-4556-4024-bb01-13c7a0210525

I haven't investigated, but I suspect this "message" is written just before the call that hangs. It would normally be zeroed out again if that call succeeds. But in this case it didn't.

Edit: This is exactly what happens.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

Attachment: Entire callstack for crashing thread (bp-f56a9f79-4556-4024-bb01-13c7a0210525)

Comment 7

[Stephen A Pohl [:spohl]]

Steven, I'm not familiar with mac_crash_info and I'll make sure to read up on it in bug 1577886. In the meantime, could you tell us how the mac_crash_info data in this instance is adding more info? It looks like frame 45 in the call stack indicated that we were requesting a window to close, and we are dealing with a hang since the process is ultimately killed (see MOZ_CRASH reason "Shutdown hanging at step quit-application. Something is blocking the main-thread."). Thank you

Comment 8

[Steven Michaud [:smichaud] (Retired)]

I still haven't investigated this bug in any detail. But I suspect that, in this one case at least, there's something wrong with the "delegate", or maybe with "ToolbarWindow 0x115acc800". For example it might already have been destroyed.

Comment 9

[Steven Michaud [:smichaud] (Retired)]

Here's another (possibly more informative) crash report with the same mac_crash_info:

bp-04da2905-07f3-44b2-843b-757c40210525

Comment 10

[Steven Michaud [:smichaud] (Retired)]

I'll tentatively assign this to myself. It'll be a while before I can spend much time on it.

I'll unassign it if I decide I don't have the time, or if I give up on it.

Comment 11

[Steven Michaud [:smichaud] (Retired)]

It turns out that most of this bug's crash reports that could have mac_crash_info don't have it. So I'm now less sure that I'm right about this being a Cocoa Widgets bug. But I'll leave myself assigned, and look into it sometime in the next few weeks.

https://crash-stats.mozilla.org/search/?signature=~__CFRunLoopServiceMachPort&build_id=%3E%3D20210504214950&version=90.0a1&platform=Mac%20OS%20X&date=%3E%3D2021-05-13T17%3A01%3A00.000Z&date=%3C2021-05-26T16%3A19%3A00.000Z&_facets=signature&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

Comment 12

[Steven Michaud [:smichaud] (Retired)]

I'm giving up on this bug, for the time being at least.

I've found another crash report, with different mac_crash_info, that might be relevant:

bp-516f791e-e4b4-47c2-acc2-1aab00210603

    {
      "num_records": 1,
      "records": [
        {
          "message": "Performing @selector(menuItemHit:) from sender NSMenuItem 0x1308c9580",
          "module": "/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit"
        }
      ]
    }

But the evidence is still too sparse.

With luck we'll see more mac_crash_info on crashes with this signature. Once more has accumulated, I may come back to this bug.

Comment 13

[Intermittent Failures Robot]

1 failures in 2387 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1610213&startday=2021-07-05&endday=2021-07-11&tree=all

Comment 14

[Jens Stutte [:jstutte]]

I think this is mostly something very similar if not the same as bug 1705365. If you look at the "Xpcom spin event loop stack" annotations of these crashes you will find many different recursive nestings of prompts/Prompter.jsm:openPromptSync.

Comment 15

[Steven Michaud [:smichaud] (Retired)]

If you look at the "Xpcom spin event loop stack" annotations of these crashes you will find many different recursive nestings of prompts/Prompter.jsm:openPromptSync.

Where in the crash reports do you find these? (I looked at a few and didn't see any.) Are they in a searchable field?

Comment 16

[Jens Stutte [:jstutte]]

Lookout for XPCOMSpinEventLoopStack on the Crash Annotations tab of a single crash report like this. You can also search, filter and aggregate this field.

Comment 17

[Jens Stutte [:jstutte]]

(If you look at content processes, be aware of bug 1741131)

Comment 18

[Steven Michaud [:smichaud] (Retired)]

Interesting! And thanks for the info.

Not all crashes with this bug's signature have this annotation. But it's still a large proportion -- a little more than half (among recent crash reports):

https://crash-stats.mozilla.org/search/?signature=~__CFRunLoopServiceMachPort&platform=Mac%20OS%20X&date=%3E%3D2021-11-10T17%3A08%3A00.000Z&date=%3C2021-11-17T17%3A08%3A00.000Z&_facets=signature&_facets=xpcom_spin_event_loop_stack&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-xpcom_spin_event_loop_stack

I assume prompts/Prompter.jsm:openPromptSync displays a modal dialog.

Comment 19

[Steven Michaud [:smichaud] (Retired)]

Note that, in most cases where this annotation is present, there's only one instance of prompts/Prompter.jsm:openPromptSync. I assume this means there are no nested dialogs.

Comment 20

[Jens Stutte [:jstutte]]

(In reply to Steven Michaud [:smichaud] (Retired) from comment #19)

Note that, in most cases where this annotation is present, there's only one instance of prompts/Prompter.jsm:openPromptSync. I assume this means there are no nested dialogs.

Yes, that's how to read it. These might also be cases (or regressions) not fixed by bug 1696397.

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criteria:

• Top 5 desktop browser crashes on Mac on beta
• Top 5 desktop browser crashes on Mac on release (startup)

:ghess, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 22

[Greg Hess]

Crash counts are not increasing, no need to increase severity at this time.

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit auto_nag documentation.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criterion:

• Top 5 desktop browser crashes on Mac on release (startup)

:haik, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 25

[Haik Aftandilian [:haik]]

S2 seems like the correct severity for this bug. It is flagged as a startup crash here, but 97% of the crashes (4717 in number) over the last 3 months are not startup crashes.

Comment 26

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.

Comment 27

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.