WebRTC hang during shutdown [@QTCaptureSession::addInput:error]

RESOLVED WORKSFORME

Status

()

Core
WebRTC: Audio/Video
P2
major
Rank:
25
RESOLVED WORKSFORME
5 years ago
2 years ago

People

(Reporter: posidron, Assigned: jesup)

Tracking

(Blocks: 1 bug, {hang})

Trunk
x86_64
Mac OS X
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

5 years ago
Created attachment 740328 [details]
callstack

+++ This bug was initially created as a clone of Bug #864255 +++

This hang is happening during shutdown by randomly closing the Firefox instance while fuzzing JSEP.

Tested with m-i changeset: 129433:a1a2af3cebff
(Assignee)

Comment 1

5 years ago
Thread 38 indicates we're stuck waiting for QTKit ([QTCaptureSession addInput:error:]).  Internally, I believe that sends native events to the main thread (we've run into this before) and can be blocked because the main thread stops handling native events.  (bsmedberg and I dealt with one of these when closing a session; this may be the inverse of that when opening one.)  For added fun, GDB got confused trying to dump all the thread stacks.

Blocking- for now, since we haven't seen it outside of fuzzing, but we might bump that depending on what we find.

Steve - Is this another function that needs to be proxied to the Main thread?  BSmedberg and I had previously proxied a release because in shutdown, the Main Thread stops processing native events before everything is shut down.  See bug 829907 and the comments in MediaEngineWebRTCVideo.cpp
Priority: -- → P1
Whiteboard: [WebRTC] [blocking-webrtc-]
(Assignee)

Comment 2

5 years ago
-> blocking-webrtc? to keep it on our radar until we figure out more
-> needinfo to smichaud
Assignee: nobody → rjesup
Flags: needinfo?(smichaud)
Whiteboard: [WebRTC] [blocking-webrtc-] → [WebRTC] [blocking-webrtc?]
> Steve - Is this another function that needs to be proxied to the Main thread?

I don't know.  And I won't really be able to find out until someone finds good STR for this bug, and I have a day or two to spend on it.

It might help if there are any interesting messages in the system console.  That turned out to be crucial for figuring out bug 837539.
Flags: needinfo?(smichaud)
(In reply to Steven Michaud from comment #3)
> > Steve - Is this another function that needs to be proxied to the Main thread?
> 
> I don't know.  And I won't really be able to find out until someone finds
> good STR for this bug, and I have a day or two to spend on it.
> 
> It might help if there are any interesting messages in the system console. 
> That turned out to be crucial for figuring out bug 837539.

cdiehl, can you help with providing STR and any interesting messages in the system console?
Flags: needinfo?(cdiehl)
(Reporter)

Comment 5

5 years ago
I have added him to the initial bug which contains log of the whole fuzzing session.
Flags: needinfo?(cdiehl)
I take it that means you don't have STR for the bug, or a testcase (reduced or not) that can reliably reproduce it.

If so, that doesn't help very much.  If at all possible, please provide one or both of these.  If you'd prefer, you can do that in bug 864255.

I didn't see any console messages at all in the fuzzing session log ... but I'm not sure I looked in the right place(s).
Flags: needinfo?(cld71)
Flags: needinfo?(cld71) → needinfo?(cdiehl)
(Reporter)

Comment 7

5 years ago
Sorry for the late response but I had trouble with my ISP.

> I didn't see any console messages at all in the fuzzing session log ... but
> I'm not sure I looked in the right place(s).

It's all the in the file: fuzzing-session
Flags: needinfo?(cdiehl)
(Reporter)

Comment 8

5 years ago
Created attachment 741630 [details]
callstack

I have attached a non-minimized testcase which reproduces the hang.

Load the testcase in a debug build of Firefox and randomly close Firefox after a few seconds.

If you are successful the last line you should see on the console is:

WARNING: nsAppShell::Exit() called redundantly: file /Users/cdiehl/dev/repos/mozilla/mozilla-inbound/widget/cocoa/nsAppShell.mm, line 757
Attachment #740328 - Attachment is obsolete: true
(Reporter)

Comment 9

5 years ago
Created attachment 741631 [details]
testcase
(Reporter)

Updated

5 years ago
Keywords: testcase
(Assignee)

Comment 10

5 years ago
Steven - does this testcase work for you?  It may take a number of attempts from what I read
Flags: needinfo?(smichaud)
> If you are successful the last line you should see on the console is:
>
> WARNING: nsAppShell::Exit() called redundantly: file
> /Users/cdiehl/dev/repos/mozilla/mozilla-inbound/widget/cocoa/nsAppShell.mm,
> line 757

I see this every time I quit today's mozilla-central-debug nightly, so
it doesn't have any relation to this bug.

I also can't reproduce anything resembling a hang, testing with either
today's mozilla-central-debug nightly or today's mozilla-central
nightly.  I tried five or six times with each, doing Cmd-q either
just after the testcase loaded or waiting 5, 10 or 15 seconds.
Flags: needinfo?(smichaud)
> If you are successful the last line you should see on the console is:
>
> WARNING: nsAppShell::Exit() called redundantly: file
> /Users/cdiehl/dev/repos/mozilla/mozilla-inbound/widget/cocoa/nsAppShell.mm,
> line 757

And no, this was never the last line of output.
(Assignee)

Comment 13

5 years ago
Cristoph: what OS and webcam (and driver rev) are in use for this test?  Also, just quickly verify this still happens for you.  Can you try it on any other HW/OS combos?
Flags: needinfo?(cdiehl)
(Reporter)

Comment 14

5 years ago
ProductName:	Mac OS X
ProductVersion:	10.8.3
BuildVersion:	12D78

It happens with the default MacBook Pro 2012 edition and with the Apple Thunderbold display.
Flags: needinfo?(cdiehl)
(Reporter)

Comment 15

5 years ago
Still reproducible with m-i changeset: 131012:e6e9cc3a5b10
(Assignee)

Updated

5 years ago
Whiteboard: [WebRTC] [blocking-webrtc?] → [WebRTC][blocking-webrtc-][webrtc-uplift]

Updated

3 years ago
backlog: --- → webRTC+
Rank: 15
Whiteboard: [WebRTC][blocking-webrtc-][webrtc-uplift] → [webrtc-uplift]
(Assignee)

Comment 16

3 years ago
I suspect this is long fixed, but would like some verification - can you try?
Rank: 15 → 25
Flags: needinfo?(jbecerra)
Flags: needinfo?(cdiehl)
Priority: P1 → P2
Whiteboard: [webrtc-uplift]
(Assignee)

Updated

3 years ago
Severity: critical → major
(Reporter)

Comment 17

3 years ago
Okay, I will look at this ASAP but I am also suspecting this is fixed after that time. :-)
Flags: needinfo?(cdiehl)
I haven't been able to load the testcase in comment #9. I have been trying on Nightly and Dev Edition. I also searched crash-stats for this signature [@QTCaptureSession::addInput:error], and I found no results.

Christoph, can you take a look when you get a chance and see if this still happens for you?
Flags: needinfo?(jbecerra) → needinfo?(cdiehl)
Keywords: testcase → testcase-wanted
(Reporter)

Comment 19

3 years ago
Yes, same here. This issue seems gone.
Flags: needinfo?(cdiehl)
(Assignee)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WORKSFORME
Keywords: testcase-wanted
You need to log in before you can comment on or make changes to this bug.