1409224 - Firefox 57.0b8 Crash Report [@ EMPTY: no crashing thread identified; OK ]

Status: VERIFIED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[xpeng]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Build ID: 20170926190823

Steps to reproduce:

1. use the demo to getusermedia 
https://webrtc.github.io/samples/src/content/devices/input-output/
2.plug out the device in use 



Actual results:

the firefox 57.0b8 crashes:
https://crash-stats.mozilla.com/report/index/4aec11f8-aae5-4d4a-89df-c469c0171016

Build ID 	20171013042429
OS 	OS X 10.12
OS Version 	10.12.6 16G29


Expected results:

everything should be ok.

Comment 1

[Andreas Pehrson [:pehrsons]]

Munro, do you have time to look at this? Sounds like it's related to devicechange.

Comment 2

[Andreas Pehrson [:pehrsons]]

Making this a P2 for now, please update when we know how widespread this is.

Comment 3

[Andreas Pehrson [:pehrsons]]

xpeng, if you could figure out what caused this with mozregression, [1], that would be of great help in nailing this down too. 

Also some more details on your setup would be good:
Did you pull an audio or a video device?
When this device was pulled, did the os have any more active devices of the same time?

[1] http://mozilla.github.io/mozregression/

Comment 4

[Andreas Pehrson [:pehrsons]]

I've learned about similar issues that were fixed in 57 recently, bug 1398356 and bug 1400465. But then your report is for 57b8 which is newer than either of those.

Alex, maybe you can take a look instead? This sounds very similar to the ones you fixed before.

I'm bumping to P1 given how recent the affected build is. But we still need to figure out the scope of this crash, so any additional info on the setup, steps and reproducibility would be useful.

Comment 5

[Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018]

xpeng,

Can you reproduce the issue 100%?
Do you choose external camera & external mic as the video & audio source?
I have no luck to reproduce it on my mbp for 10+ times.

Comment 6

[xpeng]

yes we can reproduce the issue 100% , I reproduce it in mac mini , we find that the issue can not be reproduced if there are two audio devices in mac mini. I guess the mbp can not be reproduced because of the internal audio device.

Comment 7

[Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018]

The symptom is similar to what I found in bug 1384805 despite bug 1384805 was found on Windows.

Comment 8

[Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018]

We can reproduce the issue with Mac mini.
LLDB shows the call stack:

* thread #38: tid = 0x2b0bd1, 0x00007fff976bdf06 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'org.mozilla.cubeb', stop reason = signal SIGABRT
  * frame #0: 0x00007fff976bdf06 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff82b6f4ec libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff96bbe6df libsystem_c.dylib`abort + 129
    frame #3: 0x00007fff96b85dd8 libsystem_c.dylib`__assert_rtn + 321
    frame #4: 0x000000010ad8eef5 XUL`___lldb_unnamed_function158235$$XUL + 965
    frame #5: 0x00007fff923d393d libdispatch.dylib`_dispatch_call_block_and_release + 12
    frame #6: 0x00007fff923c840b libdispatch.dylib`_dispatch_client_callout + 8
    frame #7: 0x00007fff923cd03b libdispatch.dylib`_dispatch_queue_drain + 754
    frame #8: 0x00007fff923d3707 libdispatch.dylib`_dispatch_queue_invoke + 549
    frame #9: 0x00007fff923cbd53 libdispatch.dylib`_dispatch_root_queue_drain + 538
    frame #10: 0x00007fff923cbb00 libdispatch.dylib`_dispatch_worker_thread3 + 91
    frame #11: 0x00007fff82b6c4de libsystem_pthread.dylib`_pthread_wqthread + 1129
    frame #12: 0x00007fff82b6a341 libsystem_pthread.dylib`start_wqthread + 13

Chun-min is checking now.

Comment 9

[C.M.Chang[:chunmin]]

I guess it's cubeb issue. 
Alex, what will happen when there is no available input device when device switching? Will it fire a state-error-notification to AudioCallbackDriver::StateCallback[0]?

[0] https://searchfox.org/mozilla-central/rev/d0448c970093f94bd986a21d3a9e8c366b316eb4/dom/media/GraphDriver.cpp#1044

Comment 10

[Alex Chronopoulos [:achronop]]

If that's the case it should crash in the following assert: http://searchfox.org/mozilla-central/source/media/libcubeb/src/cubeb_audiounit.cpp#654

Can you confirm that?

Comment 11

[C.M.Chang[:chunmin]]

(In reply to Alex Chronopoulos [:achronop] from comment #10)
> If that's the case it should crash in the following assert:
> http://searchfox.org/mozilla-central/source/media/libcubeb/src/
> cubeb_audiounit.cpp#654
> 
> Can you confirm that?
Yes, it crashes there. In that case, we cannot get any default device so `audiounit_set_device_info` fails here[0].

[0] http://searchfox.org/mozilla-central/rev/8a6a6bef7c54425970aa4fb039cc6463a19c0b7f/media/libcubeb/src/cubeb_audiounit.cpp#605

Comment 12

[Nils Ohlmeier [:drno]]

Marking as New as it appears to be reproducible.

Assigning to Alex for now, as P1's need owners. Feel free to re-assign to someone else if needed.

Comment 13

[Alex Chronopoulos [:achronop]]

Fix is pushed for review in cubeb repo: https://github.com/kinetiknz/cubeb/pull/371

Comment 14

[C.M.Chang[:chunmin]]

(In reply to Alex Chronopoulos [:achronop] from comment #13)
> Fix is pushed for review in cubeb repo:
> https://github.com/kinetiknz/cubeb/pull/371
After applying the patch, there is no crash when unplugging the device. However, it crashes in another assertion[0] when I plug the device back. The `flags & DEV_INPUT` is true and others are all false then.

In this case, do we also need to handle `CUBEB_STATE_STOPPED` in `AudioCallbackDriver::StateCallback`[1] since the state callback fired by `audiounit_reinit_stream` will return `CUBEB_STATE_STOPPED`? 

[0] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/media/libcubeb/src/cubeb_audiounit.cpp#628
[1] https://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/dom/media/GraphDriver.cpp#1048

Comment 15

[Alex Chronopoulos [:achronop]]

> After applying the patch, there is no crash when unplugging the device.
> However, it crashes in another assertion[0] when I plug the device back. The
> `flags & DEV_INPUT` is true and others are all false then.
Nice catch, thanks for checking. The problem is that "default device change" event is still registered. When you plug back in we try to reinit a non working stream. Since there is no initiated AU we hit the assert. I will take care.

> In this case, do we also need to handle `CUBEB_STATE_STOPPED` in
> `AudioCallbackDriver::StateCallback`[1] since the state callback fired by
> `audiounit_reinit_stream` will return `CUBEB_STATE_STOPPED`? 
We do fire a CUBEB_STATE_STOPPED in the fail of reinit but do we need a special handling for that? The existing is not enough?

Comment 16

[C.M.Chang[:chunmin]]

(In reply to Alex Chronopoulos [:achronop] from comment #15)
> > After applying the patch, there is no crash when unplugging the device.
> > However, it crashes in another assertion[0] when I plug the device back. The
> > `flags & DEV_INPUT` is true and others are all false then.
> Nice catch, thanks for checking. The problem is that "default device change"
> event is still registered. When you plug back in we try to reinit a non
> working stream. Since there is no initiated AU we hit the assert. I will
> take care.
I could help you to check when the updated patch is available!

> > In this case, do we also need to handle `CUBEB_STATE_STOPPED` in
> > `AudioCallbackDriver::StateCallback`[1] since the state callback fired by
> > `audiounit_reinit_stream` will return `CUBEB_STATE_STOPPED`? 
> We do fire a CUBEB_STATE_STOPPED in the fail of reinit but do we need a
> special handling for that? The existing is not enough?

`AudioCallbackDriver::StateCallback` only handle `CUBEB_STATE_ERROR`[0] now while we will return `CUBEB_STATE_STOPPED`[1] in this case. I ask this because the behavior here is different from what I've learnt on Windows. On Windows, we will fire `CUBEB_STATE_ERROR` to `AudioCallbackDriver::StateCallback` when we fail to switch device, and fall back to use `SystemClockDriver`. If we don't align with this policy, MediaStreamGraph will stop working and it cannot be closed when Firefox is terminated.

[0] https://searchfox.org/mozilla-central/rev/d0448c970093f94bd986a21d3a9e8c366b316eb4/dom/media/GraphDriver.cpp#1048
[1] https://github.com/kinetiknz/cubeb/blob/a0f5dc6286f95db139774f199db1bb0c738fc71e/src/cubeb_audiounit.cpp#L784
[2] http://searchfox.org/mozilla-central/rev/8a6a6bef7c54425970aa4fb039cc6463a19c0b7f/media/libcubeb/src/cubeb_wasapi.cpp#947,987

Comment 17

[Alex Chronopoulos [:achronop]]

Paul can you check the comment above, about `AudioCallbackDriver::StateCallback`. Can you tell why in windows we return a CUBEB_STATE_ERROR. In any case it's very easy to change that in audiounit and make it return an error state instead of stop in order to align the behaviors.

Comment 18

[Paul Adenot (:padenot)]

All backends should work in the same way (that's the whole point of having cubeb :-)). If you can't reconfigure, it's an error, CUBEB_STATE_ERROR should be reported indeed.

Comment 20

[Alex Chronopoulos [:achronop]]

This is solved by cubeb import on Bug 1411866.

Comment 21

[Simon Basca [:SimonB]]

I was unable to reproduce this issue following the STR with Firefox 57.0b7 on two different mini Mac both running Sierra 10.12.6. Tried unplugging my external webcam and plugging back in several time on https://webrtc.github.io/samples/src/content/devices/input-output/ but Firefox didn't crash. Also note that there were no other audio devices connected to the mini Mac.

Xpeng, could you please check if this is fixed in 57 and 58 since you were able to reproduce this constantly?

Comment 22

[Simon Basca [:SimonB]]

Correcting typo in Comment 21: Tested with Firefox 57.0b8.

Comment 23

[xpeng]

I can not reproduce the issue in FF 57 release.

Comment 24

[Simon Basca [:SimonB]]

Thanks Xpeng! Based on comment 23 this can be closed as verified fixed.