1435670 - Crash in std::__1::__function::__func<T>::operator()

Status: VERIFIED FIXED

(Core :: WebRTC: Audio/Video, defect, P2)

Description

[Andreas Pehrson [:pehrsons]]

This bug was filed from the Socorro interface and is
report bp-75675863-0977-4113-9df0-7d2770180201.
=============================================================

Top 10 frames of crashing thread:

0 XUL std::__1::__function::__func<mozilla::camera::CamerasParent::RecvStartCapture dom/media/systemservices/CamerasParent.cpp:888
1 XUL mozilla::camera::VideoEngine::WithEntry clang/include/c++/v1/functional:1898
2 XUL mozilla::media::LambdaRunnable<mozilla::camera::CamerasParent::RecvStartCapture dom/media/systemservices/CamerasParent.cpp:855
3 XUL nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:1040
4 XUL NS_ProcessNextEvent xpcom/threads/nsThreadUtils.cpp:517
5 XUL mozilla::ipc::MessagePumpForNonMainThreads::Run ipc/glue/MessagePump.cpp:364
6 XUL MessageLoop::Run ipc/chromium/src/base/message_loop.cc:326
7 XUL base::Thread::ThreadMain ipc/chromium/src/base/thread.cc:181
8 XUL ThreadFunc ipc/chromium/src/base/platform_thread_posix.cc:38
9 libsystem_pthread.dylib libsystem_pthread.dylib@0x3aba 

=============================================================


We're hitting one of the diagnostic asserts added in bug 1419374 here. In release we still have code to handle these cases gracefully so it's not terribly bad. It would be good to figure out the root cause however.

Comment 1

[Paul Adenot (:padenot)]

Sorry, wrong bug.

Comment 3

[Andreas Pehrson [:pehrsons]]

See bug 1436299 comment 10 for an analysis.

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1435670 - Remove assert when there's no capability for a device.

Not all devices have capabilities. Our code is already setup to handle that
case by defaulting to a capability with width,height,maxFPS=0 and propagating
the failure to start.

Review commit: https://reviewboard.mozilla.org/r/218712/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/218712/

Comment 5

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Comment on attachment 8949323 [details]
Bug 1435670 - Remove assert when there's no capability for a device.

https://reviewboard.mozilla.org/r/218712/#review224852

Lgtm.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4d8ea2ca180f
Remove assert when there's no capability for a device. r=jib

Comment 7

[Andreas Pehrson [:pehrsons]]

Comment on attachment 8949323 [details]
Bug 1435670 - Remove assert when there's no capability for a device.

Approval Request Comment
[Feature/Bug causing the regression]: bug 1388667
[User impact if declined]: No impact on release (diagnostic asserts are noops) but we could hit them in opt beta and Nightly builds, so it'll look like crashes.
[Is this code covered by automated tests?]: No
[Has the fix been verified in Nightly?]: Not yet
[Needs manual test from QE? If yes, steps to reproduce]: Yes. I can reproduce with an unconfigured v4l2loopback device on linux. :ovidiu can also repro on Mac, see bug 1436299.
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: Not at all.
[Why is the change risky/not risky?]: Removes some diagnostic asserts that only affect debug, beta and Nightly.
[String changes made/needed]: None

Comment 8

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/4d8ea2ca180f

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/4d8ea2ca180f

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8949323 [details]
Bug 1435670 - Remove assert when there's no capability for a device.

Removes some asserts that show up in crash-stats. 
Seems ok for uplift for beta 10.

Comment 11

[Liz Henry (:lizzard) (relman/hg->git project)]

FYI. There is one crash showing up on release on 58.0.  
https://crash-stats.mozilla.com/report/index/b8778602-e830-4e4c-8eea-310700180125

Comment 12

[Andreas Pehrson [:pehrsons]]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #11)
> FYI. There is one crash showing up on release on 58.0.  
> https://crash-stats.mozilla.com/report/index/b8778602-e830-4e4c-8eea-
> 310700180125

Thanks. This one is unrelated. The function we crash in is anonymous and in this case the rest of the stack is different.

Comment 13

[ovidiu boca[:Ovidiu]]

Hi Andreas,

I tested this following the steps from bug 1436299 on Mac OS X 10.12 with Nightly 60.0a1(2018-02-13)and here are the results:

After I plugged back the camera "Microsoft LifeCam HD-3000 (USB)" click on refresh the page and select the same camera to share the tab crashes and if I click on restore tab the hall browser crashes, here is the crash signature: [@ webrtc::videocapturemodule::VideoCaptureImpl::IncomingFrame ] and the crash id: https://crash-stats.mozilla.com/report/index/fca2c409-91d0-4c34-a797-8bbe40180213#tab-metadata 

From what I remember you told me that this is related to a Mac issue, but my question is what should we do with this bug since us marked as fixed?

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-beta/rev/18ee6c43696d

Comment 15

[Andreas Pehrson [:pehrsons]]

(In reply to ovidiu boca[:Ovidiu] from comment #13)
> Hi Andreas,
> 
> I tested this following the steps from bug 1436299 on Mac OS X 10.12 with
> Nightly 60.0a1(2018-02-13)and here are the results:
> 
> After I plugged back the camera "Microsoft LifeCam HD-3000 (USB)" click on
> refresh the page and select the same camera to share the tab crashes and if
> I click on restore tab the hall browser crashes, here is the crash
> signature: [@ webrtc::videocapturemodule::VideoCaptureImpl::IncomingFrame ]
> and the crash id:
> https://crash-stats.mozilla.com/report/index/fca2c409-91d0-4c34-a797-
> 8bbe40180213#tab-metadata 
> 
> From what I remember you told me that this is related to a Mac issue, but my
> question is what should we do with this bug since us marked as fixed?

This bug, the crash mentioned in comment 0, is fixed. The re-plugging issue on mac is bug 1436299.

Comment 16

[Oana Botisan, Desktop QA]

I managed to reproduce the bug only on macOS 10.13 using an older version of Nightly (2018-02-07). After the refresh the browser crashed with the same signature [@ std::__1::__function::__func<T>::operator() ].

On Ubuntu 16.04 x64 I couldn't managed to reproduce it though. 

I verified on macOS 10.13 on beta 59.0b10 and latest Nightly 60.0a1 and the bug is not reproducing. As it was mentioned before, the tab and the browser did crash, but the signature was different. I think that issue is regarding bug 1436299.

Considering these results, can I mark the bug as verified? Or can you please check to see if the bug is still reproducing? 
Thank you.

Comment 17

[Andreas Pehrson [:pehrsons]]

What you're hitting on mac could indeed be bug 1436299.

In comment 7 I mention "with an unconfigured v4l2loopback device" for reproducing on linux. You can get one by installing and loading the v4l2loopback kernel module. Try (wfm on Ubuntu 17.10):
> sudo apt install v4l2loopback-dkms
> sudo dkms install v4l2loopback/0.10.0
> sudo modprobe v4l2loopback

If these all succeed you'll have a new video device to select when reproducing, which on linux should cause this crash.

Comment 18

[Oana Botisan, Desktop QA]

I managed to reproduce the bug on an older version of Nightly (2018-02-05) using Ubuntu 16.04 x64. After I selected the new video device (v4l2loopback kernel module) the browser crashed with this signature:  std::__1::__function::__func<T>::operator().

I retested everything using latest Nightly 60.0a1 and beta 59.0b10 on the same platform and the bug is not reproducing anymore.