1257211 - Intermittent Crash [@ mozilla::OffTheBooksMutex::Lock] with H264 decoding

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Christian Holler (:decoder)]

The attached testcase crashes on mozilla-inbound revision f30fc906416f (build with --enable-optimize --disable-debug --enable-address-sanitizer).

For detailed crash information, see attachment.

To reproduce the issue, you can run the testcase through the "MediaDataDecoder.H264" gtest. Example STR:

1. Change into objdir/dist/bin of your Firefox build
2. Place attached testcase into objdir, keep the name "gizmo.mp4"
3. Run: GTEST_FILTER=MediaDataDecoder.H264 MOZ_RUN_GTEST=1 ./firefox -unittest


Unfortunately the test is intermittent. The attached sample reproduces fairly well for me (at least 1 of 10 runs). Fixing this issue is quite important since it causes a lot of crashes in fuzzing. Marking as fuzzblocker and stopping this type of fuzzing until this crash is fixed.

Comment 1

[Christian Holler (:decoder)]

Attachment: Detailed Crash Information

Comment 2

[Christian Holler (:decoder)]

Attachment: Testcase

Comment 3

[Jean-Yves Avenard [:jya]]

Which version of ffmpeg are you using?

Can you reproduce with the latest? (3.0)

Comment 4

[Jean-Yves Avenard [:jya]]

Seems to me that this bug is within Benchmark class and not in the decoder or demuxer. 

My guess is that it's possible a task is dispatched at the same time the task queue is being shutdown. 

Now I don't know if we simply work around In Benchmark by making mFinished a mutable and testing it before dispatching the task. Or there could be an issue in TaskQueues themselves.

Comment 5

[Christian Holler (:decoder)]

I'm just using what is in mozilla-inbound. I can try a fix if you have one :)

Comment 6

[Jean-Yves Avenard [:jya]]

(In reply to Christian Holler (:decoder) from comment #5)
> I'm just using what is in mozilla-inbound. I can try a fix if you have one :)

we don't ship ffmpeg. So if you are decoding h264, you must have ffmpeg installed somewhere.

Comment 7

[Jean-Yves Avenard [:jya]]

Attachment: MozReview Request: Bug 1257211: Exit early if already shutting down. r?gerald

Review commit: https://reviewboard.mozilla.org/r/40673/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/40673/

Comment 8

[Gerald Squelart (he/him)]

Comment on attachment 8731510 [details]
MozReview Request: Bug 1257211: Exit early if already shutting down. r?gerald

https://reviewboard.mozilla.org/r/40673/#review37183

Comment 9

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c03462e7761d

Comment 10

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/c03462e7761d

Comment 11

[Jean-Yves Avenard [:jya]]

Comment on attachment 8731510 [details]
MozReview Request: Bug 1257211: Exit early if already shutting down. r?gerald

Patch request is for all changeset

Approval Request Comment
[Feature/regressing bug #]: 1256626
[User impact if declined]: We aim to enable VP9 to all machines fast enough to support it. It would also reduce issues seen with h264 hardware decoder.
[Describe test coverage new/current, TreeHerder]: in central for a few weeks.
[Risks and why]: none, prevent crash
[String/UUID change made/needed]: none

Comment 12

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8731510 [details]
MozReview Request: Bug 1257211: Exit early if already shutting down. r?gerald

Media team wants to enable VP9 on faster machines in 47, I have been told this is mostly a staged rollout and quality is acceptable. Aurora47+

Comment 13

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/9985cfb9f7ae

Comment 14

[vihsa]

crash on fennec 49.0.
crash ID: ebda8ad2-b5b3-40af-99b0-f13802161009
crash stats: https://crash-stats.mozilla.com/report/index/ebda8ad2-b5b3-40af-99b0-f13802161009