1417797 - UAF in H264 decoder shutdown in VCMDecodedFrameCallback::Decoded()

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[Randell Jesup [:jesup] (needinfo me)]

Attachment: gmp.asan

During shutdown of a PeerConnection (in this case navigating away from http://mixer.com/channelone to see other channels), it's possible that the "external" H264 decoder is released and destroyed while a frame decode is still pending from the GMP decoder, which is asynchronous.

Fairly certain this affects all current versions.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

Managed to repro the failure (again by leaving a channel using the << in the upper-left)

Comment 2

[Dan Minor [:dminor]]

I'll have a look.

Comment 3

[Nils Ohlmeier [:drno]]

sec-high should be P1

Comment 4

[Dan Minor [:dminor]]

Attachment: Stop decode callback in WebrtcGmpVideoDecoder::ReleaseGmp

I was able to reproduce the bug several times in an ASAN build but never with this patch applied.

The locking in RegisterDecodeCompleteCallback and Decoded should ensure that callbacks stop prior to GMP shutdown. mCallback is already nullchecked prior to use, so this should be safe.

Comment 5

[Dan Minor [:dminor]]

Comment on attachment 8930156 [details] [diff] [review]
Stop decode callback in WebrtcGmpVideoDecoder::ReleaseGmp

[Security approval request comment]
How easily could an exploit be constructed based on the patch?
You could infer from the patch we were getting a late callback during shutdown, but I don't think it would be that easy to exploit because it is timing dependent.
Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?
No.
Which older supported branches are affected by this flaw?
All.
If not all supported branches, which bug introduced the flaw?

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be?
Easy to create.
How likely is this patch to cause regressions; how much testing does it need?
Unlikely, this should be a safe change given that we already allow mCallback to be nullptr.

Comment 6

[Al Billings [:abillings - ex-MoCo]]

sec-approval+ for checkin on November 28. We'll want branch patches nominated at this time as well.

Comment 7

[Dan Minor [:dminor]]

Comment on attachment 8930156 [details] [diff] [review]
Stop decode callback in WebrtcGmpVideoDecoder::ReleaseGmp

Approval Request Comment
[Feature/Bug causing the regression]:
[User impact if declined]:
Crashes / security problems.
[Is this code covered by automated tests?]:
This is a shutdown bug so it is covered by the existing tests.
[Has the fix been verified in Nightly?]:
No.
[Needs manual test from QE? If yes, steps to reproduce]: 
No.
[List of other uplifts needed for the feature/fix]:
None.
[Is the change risky?]:
No.
[Why is the change risky/not risky?]:
Locking ensures that all callbacks will end before we can set the callback to null and we already allowed a null callback in the existing code.
[String changes made/needed]:
None.

Comment 8

[Dan Minor [:dminor]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d0eb2fe1a7389d40d1abdccbeeaef9c4b30ae7b1

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/d0eb2fe1a738

Comment 10

[Ryan VanderMeulen [:RyanVM]]

This grafts cleanly to ESR52 also. Is it ready for an uplift approval there too or does it need more rebasing?

Comment 11

[Dan Minor [:dminor]]

Comment on attachment 8930156 [details] [diff] [review]
Stop decode callback in WebrtcGmpVideoDecoder::ReleaseGmp

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined:
Crashes / security problems. 
Fix Landed on Version:
59
Risk to taking this patch (and alternatives if risky):
Low risk. This grafts cleanly and I ran manual tests on ESR52 yesterday.
String or UUID changes made by this patch: 
None.

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.

Comment 12

[Gerry Chang [:gchang]]

Comment on attachment 8930156 [details] [diff] [review]
Stop decode callback in WebrtcGmpVideoDecoder::ReleaseGmp

Fix a sec-high. Beta58+ & ESR52+.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/fbcc274fd508

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr52/rev/457d023c167e