1496501 - Shaka player demo MEDIA.VIDEO_ERROR when changing video resolution

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[laszlo.bialis]

Attachment: widevine_shaka.png

[Note]:
- The issue was discovered while testing the Widevine CDM 4.10.1146.0 feature

[Affected versions]:
FF Nightly 64.0a1 Build ID: 20181004100222

[Affected platforms]:
 Win10 x64, MacOS 10.14, Ubuntu 18.04

[Steps to reproduce]:
1. Open Firefox Nightly 
2. Access the following link https://shaka-player-demo.appspot.com
3. From the Asset drop down menu select Sintel 4k (multicodec, Widevine)
4. Click on Load
5. Leave the video to play for a while 
6. From the Info menu change video resolution from the Video+audio track combinations drop down menu

[Expected result]:
- The video should change to the selected resolution 
- The browser window shouldn't display any error as on Chrome.

[Actual result]:
- The following error is shown: "Shaka Error MEDIA.VIDEO_ERROR (3,,NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005) - virtual ipc::IPCResult __cdecl mozilla::gmp::ChromiumCDMParent::RecvDecodeFailed(const uint32_t &): ChromiumCDMParent::RecvDecodeFailed)" 


[Additional notes]:
- The following is displayed in the browser console:
Player error <unavailable> main.js:696:3
shakaDemo.onError_
https://shaka-player-demo.appspot.com/demo/main.js:696:3
shakaDemo.onErrorEvent_
https://shaka-player-demo.appspot.com/demo/main.js:687:3
shaka.util.FakeEventTarget.prototype.dispatchEvent
https://shaka-player-demo.appspot.com/lib/util/fake_event_target.js:117:9
shaka.cast.CastProxy.prototype.playerProxyLocalEvent_
https://shaka-player-demo.appspot.com/lib/cast/cast_proxy.js:572:3
shaka.util.FakeEventTarget.prototype.dispatchEvent
https://shaka-player-demo.appspot.com/lib/util/fake_event_target.js:117:9
shaka.Player.prototype.onError_
https://shaka-player-demo.appspot.com/lib/player.js:3063:3
shaka.Player.prototype.onVideoError_
https://shaka-player-demo.appspot.com/lib/player.js:3120:3
shaka.util.EventManager.Binding_
https://shaka-player-demo.appspot.com/lib/util/event_manager.js:149:3
shaka.util.EventManager.prototype.listen
https://shaka-player-demo.appspot.com/lib/util/event_manager.js:69:17
shaka.Player.prototype.attach
https://shaka-player-demo.appspot.com/lib/player.js:521:3
shaka.Player
https://shaka-player-demo.appspot.com/lib/player.js:186:5
shakaDemo.init/<
https://shaka-player-demo.appspot.com/demo/main.js:190:25
(Async: promise callback) shakaDemo.init
https://shaka-player-demo.appspot.com/demo/main.js:185:33
(Async: EventListener.handleEvent) <anonymous>
https://shaka-player-demo.appspot.com/demo/main.js:751:5
Media resource blob:https://shaka-player-demo.appspot.com/b20e89f7-30b5-443a-b33f-9116cb1a0314 could not be decoded.
Error: This error message will be blank when privacy.resistFingerprinting = true.  If it is really necessary, please add it to the whitelist in MediaError::GetMessage: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005) - virtual ipc::IPCResult __cdecl mozilla::gmp::ChromiumCDMParent::RecvDecodeFailed(const uint32_t &): ChromiumCDMParent::RecvDecodeFailed player.js:3118:7
(video:17) failed fetch and append: code=3015 streaming_engine.js:1555:7
shaka.media.StreamingEngine.prototype.fetchAndAppend_/<
https://shaka-player-demo.appspot.com/lib/media/streaming_engine.js:1555:7

If after the error is shown one clicks on the "Load" button the video is played automatically, but after a while the error is shown again.

Comment 1

[Andreas Pehrson [:pehrsons]]

FWIW I cannot repro this with Widevine 1.4.9.1088 (ubuntu nightly 63 from Aug 14 and mozilla devedition 63.0b11) but I can with Widevine 4.10.1146.0 (mozilla nightly 64 from Oct 4).

A more narrow STR for the error in 64 is to choose the 426x182 resolution and seek to ~20s. When playback hits 25s I consistently get the error.

This URL gets you to the right video and resolution:
https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/shaka-demo-assets/sintel-widevine/dash.mpd;lang=en-US;noadaptation;build=uncompiled

The GMP log for this has some interesting information (this starts with the last successfully decrypted and decoded frame and ends with the error):
> 2018-10-05 10:09:15.055304 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::DecryptAndDecodeFrame t=26042000
> 2018-10-05 10:09:15.055473 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecryptAndDecodeFrame() t=26042000)
> 2018-10-05 10:09:15.055493 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::WidevineVideoFrame() this=0x7fff6539ee90
> 2018-10-05 10:09:15.056151 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetPlaneOffset(0, 0) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056163 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetSize(426,182) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056171 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetFormat(1) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056178 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetPlaneOffset(1, 93184) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056185 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetPlaneOffset(2, 116480) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056192 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetStride(0, 512) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056199 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetStride(1, 256) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056205 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetStride(2, 256) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056219 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::Allocate(capacity=139776) bufferSizes={211507,211507,211507,211507,211507,211507}
> 2018-10-05 10:09:15.056227 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=211507) created
> 2018-10-05 10:09:15.056234 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::SetSize(size=211507)
> 2018-10-05 10:09:15.056241 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetFrameBuffer(0x7f784eb40540) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056269 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::SetTimestamp(26042000) this=0x7fff6539ee90
> 2018-10-05 10:09:15.056277 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecryptAndDecodeFrame() t=26042000 CDM decoder rv=0
> 2018-10-05 10:09:15.056299 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::Destroy(size=139776)
> 2018-10-05 10:09:15.056306 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=139776) destructed writable=0
> 2018-10-05 10:09:15.056366 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::RecvDecodedShmem(this=0x7fd4543e8010) time=26042000 duration=41000
> 2018-10-05 10:09:15.056706 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvGiveBuffer(capacity=211507) bufferSizes={211507,211507,211507,211507,211507,211507} mDecoderInitialized=1
> 2018-10-05 10:09:15.083832 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::SendBufferToCDM() size=350
> 2018-10-05 10:09:15.084024 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvGiveBuffer(capacity=350) bufferSizes={211507,211507,211507,211507,211507,211507,350} mDecoderInitialized=1
> 2018-10-05 10:09:15.084040 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecrypt()
> 2018-10-05 10:09:15.084051 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::Allocate(capacity=350) bufferSizes={211507,211507,211507,211507,211507,211507,350}
> 2018-10-05 10:09:15.084058 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=350) created
> 2018-10-05 10:09:15.084145 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::SetSize(size=350)
> 2018-10-05 10:09:15.084159 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::Destroy(size=350)
> 2018-10-05 10:09:15.084164 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=350) destructed writable=0
> 2018-10-05 10:09:15.084208 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::RecvDecrypted(this=0x7fd4543e8010, id=596, status=0)
> 2018-10-05 10:09:15.084951 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::SendBufferToCDM() size=346
> 2018-10-05 10:09:15.085085 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvGiveBuffer(capacity=346) bufferSizes={211507,211507,211507,211507,211507,211507,346} mDecoderInitialized=1
> 2018-10-05 10:09:15.085093 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecrypt()
> 2018-10-05 10:09:15.085101 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::Allocate(capacity=346) bufferSizes={211507,211507,211507,211507,211507,211507,346}
> 2018-10-05 10:09:15.085107 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=346) created
> 2018-10-05 10:09:15.085133 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::SetSize(size=346)
> 2018-10-05 10:09:15.085143 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer::Destroy(size=346)
> 2018-10-05 10:09:15.085147 UTC - [GMP 16156: Main Thread]: D/GMP CDMShmemBuffer(size=346) destructed writable=0
> 2018-10-05 10:09:15.085190 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::RecvDecrypted(this=0x7fd4543e8010, id=597, status=0)
> 2018-10-05 10:09:15.096958 UTC - [Child 15847: GMPThread]: D/GMP ChromiumCDMParent::DecryptAndDecodeFrame t=26083000
> 2018-10-05 10:09:15.097334 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecryptAndDecodeFrame() t=26083000)
> 2018-10-05 10:09:15.097347 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::ConvertToCdmEncryptionScheme() got scheme marked as encrypted, but with no cipher bytes! This should be caught earlier, preferably by the demuxer! Returning cdm::EncryptionScheme::kUnencrypted
> 2018-10-05 10:09:15.097357 UTC - [GMP 16156: Main Thread]: D/GMP WidevineVideoFrame::WidevineVideoFrame() this=0x7fff6539ee90
> 2018-10-05 10:09:15.097393 UTC - [GMP 16156: Main Thread]: D/GMP ChromiumCDMChild::RecvDecryptAndDecodeFrame() t=26083000 CDM decoder rv=5
> [Child 15847, MediaPlayback #3] WARNING: Decoder=7fd4543e83f0 Decode error: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005) - virtual ipc::IPCResult mozilla::gmp::ChromiumCDMParent::RecvDecodeFailed(const uint32_t &): ChromiumCDMParent::RecvDecodeFailed: file /builds/worker/workspace/build/src/dom/media/MediaDecoderStateMachine.cpp, line 3429

Alas
> ChromiumCDMChild::ConvertToCdmEncryptionScheme() got scheme marked as encrypted, but with no cipher bytes! This should be caught earlier, preferably by the demuxer! Returning cdm::EncryptionScheme::kUnencrypted

Comment 2

[Andreas Pehrson [:pehrsons]]

Not sure exactly how we're going to ship the new Widevine CDM but this seems like it will regress 64, so I'm increasing to P2. Bryce, could you take a look?

Comment 3

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

This appears to be due to how the new CDM requires us to flag input samples with an encryption flag. Digging further is look like the CDM is even more picky about its input than previously realized.

Background:

Previously when providing the CDM with samples we would give it lists of encrypted and clear bytes and it would figure out if a packet was encrypted or not. We still pass these lists to the CDM, but we also pass an enum to explicitly state the kind of encryption on a sample now too. Passing the wrong enum results in the CDM throwing errors like what we see here.

Bug 1491117 changed how our WebM demuxer handled encrypted samples to accommodate this. In bug 1494178 I added some fallback code to remark samples with no encrypted ranges as unencrypted as well as log the message we're seeing above. It appears now that the bug 1494178 handling is incorrect in some cases.

Issue:

With the new CDM we were encountering issues where we would label unencrypted samples from encrypted tracks as encrypted before we fed them to the CDM (using the new enum field). This would cause an error, so handling was changed to mark clear samples as unencrypted. However, this bug hightlights that there is a further state I had not anticipated: samples that have no encrypted ranges, but that have certain encryption metadata should be labelled as encrypted.

Or in other words, if we have a sample of size N, if that sample has 0 encrypted bytes and N clear bytes, we cannot just label it as unencrypted -- this is what we're doing now, and we get this issue. We must label it encrypted or unencrypted depending on if it has certain encryption metadata present.

I'm more familiar with the WebM case and I appears we should set the encryption enum based on the value of the signal bytes used in WebMs. The demuxer does this, so I think we'd be okay for that case if I remove the fallback handling introduced by bug 1494178.

However, I think the MP4 case needs further examination before we can remove the fallback handling that is causing this. I've been keeping an eye on the cases our mp4 demuxer could produce output that would be rejected by the new CDM, and believe it's possible that we could have issues if the fallback were removed with adjusting the demuxer.

Comment 4

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

When adding the fallback code to catch 0 encrypted byte samples and mark them as encrypted[0], I had been testing using the shaka-player Angel One (HLS, MP4, Multilingual, Widevine) media. I am fairly certain I was seeing issues with the lowest resolution video -- as that stream contains some clear samples in an otherwise encrypted mdat, specifically the 3rd segment[1]. This video is 144p, and it seems like it can't be manually selected, you have to refresh until you get the low res and/or make sure most of your bandwidth is being consumed so you get low res.

However, after removing the above code and testing today, I do not see issue. As such, I wonder if some other combination of work in progress I had was causing the issue. I've looked more into the MP4 demuxer, and we appear to be preparing our data similar to Chromium, which seems like a good precedent to follow for CDM compat.

Chromium does a similar check[2] to our code, where we will mark a sample as encrypted if its the first from an encrypted track[3] or if it has associated crypto information[4].

Given the above, it looks like the fallback code can be removed.

[0]: https://searchfox.org/mozilla-central/rev/29aea2a2a3bd0f5e25ce0b60a76053fb25ba5149/dom/media/gmp/ChromiumCDMChild.cpp#696
[1]: https://storage.googleapis.com/shaka-demo-assets/angel-one-widevine-hls/v-0144p-0100k-libx264-s3.mp4
[2]: https://cs.chromium.org/chromium/src/media/formats/mp4/track_run_iterator.cc?l=785&rcl=d7db601c688d4b4d0fdb492af5331e1e650743a3
[3]: https://searchfox.org/mozilla-central/rev/29aea2a2a3bd0f5e25ce0b60a76053fb25ba5149/dom/media/mp4/Index.cpp#125
[4]: https://searchfox.org/mozilla-central/rev/29aea2a2a3bd0f5e25ce0b60a76053fb25ba5149/dom/media/mp4/Index.cpp#139

Comment 5

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1496501 - Do not mark CDM input as unencrypted even if it has no encrypted bytes. r=cpearce

Bug 1494178 added code to mark samples with 0 encrypted ranges as unencrypted
before they were fed to the CDM. This was to catch issues where we could mark
such unencrypted samples as encrypted. However, the CDM expects certain samples
that are clear to still be marked as encrypted.

Specifically, WebM samples should be marked as encrypted if they are from an
encrypted track and have the signal byte set (a marker for if the packet is
encrypted), even if they have no encrypted ranges.

The WebM demuxer is already doing this. Further inspection and testing of the
mp4 demuxer shows it is behaving in line with Chromium's current mp4 parser,
which we can expect prepares its data sensibly for Widevine.

As the code removed here was added as a safety fallback, but is causing issues,
and as the demuxers already appear to be doing the right thing, the fallback
code can be removed.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cdf1848b6a7
Do not mark CDM input as unencrypted even if it has no encrypted bytes. r=cpearce

Comment 7

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/2cdf1848b6a7

Comment 8

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: WebMDecryptFlowchart.png

Attaching flow chart showing how encryption information is set for WebM samples following the the landing of changes. The case that was problematic is highlighted in the lower right: if a sample is marked for subsample encryption, but no subsamples are encrypted, we should still mark it as encrypted, which we are now doing.

Comment 9

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: WebMDecryptFlowchart.png

And fixing a typo in the last chart.

Comment 10

[laszlo.bialis]

I've checked on the latest Nightly 64.0a1 (2018-10-10) on Win 10 x64, Ubuntu 16.04, MacOs 10.14 and the issue is not reproducible. Marking as Verified/Fixed.

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0e34c6a8832e77846f947d4f4809ae104c48d571

Comment 12

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Comment on attachment 9015327 [details]
Bug 1496501 - Do not mark CDM input as unencrypted even if it has no encrypted bytes. r=cpearce

[ESR Uplift Approval Request]

If this is not a sec:{high,crit} bug, please state case for ESR consideration: Please see Bug 1518525 for details and motivation.

Further notes: this is the (current) final bug in a series tracked by bug 1518525. Please see https://treeherder.mozilla.org/#/jobs?repo=try&revision=b9976fb4de94bf0933f1faa8118154961fac20fb&selectedJob=220243631 for a try push where these patches have been grafted onto esr60.

User impact if declined: Failure to playback premium media.

Fix Landed on Version: 64

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): This code landed and baked since 64. I have manually verified that it works when grafted onto esr60.

String or UUID changes made by this patch: None.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9015327 [details]
Bug 1496501 - Do not mark CDM input as unencrypted even if it has no encrypted bytes. r=cpearce

Needed for ESR60 to continue to support Widevine video playback in the very near future. Approved for 60.5.0esr.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/8f436181b5e8