1533129 - Youtube TV - No option to play 4K video on FFTV + GV 67

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[Simon Chae]

Related FFTV ticket: https://github.com/mozilla-mobile/firefox-tv/issues/1910

STR:

• Check out FFTV repo
• Select geckoDebug variant
• Go to Youtube TV and select a 4K video
• Go to context menu for choosing resolution

Expected:

• Show 4k resolution option

Actual:

• Show HD (1080p) as max supported resolution

** You need a 4K display + 4K supported fire-tv device (4k pendant, cube, etc) **

Stacktrace:
2019-03-06 11:21:55.841 3134-3163/? I/OmxVideoDecoder: OMX_IndexParamVideoHWComposed set m4kOsd=1
2019-03-06 11:21:55.853 3134-3649/? D/OmxVideoDecoder: 4K file codec mm set tvp_Set changed to 2
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.DecoderErrorFatal = 1
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.ASAP-P2Ms = 30
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.ASAP-P3Ms = -161055
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.FirstFrameReadyMs = 0```

Gecko nightly version: 67.0.20190304093752

Comment 1

[Chris Peterson [:cpeterson]]

@ Nils, Simon says the max supported YouTube resolution in FFTV+GV is 1080p.

@ Simon, just to be sure: does 4K work in FFTV+WebView?

Comment 2

[Simon Chae]

@ Chris, yes. Not being able to play 4K is considered a release blocker on FFTV

Comment 3

[Nils Ohlmeier [:drno]]

This looks to me like a highly likely side effect of bug 1533252. Theory is that YT is querying all the different resolutions, but if MediaCapabilities fails early on YT thinks 4K is not possible and therefore doesn't offer it

Comment 4

[Nils Ohlmeier [:drno]]

To verify if this is caused by MediaCapabilities we would need to run with media.media-capabilities.enabled set to false.

Comment 5

[Chris Peterson [:cpeterson]]

Simon, can you please test the 4K menu on FFTV with the about:config pref media.media-capabilities.enabled = false?

Also, does 4K work in FFTV+WebView?

Comment 6

[Jean-Yves Avenard [:jya]]

(In reply to Nils Ohlmeier [:drno] from comment #3)

This looks to me like a highly likely side effect of bug 1533252. Theory is that YT is querying all the different resolutions, but if MediaCapabilities fails early on YT thinks 4K is not possible and therefore doesn't offer it

MediaCapabilities giving an invalid result means we will say "it's not smooth" or "it's not power efficient". Based on that YouTube will not by default automatically select those resolutions.
But those resolutions would still be offered to the user to manually select them.

Media Capabilities and bug 1533252 won't prevent 4K to be offered to the user.

The issue would be elsewhere.

Comment 7

[Simon Chae]

(In reply to Chris Peterson [:cpeterson] from comment #5)

Simon, can you please test the 4K menu on FFTV with the about:config pref media.media-capabilities.enabled = false?

Also, does 4K work in FFTV+WebView?

Yes 4k works in FFTV+WebView

Comment 8

[Chris Peterson [:cpeterson]]

Simon, can you please confirm whether 4K video works with on FFTV with a recent GV 68 Nightly builds? MediaCapabilities bug 1533252 has been fixed, which is expected to fix this problem.

Comment 9

[Chris Pearce [:cpearce (Not reading bugmail)]]

I've tested 4K video on the FireStick 4K in a custom src=url page, and it's technically playable, but since either the FireOS update to 6.2.6.3(NS6263/2349) this morning, or since I updated m-c this morning, I'm seeing black frames injected every 30 frames or so, and I often see playback stalls where the decoder pipeline appears to error.

Independent of this, I looked at the issue of 4K being offered to GV-example on youtube.com/tv, and as already reported elsewhere by others I see that youtube.com/tv is querying our MediaCapabilities API and we're returning that we're supported, smooth, and power efficient for all configs tested. I tried spoofing our UA to Silk's, and we still don't get offered 4K (Silk does).

So I'm guessing that Silk is spoofing its UA to something else when it's accessing YouTube.com/tv, I'm haven't been able to figure out what it's spoofing its UA to yet.

I've emailed YouTube to ask if they can rely on MediaCapbilities when it's present in preference to UA sniffing, since our UA strings on GV/mobile products are so heterogeneous.

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #9)

I've tested 4K video on the FireStick 4K in a custom src=url page, and it's technically playable, but since either the FireOS update to 6.2.6.3(NS6263/2349) this morning, or since I updated m-c this morning, I'm seeing black frames injected every 30 frames or so, and I often see playback stalls where the decoder pipeline appears to error.

Should I file a new bug for this black frame issue? Or can the existing "Dropped frames on youtube.com/tv" bug 1529486 cover this new issue, too?

Comment 11

[Chris Pearce [:cpearce (Not reading bugmail)]]

So YT responded that they're using YT specific extensions to MediaSource.isTypeSupported to detect whether 4K is available.

To detect whether 4K is available, they make requests like:

IsTypeSupported(aType=video/webm; codecs="vp9"; width=3840; height=2160; bitrate=2000000)

They also make requests with ridiculously high values (height=9999 or bitrate=20000000) to detect whether the client is actually considering the details of the request.

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

So to fix this bug, we'll need to parse the YT specific extensions, and ensure we reject the spurious requests. We can probably convert their extended string into a MediaCapabilities query.

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Chris Peterson [:cpeterson] from comment #10)

Should I file a new bug for this black frame issue? Or can the existing "Dropped frames on youtube.com/tv" bug 1529486 cover this new issue, too?

I filed bug 1542662 for this, since it may be a new regression.

Comment 13

[Nils Ohlmeier [:drno]]

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

Chris can you please link that new bug here?

Comment 14

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Nils Ohlmeier [:drno] from comment #13)

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

Chris can you please link that new bug here?

Bug 1542956.

Comment 15

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Bug 1533129 - Add extended MIME parameter support to MediaSource.isTypeSupported on YouTube only. r?jya

YouTube.com/tv uses YouTube specific extensions to MediaSource.isTypeSupported
in order to determine whether it serves 4K. It checks with bogus values, and if
we reject the bogus values, it assumes we're responding truthfully to the other
queries. So add support to reject the bogus values on YouTube.com.

With this patch, we can play 4K on YouTube.com/tv.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e80a847a8b9b
Add extended MIME parameter support to MediaSource.isTypeSupported on YouTube only. r=jya

Comment 17

[Stefan Hindli [:stefan_hindli]]

Backed out changeset e80a847a8b9b (Bug 1533129) for causing perma mda failures in dom/media/mediasource/test/test_isTypeSupportedExtensions.html

failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=239252210

log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=239252210&repo=autoland&lineNumber=3088

02:56:22 INFO - 39 INFO TEST-START | dom/media/mediasource/test/test_SetModeThrows.html
02:56:22 INFO - 40 INFO TEST-OK | dom/media/mediasource/test/test_SetModeThrows.html | took 81ms
02:56:22 INFO - 41 INFO TEST-START | dom/media/mediasource/test/test_SplitAppend.html
02:56:22 INFO - 42 INFO TEST-OK | dom/media/mediasource/test/test_SplitAppend.html | took 4312ms
02:56:22 INFO - 43 INFO TEST-START | dom/media/mediasource/test/test_SplitAppendDelay.html
02:56:32 INFO - 44 INFO TEST-OK | dom/media/mediasource/test/test_SplitAppendDelay.html | took 5308ms
02:56:32 INFO - 45 INFO TEST-START | dom/media/mediasource/test/test_TruncatedDuration.html
02:56:32 INFO - 46 INFO TEST-OK | dom/media/mediasource/test/test_TruncatedDuration.html | took 252ms
02:56:32 INFO - 47 INFO TEST-START | dom/media/mediasource/test/test_WebMTagsBeforeCluster.html
02:56:32 INFO - 48 INFO TEST-OK | dom/media/mediasource/test/test_WebMTagsBeforeCluster.html | took 201ms
02:56:32 INFO - 49 INFO TEST-START | dom/media/mediasource/test/test_isTypeSupportedExtensions.html
03:01:53 INFO - 50 INFO TEST-UNEXPECTED-FAIL | dom/media/mediasource/test/test_isTypeSupportedExtensions.html | Test timed out.
03:01:53 INFO - SimpleTest.ok@SimpleTest/SimpleTest.js:275:18
03:01:53 INFO - reportError@SimpleTest/TestRunner.js:121:22
03:01:53 INFO - TestRunner._checkForHangs@SimpleTest/TestRunner.js:142:7
03:02:03 INFO - 51 INFO TEST-OK | dom/media/mediasource/test/test_isTypeSupportedExtensions.html | took 326411ms
03:02:03 INFO - 52 INFO TEST-START | Shutdown
03:02:03 INFO - 53 INFO Passed: 253
03:02:03 WARNING - 54 INFO Failed: 1
03:02:03 WARNING - One or more unittests failed.

Comment 18

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/52aaf00b3506
Backed out changeset e80a847a8b9b for causing perma mda failures in dom/media/mediasource/test/test_isTypeSupportedExtensions.html

Comment 19

[Chris Peterson [:cpeterson]]

Adding [gvtv:p1] whiteboard tag because this 4K bug should block a GVTV MVP.

Is this fix relevant for non-TV Android devices? Should we uplift this fix to Fennec 67 Beta?

Comment 20

[Jean-Yves Avenard [:jya]]

I don't think so, on Android devices clicking a YT link typically opens the YT app.

Comment 21

[Chris Peterson [:cpeterson]]

(In reply to Jean-Yves Avenard [:jya] from comment #20)

I don't think so, on Android devices clicking a YT link typically opens the YT app.

SGTM. Setting 67=wontfix.

Comment 22

[Chris Pearce [:cpearce (Not reading bugmail)]]

It seems other origins served via https doesn't work on android, also a problem in bug 1335740.

I can make the fake YouTube origin in the test http instead.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2f63a0a80a2d5c4ee48d263ad47760ed5b867a5c

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43b78a85f326
Add extended MIME parameter support to MediaSource.isTypeSupported on YouTube only. r=jya

Comment 24

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/43b78a85f326

Comment 25

[Intermittent Failures Robot]

5 failures in 3025 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533129&startday=2019-04-08&endday=2019-04-14&tree=all