Closed Bug 1533129 Opened 1 year ago Closed 1 year ago

Youtube TV - No option to play 4K video on FFTV + GV 67

Categories

(Core :: Audio/Video: Playback, defect, P1)

67 Branch
Unspecified
Android
defect

Tracking

()

RESOLVED FIXED
mozilla68
Tracking Status
firefox65 --- wontfix
firefox66 --- wontfix
firefox67 --- wontfix
firefox68 --- fixed

People

(Reporter: schae, Assigned: cpearce)

References

Details

(Whiteboard: [gvtv:p1] [geckoview:p2] [media-q2] [media-triaged])

Attachments

(1 file)

Related FFTV ticket: https://github.com/mozilla-mobile/firefox-tv/issues/1910

STR:

  • Check out FFTV repo
  • Select geckoDebug variant
  • Go to Youtube TV and select a 4K video
  • Go to context menu for choosing resolution

Expected:

  • Show 4k resolution option

Actual:

  • Show HD (1080p) as max supported resolution

** You need a 4K display + 4K supported fire-tv device (4k pendant, cube, etc) **

Stacktrace:
2019-03-06 11:21:55.841 3134-3163/? I/OmxVideoDecoder: OMX_IndexParamVideoHWComposed set m4kOsd=1
2019-03-06 11:21:55.853 3134-3649/? D/OmxVideoDecoder: 4K file codec mm set tvp_Set changed to 2
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.DecoderErrorFatal = 1
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.ASAP-P2Ms = 30
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.ASAP-P3Ms = -161055
2019-03-06 11:21:55.931 7239-7290/org.mozilla.tv.firefox.gecko.debug:media I/MediaCodecLogger: org.mozilla.tv.firefox.gecko.debug.4K.HW.video.vp9.FirstFrameReadyMs = 0```

Gecko nightly version: 67.0.20190304093752

@ Nils, Simon says the max supported YouTube resolution in FFTV+GV is 1080p.

@ Simon, just to be sure: does 4K work in FFTV+WebView?

Component: General → Audio/Video: Playback
Flags: needinfo?(schae)
Flags: needinfo?(drno)
OS: All → Android
Product: GeckoView → Core
Whiteboard: [gvtv]
Version: unspecified → 67 Branch

@ Chris, yes. Not being able to play 4K is considered a release blocker on FFTV

Flags: needinfo?(schae)
Whiteboard: [gvtv] → [gvtv:p2] [geckoview:p2]

This looks to me like a highly likely side effect of bug 1533252. Theory is that YT is querying all the different resolutions, but if MediaCapabilities fails early on YT thinks 4K is not possible and therefore doesn't offer it

Depends on: 1533252
Flags: needinfo?(drno)

To verify if this is caused by MediaCapabilities we would need to run with media.media-capabilities.enabled set to false.

Simon, can you please test the 4K menu on FFTV with the about:config pref media.media-capabilities.enabled = false?

Also, does 4K work in FFTV+WebView?

Flags: needinfo?(schae)

(In reply to Nils Ohlmeier [:drno] from comment #3)

This looks to me like a highly likely side effect of bug 1533252. Theory is that YT is querying all the different resolutions, but if MediaCapabilities fails early on YT thinks 4K is not possible and therefore doesn't offer it

MediaCapabilities giving an invalid result means we will say "it's not smooth" or "it's not power efficient". Based on that YouTube will not by default automatically select those resolutions.
But those resolutions would still be offered to the user to manually select them.

Media Capabilities and bug 1533252 won't prevent 4K to be offered to the user.

The issue would be elsewhere.

(In reply to Chris Peterson [:cpeterson] from comment #5)

Simon, can you please test the 4K menu on FFTV with the about:config pref media.media-capabilities.enabled = false?

Also, does 4K work in FFTV+WebView?

Yes 4k works in FFTV+WebView

Flags: needinfo?(schae)
Priority: -- → P2
Whiteboard: [gvtv:p2] [geckoview:p2] → [gvtv:p2] [geckoview:p2] [media-q2]

Simon, can you please confirm whether 4K video works with on FFTV with a recent GV 68 Nightly builds? MediaCapabilities bug 1533252 has been fixed, which is expected to fix this problem.

Flags: needinfo?(schae)

I've tested 4K video on the FireStick 4K in a custom src=url page, and it's technically playable, but since either the FireOS update to 6.2.6.3(NS6263/2349) this morning, or since I updated m-c this morning, I'm seeing black frames injected every 30 frames or so, and I often see playback stalls where the decoder pipeline appears to error.

Independent of this, I looked at the issue of 4K being offered to GV-example on youtube.com/tv, and as already reported elsewhere by others I see that youtube.com/tv is querying our MediaCapabilities API and we're returning that we're supported, smooth, and power efficient for all configs tested. I tried spoofing our UA to Silk's, and we still don't get offered 4K (Silk does).

So I'm guessing that Silk is spoofing its UA to something else when it's accessing YouTube.com/tv, I'm haven't been able to figure out what it's spoofing its UA to yet.

I've emailed YouTube to ask if they can rely on MediaCapbilities when it's present in preference to UA sniffing, since our UA strings on GV/mobile products are so heterogeneous.

Flags: needinfo?(schae)

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #9)

I've tested 4K video on the FireStick 4K in a custom src=url page, and it's technically playable, but since either the FireOS update to 6.2.6.3(NS6263/2349) this morning, or since I updated m-c this morning, I'm seeing black frames injected every 30 frames or so, and I often see playback stalls where the decoder pipeline appears to error.

Should I file a new bug for this black frame issue? Or can the existing "Dropped frames on youtube.com/tv" bug 1529486 cover this new issue, too?

So YT responded that they're using YT specific extensions to MediaSource.isTypeSupported to detect whether 4K is available.

To detect whether 4K is available, they make requests like:

IsTypeSupported(aType=video/webm; codecs="vp9"; width=3840; height=2160; bitrate=2000000)

They also make requests with ridiculously high values (height=9999 or bitrate=20000000) to detect whether the client is actually considering the details of the request.

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

So to fix this bug, we'll need to parse the YT specific extensions, and ensure we reject the spurious requests. We can probably convert their extended string into a MediaCapabilities query.

(In reply to Chris Peterson [:cpeterson] from comment #10)

Should I file a new bug for this black frame issue? Or can the existing "Dropped frames on youtube.com/tv" bug 1529486 cover this new issue, too?

I filed bug 1542662 for this, since it may be a new regression.

Flags: needinfo?(cpearce)
Assignee: nobody → cpearce
Priority: P2 → P1
Summary: No option to play 4K video on FFTV + GV 67 → Youtube TV - No option to play 4K video on FFTV + GV 67
Whiteboard: [gvtv:p2] [geckoview:p2] [media-q2] → [gvtv:p2] [geckoview:p2] [media-q2] [media-triaged]

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

Chris can you please link that new bug here?

Flags: needinfo?(cpearce)

(In reply to Nils Ohlmeier [:drno] from comment #13)

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #11)

If I patch isTypeSupported() to return false for the spurious requests, I get offered 4K. Playback fails hilariously for other reasons, but I'll file a bug about that.

Chris can you please link that new bug here?

Bug 1542956.

Flags: needinfo?(cpearce)

YouTube.com/tv uses YouTube specific extensions to MediaSource.isTypeSupported
in order to determine whether it serves 4K. It checks with bogus values, and if
we reject the bogus values, it assumes we're responding truthfully to the other
queries. So add support to reject the bogus values on YouTube.com.

With this patch, we can play 4K on YouTube.com/tv.

Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e80a847a8b9b
Add extended MIME parameter support to MediaSource.isTypeSupported on YouTube only. r=jya

Backed out changeset e80a847a8b9b (Bug 1533129) for causing perma mda failures in dom/media/mediasource/test/test_isTypeSupportedExtensions.html

failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=239252210

log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=239252210&repo=autoland&lineNumber=3088

02:56:22 INFO - 39 INFO TEST-START | dom/media/mediasource/test/test_SetModeThrows.html
02:56:22 INFO - 40 INFO TEST-OK | dom/media/mediasource/test/test_SetModeThrows.html | took 81ms
02:56:22 INFO - 41 INFO TEST-START | dom/media/mediasource/test/test_SplitAppend.html
02:56:22 INFO - 42 INFO TEST-OK | dom/media/mediasource/test/test_SplitAppend.html | took 4312ms
02:56:22 INFO - 43 INFO TEST-START | dom/media/mediasource/test/test_SplitAppendDelay.html
02:56:32 INFO - 44 INFO TEST-OK | dom/media/mediasource/test/test_SplitAppendDelay.html | took 5308ms
02:56:32 INFO - 45 INFO TEST-START | dom/media/mediasource/test/test_TruncatedDuration.html
02:56:32 INFO - 46 INFO TEST-OK | dom/media/mediasource/test/test_TruncatedDuration.html | took 252ms
02:56:32 INFO - 47 INFO TEST-START | dom/media/mediasource/test/test_WebMTagsBeforeCluster.html
02:56:32 INFO - 48 INFO TEST-OK | dom/media/mediasource/test/test_WebMTagsBeforeCluster.html | took 201ms
02:56:32 INFO - 49 INFO TEST-START | dom/media/mediasource/test/test_isTypeSupportedExtensions.html
03:01:53 INFO - 50 INFO TEST-UNEXPECTED-FAIL | dom/media/mediasource/test/test_isTypeSupportedExtensions.html | Test timed out.
03:01:53 INFO - SimpleTest.ok@SimpleTest/SimpleTest.js:275:18
03:01:53 INFO - reportError@SimpleTest/TestRunner.js:121:22
03:01:53 INFO - TestRunner._checkForHangs@SimpleTest/TestRunner.js:142:7
03:02:03 INFO - 51 INFO TEST-OK | dom/media/mediasource/test/test_isTypeSupportedExtensions.html | took 326411ms
03:02:03 INFO - 52 INFO TEST-START | Shutdown
03:02:03 INFO - 53 INFO Passed: 253
03:02:03 WARNING - 54 INFO Failed: 1
03:02:03 WARNING - One or more unittests failed.

Flags: needinfo?(cpearce)
Backout by shindli@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/52aaf00b3506
Backed out changeset e80a847a8b9b for causing perma mda failures in dom/media/mediasource/test/test_isTypeSupportedExtensions.html

Adding [gvtv:p1] whiteboard tag because this 4K bug should block a GVTV MVP.

Is this fix relevant for non-TV Android devices? Should we uplift this fix to Fennec 67 Beta?

Whiteboard: [gvtv:p2] [geckoview:p2] [media-q2] [media-triaged] → [gvtv:p1] [geckoview:p2] [media-q2] [media-triaged]

I don't think so, on Android devices clicking a YT link typically opens the YT app.

(In reply to Jean-Yves Avenard [:jya] from comment #20)

I don't think so, on Android devices clicking a YT link typically opens the YT app.

SGTM. Setting 67=wontfix.

It seems other origins served via https doesn't work on android, also a problem in bug 1335740.

I can make the fake YouTube origin in the test http instead.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2f63a0a80a2d5c4ee48d263ad47760ed5b867a5c

Flags: needinfo?(cpearce)
Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/43b78a85f326
Add extended MIME parameter support to MediaSource.isTypeSupported on YouTube only. r=jya
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Regressions: 1580695
You need to log in before you can comment on or make changes to this bug.