Closed Bug 1628407 Opened 4 years ago Closed 4 years ago

fix hardware acceleration for videos in flatpaks

Categories

(Release Engineering :: Release Automation: Other, defect)

Product:
Release Engineering
Component:
Release Automation: Other
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox75 wontfix, firefox76 wontfix, firefox77 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox75 --- wontfix
firefox76 --- wontfix
firefox77 --- fixed

People

(Reporter: mtabara, Assigned: mtabara)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

Bug 1628407 - Backed out changeset 91d0340d9856. r=jcristau
47 bytes, text/x-phabricator-request
Details | Review

h264 acceleration for YouTube and other websites is still broken. We should fix that.

H264 hardware decoding is bug 1622425.
H264 software decoding seems to use slow OpenH264: bug 1628428

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #1)

H264 hardware decoding is bug 1622425.
H264 software decoding seems to use slow OpenH264: bug 1628428

*** This bug has been marked as a duplicate of bug 1622425 ***

Agreed. However, until we sort that openh264 out properly, my understanding is that we can enable ffmpeg-full as ffmpeg just ships its own openh264. We can re-work this once we have a proper solution in bug 1622425. What do you think?

Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Flags: needinfo?(jan)
Assignee: nobody → mtabara
Blocks: flatpak
See Also: → 1628428

Thanks! How could this be tested after it merged? Does Taskcluster output a some flatpak artifact (or when would the beta repo receive this patch)?

Flags: needinfo?(jan)

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #4)

Thanks! How could this be tested after it merged? Does Taskcluster output a some flatpak artifact (or when would the beta repo receive this patch)?

I've been testing locally these patches on my machine. Normally the patch rides the trains, first to central (for nightlies where unfortunately we don't yet have Flatpak support), then to beta (where we have Flatpak beta that goes to the beta channel in Flathub) and then to release (which publishes to stable channel - https://flathub.org/apps/details/org.mozilla.firefox).

I'll land this patch now to the integration branch and then request to be directly uplifted to beta so that next week's beta can have it.
Should you want to test earlier, I'm happy to regenerate this locally and send it over to you via Firefox Send or so.

Comment on attachment 9140380 [details]
Bug 1628407 - enable ffmpeg full extension in flatpaks.r=rail

Beta/Release Uplift Approval Request

  • User impact if declined: Flatpak fix for video acceleration.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce: Flatpak fix for video acceleration.
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Flatpak fix for video acceleration.
  • String changes made/needed:
Attachment #9140380 - Flags: approval-mozilla-beta?

Comment on attachment 9140380 [details]
Bug 1628407 - enable ffmpeg full extension in flatpaks.r=rail

Beta/Release Uplift Approval Request

  • User impact if declined: Flatpak fix for video acceleration.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce: Flatpak fix for video acceleration.
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Flatpak fix for video acceleration.
  • String changes made/needed:
Attachment #9140380 - Flags: approval-mozilla-release?
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/91d0340d9856
enable ffmpeg full extension in flatpaks.r=rail

https://hg.mozilla.org/mozilla-central/rev/91d0340d9856

Status: REOPENED → RESOLVED
Closed: 4 years ago → 4 years ago
status-firefox77: --- → fixed
Resolution: --- → FIXED

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #1)

H264 hardware decoding is bug 1622425.
H264 software decoding seems to use slow OpenH264: bug 1628428

*** This bug has been marked as a duplicate of bug 1622425 ***

Agreed. However, until we sort that openh264 out properly, my understanding is that we can enable ffmpeg-full as ffmpeg just ships its own openh264. We can re-work this once we have a proper solution in bug 1622425. What do you think?

Sorry, just to be sure : my understanding is that this (=software decoding by OpenH264 provided by runtime org.freedesktop.Platform.ffmpeg-full) is a workaround to slow software decoding by OpenH264 provided by runtime org.freedesktop.Platform.openh264 while still waiting for a fix for hardware decoding or am I wrong ? If I'm not mistaken the title of this bug is confusing
Thanks

(In reply to antistress from comment #10)

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #1)

H264 hardware decoding is bug 1622425.
H264 software decoding seems to use slow OpenH264: bug 1628428

*** This bug has been marked as a duplicate of bug 1622425 ***

Agreed. However, until we sort that openh264 out properly, my understanding is that we can enable ffmpeg-full as ffmpeg just ships its own openh264. We can re-work this once we have a proper solution in bug 1622425. What do you think?

Sorry, just to be sure : my understanding is that this (=software decoding by OpenH264 provided by runtime org.freedesktop.Platform.ffmpeg-full) is a workaround to slow software decoding by OpenH264 provided by runtime org.freedesktop.Platform.openh264 while still waiting for a fix for hardware decoding or am I wrong ? If I'm not mistaken the title of this bug is confusing
Thanks

Yes, that's my understnading too. AFAIK, it's workaround as ffmpeg just ships its own openh264. It's supposed to be temporarily until we sort out the open264 properly.

Attachment #9140380 - Flags: approval-mozilla-release?
Attachment #9140380 - Flags: approval-mozilla-beta?

Canceling the uplift request for now, there was something that was brought to my attention that I need to double-check for.

I have problems to understand this, too.
So, Blender feared licensing problems: https://github.com/flathub/org.blender.Blender/issues/39
Then they switched to org.freedesktop.Platform.ffmpeg-full (other flatpaks like Telegram use it as well). This seems anyway like what we want:
https://github.com/flathub/org.blender.Blender/pull/43

The Freedesktop platform now provides a limited ffmpeg which doesn't include all the codecs Blender can use, but it also provides an extension with a full ffmpeg.
This allows distributors to preinstall Blender and the platform without the nonfree/patented codecs, and users can choose to add the extension later on.

They noted ffmpeg-full uses org.freedesktop.Platform.openh264 (which can't encode, therefore they introduced an optional x264 extension - probably not possible for Mozilla).
https://github.com/flathub/org.blender.Blender/pull/47

We used to bundle our own ffmpeg with all the codecs Blender needs.

But because Endless wanted to preinstall Blender on some of its images, I made it use the runtime ffmpeg instead, or the one from the org.freedesktop.Platform.ffmpeg-full extension if it is installed.

That runtime extension has H264 support enabled… but doesn't actually contain any H264 encoder/decoder because it requires the org.freedesktop.Platform.openh264 extension for that.

  1. Hardware decoding/acceleration: org.freedesktop.Platform.ffmpeg-full seems to provide support for VAAPI hardware decoding, but that's an experimental, unfinished and disabled feature of Firefox' disabled native wayland backend (bug 1622425/bug 1610199/bug 1587060/bug 1543600). X11 VAAPI is tracked in bug 1619523, but RedHat is only interested in the Wayland implementation, as it's used by default in Fedora. Both require OpenGL rendering (gfx.webrender.all), but currently it's only enabled on Nightly (bug 1614523/bug 1491303). Software rendering is used by default.
  2. Software decoding: It seems the same OpenH264 (Cisco's binary blob) would be used. Or, ... are you already using ffmpeg-full? Or is Firefox currently using its OpenH264 Gecko Media Plugin - I thought it was only used for WebRTC, but I don't know? Maybe Flatpak is just inferior to regular Linux packages in terms of enabling proprietary codecs. If there is no other solution, Mozilla had an incentive to drive bug 1619523 faster forward.
  3. Release strategy: Why has Firefox Stable been released on Flathub without having released Nightly first, so that changes could be tested the next day?
Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7b21b61341c2
Backed out changeset 91d0340d9856. r=jcristau

(In reply to Pulsebot from comment #16)

Pushed by mtabara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7b21b61341c2
Backed out changeset 91d0340d9856. r=jcristau

The patch is incomplete, there' s some bits missing. Revert to OpenH264 for now.

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #14)

  1. Hardware decoding/acceleration: org.freedesktop.Platform.ffmpeg-full seems to provide support for VAAPI hardware decoding, but that's an experimental, unfinished and disabled feature of Firefox' disabled native wayland backend (bug 1622425/bug 1610199/bug 1587060/bug 1543600). X11 VAAPI is tracked in bug 1619523, but RedHat is only interested in the Wayland implementation, as it's used by default in Fedora. Both require OpenGL rendering (gfx.webrender.all), but currently it's only enabled on Nightly (bug 1614523/bug 1491303). Software rendering is used by default.
  2. Software decoding: It seems the same OpenH264 (Cisco's binary blob) would be used. Or, ... are you already using ffmpeg-full? Or is Firefox currently using its OpenH264 Gecko Media Plugin - I thought it was only used for WebRTC, but I don't know? Maybe Flatpak is just inferior to regular Linux packages in terms of enabling proprietary codecs. If there is no other solution, Mozilla had an incentive to drive bug 1619523 faster forward.

To be honest, this exceeds my knowledge so I can only answer to the release automation-related bits.

  1. Release strategy: Why has Firefox Stable been released on Flathub without having released Nightly first, so that changes could be tested the next day?

Looking back, that would've made the road less bumpy, I agree. When we first looked into this, our automation for nightly vs release was different and Flathub was still in its early stages and had no variety of channels (beta/stable) to push to, if I recall correctly. So we simplified the usecases, started building this for Firefox release, tested against their staging Flathub instance and then proceeded for the production instance. Both our automation and Flathub have grown, ever since, more mature, so now I think we'd be able to publish more products (ideally with different app ids).

See Also: → 1633341
Component: Release Automation: Flatpak → Release Automation: Other
Attachment #9140380 - Attachment is obsolete: true

It seems like this was backed out and never relanded. I've filed bug 1751322 about getting it back.

QA Contact: gbrown
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: