1628428 - Flatpak: H.264 videos slower than distribution build (probably caused by OpenH264 usage)

Status: RESOLVED DUPLICATE of bug 1628407

(Release Engineering :: Release Automation: Other, defect)

Description

[peter.eszlari]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0

Steps to reproduce:

1. install the Flatpak version of Firefox
2. go to www.msnbc.com
3. watch some video

distribution: Kubuntu 19.10
DE: KDE Plasma 5.16.5
GFX: Nvidia Geforce 960 GTX (driver: 435.21)

resolution: 3840x2160
layers.acceleration.force-enabled: true
gfx.webrender.all: true

Actual results:

low FPS during video playback

Expected results:

fluent video playback (like with the distribution build)

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[peter.eszlari]

I tried more sites: It seems like H.264 videos are affected, VP9 are played back fluently.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Jan, are you able to reproduce this?

Comment 4

[Darkspirit]

Yes, I had the same impression with Firefox 75 Flatpak. It did not seem to occur with the regular Firefox 75 ubuntu package.
On MSNBC some frames are just missing. On YouTube I noticed it with h264 videos.

Ubuntu:
$ sudo apt install flatpak plasma-discover-flatpak-backend # or gnome-software-plugin-flatpak
$ flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
reboot
Open https://flathub.org/apps/details/org.mozilla.firefox, click on Install, open and install with the software management app

Comment 6

[antistress]

How is it affected regarding new Firefox 75 experimental functionnality (Bug 1616185: h.264 VA-API decode by ffmpeg on GNU/Linux) ?
Thanks

Comment 7

[Darkspirit]

Offtopic: According to bug 1622425, Flatpak doesn't support VAAPI yet. Please use https://nightly.mozilla.org to have latest fixes for VAAPI. It's not finished yet.

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Jan, can you also confirm that the problem doesn't happen with VP9?

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The component has been changed since the backlog priority was decided, so we're resetting it.
For more information, please visit auto_nag documentation.

Comment 10

[peter.eszlari]

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #4)

Yes, I had the same impression with Firefox 75 Flatplak. It did not seem to occur with the regular Firefox 75 ubuntu package.

I think, I found the reason: the regular Ubuntu package of libavcodec has the native H.264 decoder enabled, while in Flatpak, libavcodec is using the wrapper for OpenH264 from the runtime extension org.freedesktop.Platform.openh264, which I guess is much slower. For whatever reason, installing the extension org.freedesktop.Platform.ffmpeg-full (which includes ffmpeg's native decoder) doesn't help.

Comment 11

[Darkspirit]

Yes, I could play a 4k vp9 YouTube video without problem. H264 had either visually missing frames without any "dropped frames" (YouTube's video stats) or both.
Yes, I have seen Flatpak/OpenH264 usage for software decoding in bug 1623438 comment 0.

Comment 12

[peter.eszlari]

https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/19.08/README.md#html5-codecs

Quote:

We intend to provide an openh264 extension, which will provide a royalty free
H.264 codec. However, this extension will only be available for flatpak version
1.4.2 or later. This is (at present) packaged in very few distros.

The main takeaway is this: if you need to use H.264 codecs, include the
ffmpeg-full extension with your app.
For an example of this, see tests/test.codecs.ffmpeg-full.json, which is a
flatpak manifest for an app using the full codecs.

https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/19.08/tests/test.codecs.ffmpeg-full.json

list with included decoders:
https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/19.08/elements/extensions/ffmpeg-full/ffmpeg.bst

Comment 13

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

From the media perspective we use whatever ffmpeg is available. I'm not that familiar with flatpak and how we do our builds, so I don't fully understand how much of this we control. Moving component to see if build folks can give an answer as to the control and flexibility we have to use the full ffmpeg extension.

Comment 15

[peter.eszlari]

How is this a duplicate of 1628407 (a bug about hardware decoding) when this bug here is about software decoding?

Comment 16

[peter.eszlari]

Anyway, when this https://phabricator.services.mozilla.com/D70830 reaches stable, the problem should be fixed.

Comment 17

[Darkspirit]

bug 1628407 was backed out. This is a wontfix as long as OpenH264 is used.

Comment 19

[Russell Haley]

This is now affecting non-flatpak builds:

https://old.reddit.com/r/Fedora/comments/k91qny/random_youtube_videos_lagging_others_are_fine_on/
https://old.reddit.com/r/Fedora/comments/k7s9xu/firefox_choppy_video_playback_on_fedora_33_amd_gpu/
https://old.reddit.com/r/Fedora/comments/k7d0p1/youtube_on_firefox_from_fedoras_repo_lags/

OpenH264 is a legal fiction that allows Firefox to advertise out-of-the-box WebRTC support. It is barely adequate for decoding the low-complexity, low-bitrate streams that a video-conferencing peer can encode in real-time and upload on an asymmetric home internet connection. It is completely unfit for purpose as an internet video decoder.

The use of OpenH264 complicates the troubleshooter's chain of reasoning. "Video doesn't play (possibly with missing codec error) -> install codec for that format," is much more obvious than, "Video stutters along at 8 FPS -> install codec for that format".

Also some websites might be able to fallback to a different codec that the user's computer could actually play, if it weren't falsely claiming support for h.264.

I suggest that the decision to use it for general-purpose decoding be re-visited.

Comment 20

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

It looks like those issues are Fedora specific, and as indicated in the last link seem to also be specific to the Fedora packaged version. By default Firefox will not use Openh264 for playback and making it do so is typically non trivial (you either need ffmpeg present, but using OpenH264 internally, or you need to set some env vars and prefs). For the above cases I'd suggest discussion with those maintaining the Fedora packages would be more productive.