Closed Bug 1057646 Opened 10 years ago Closed 4 years ago

HTML5 <video> support for H.264 via OpenH264

Categories

(Core :: Audio/Video: Playback, enhancement)

x86_64
Linux
enhancement
Not set
normal

Tracking

()

RESOLVED WONTFIX

People

(Reporter: jld, Assigned: stransky)

References

Details

Attachments

(1 file, 8 obsolete files)

44.45 KB, patch
Details | Diff | Splinter Review
We have an H.264 codec now, by way of OpenH264, and it's already being used for WebRTC.  It could be useful to allow using it for HTML5 <video>, as an alternative to the existing support for OS-provided media facilities (e.g., bug 886181 comment #7).
We'd need a solution for the audio side of things (ie. AAC) for this to be useful I think.
OpenH264 also needs to support Main and High Profile H.264 before it's really useful for <video>.
Blocks: 799318
The OpenH264 project is working on including Main and High Profile, but adding support or licensing for AAC is not on the roadmap.
(In reply to cajbir (:cajbir) from comment #1)
> We'd need a solution for the audio side of things (ie. AAC) for this to be
> useful I think.

Actually, it can be useful without that. Some services are re-encoding animated GIFs as audioless video using h.264. (e.g. Twitter)

Video decoding via OpenH264 could also be useful to provide a consistent codec implementation for video, even if a platform audio codec is needed. Opus or another free audio codec could also be used with h.264 video.
Component: Audio/Video → Audio/Video: Playback
Gecko has the support. We just need a reason to enable it.
To improve the performance of videos?
To avoid problems on pc that don't have the codecs?

If OpenH264 is only for WebRTC is useless for the major people that use the browser for the mainly stuff like social networks and see videos.
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #5)
> Gecko has the support. We just need a reason to enable it.

As far as I'm concerned, the primary reason to use OpenH264 for HTML5 <video> is that it would give a universal and _consistent_ codec to all platforms. The current support is patchy and buggy and reliant on whatever the system has. This is not a reliable system. It's impossible to test properly on all platforms, as real world systems are messy, and using a system codec means relying on that mess.
(In reply to Daniele "Mte90" Scasciafratte from comment #6)
> To improve the performance of videos?

It doesn't do that.

> To avoid problems on pc that don't have the codecs?

Are you talking about machines that have an AAC decoder but no H.264 decoder. How common are those?

(In reply to Dave Garrett from comment #7)
> As far as I'm concerned, the primary reason to use OpenH264 for HTML5
> <video> is that it would give a universal and _consistent_ codec to all
> platforms. The current support is patchy and buggy and reliant on whatever
> the system has. This is not a reliable system. It's impossible to test
> properly on all platforms, as real world systems are messy, and using a
> system codec means relying on that mess.

OpenH264 doesn't get away from system decoders because you still need an AAC decoder. The advantage of system decoders is that they support hardware acceleration. The system software decoders are generally reliable.
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #8)
> OpenH264 doesn't get away from system decoders because you still need an AAC
> decoder. The advantage of system decoders is that they support hardware
> acceleration. The system software decoders are generally reliable.

h.264 is unfortunately ubiquitous for video; AAC, on the other hand, does not have to be used. Opus is available for free to all and is currently the best lossy audio codec in existence for virtually all use-cases. There isn't a video codec that can do this, yet.

The system decoders are reliable until they aren't. At minimum, this should be available for when the system ones break.

Also, why doesn't OpenH264 support hardware acceleration? That seems like something worth doing (at least, of course, for mainstream hardware).
Excuse my ignorance but Twitter states "We currently support MP4 and MOV video formats on mobile apps.

On the web, we support the MP4 video format with H264 format with AAC audio. You can upload videos up to 512MB, however you will be prompted to edit videos to 30 seconds or less in length."

Is the lack of ACC audio support the reason video will not play in firefox? If that is so is there not a way to get the video at least to play?
(In reply to crivera99 from comment #10)
> Is the lack of ACC audio support the reason video will not play in firefox?

What platform are you referring to? Which version of Firefox?

> If that is so is there not a way to get the video at least to play?

Which video?
I have windows xp, version 42 I guess they are not supporting XP anymore and that is the problem. Twitter video not working.
(In reply to crivera99 from comment #12)
> I have windows xp, version 42 I guess they are not supporting XP anymore and
> that is the problem. Twitter video not working.

This bug is about adding a new feature, not existing video playback support. However, this new feature would help you, as I don't think we ever added support for h.264 <video> on WinXP. I think it's Vista and up due to WinXP lacking the needed support, itself. That said, supporting WinXP isn't a sufficient reason to do anything anymore, by itself, at least.
(In reply to Dave Garrett from comment #13)
> This bug is about adding a new feature, not existing video playback support.
> However, this new feature would help you, as I don't think we ever added
> support for h.264 <video> on WinXP. I think it's Vista and up due to WinXP
> lacking the needed support, itself. That said, supporting WinXP isn't a
> sufficient reason to do anything anymore, by itself, at least.

Supporting MP4 playback on Windows requires both H.264 and AAC support to get video and audio. We are looking for a solution for Windows XP.
May this topic be related with
https://bugzilla.mozilla.org/show_bug.cgi?id=1230785

?
OpenH264 isn't useful for video playback so I'm going to close this bug.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
may it be useful in case like this (desperate) one?
https://bugzilla.mozilla.org/show_bug.cgi?id=1222272#c86
> OpenH264 isn't useful for video playback so I'm going to close this bug.

Could you clarify why do you think it is not useful?
Flags: needinfo?(ajones25)
(In reply to KOLANICH from comment #20)
> > OpenH264 isn't useful for video playback so I'm going to close this bug.
> 
> Could you clarify why do you think it is not useful?

I'm curious why this comment has been associated with me...
I just got a 'Need Info' email in my work account: I can't remember using Bugzilla, and at first assumed it was a phishing attack...
In any case, I'm not ajones@mozilla.com
Flags: needinfo?(ajones25)
Flags: needinfo?(ajones)
(In reply to KOLANICH from comment #20)
> > OpenH264 isn't useful for video playback so I'm going to close this bug.
> 
> Could you clarify why do you think it is not useful?

OpenH264 is not useful because it is too slow for playback purposes and doesn't support b-frames. I'm not aware of any platforms that have an AAC decoder but are missing an H.264 decoder. You could encode adverts so that OpenH264 can play them but it would make more sense to use WebM.
Flags: needinfo?(ajones)
Attached patch openh264.patch (obsolete) — Splinter Review

For those who might be interested in using openh264 for h264 playback, I've attached the patch. With current openh264 version, only baseline profile works, also there's missing aac codec for audio playback.

There's some move forward regarding B-frame support: https://github.com/cisco/openh264/issues/2981

Assignee: nobody → jhorak
Attachment #9065016 - Attachment is obsolete: true

Added also fdk-aac for audio playback and media.ffmpeg.preferred preference for giving system ffmpeg lib higher.

Just to explain/update Jan's comments:

As of v2.0 OpenH264 supports the main and high profiles and should work for most H.264 files. AAC support can be handled by FDK-AAC-Stripped, which is a version of FDK-AAC with all of the patented bits stripped out.

FDK-AAC has a non-standard license, but Redhat legal and the FSF consider it to be free software, although Debian has placed it in their non-free repo. WRT patents, Redhat legal has green-lit the use of FDK-AAC-Stripped in mainline.

It might be time to reconsider using OpenH264 for videos, given that H264 patents will be around until ~2027. But it is more code to maintain for a feature that is best handled using system libraries or hardware decoders....

Flags: needinfo?(ajones)

I'm no longer involved in media playback, but I can give some commentary.

The forward direction for this (ignoring the long tail) is to get people using VAAPI through Wayland. The openh264 path exists but last time I checked (years ago) the performance was poor. Playback is more demanding on the CPU than video calls and openh264 was only single threaded (that may not still be true).

Widevine includes support for encrypted H.264, so once you get AAC you get most of the premium (DRM-based) services working. YouTube uses free codecs so that will also work. What you're left with is a bunch of sites that all have different player implementations and you can't rely on them responding to dropped frames should openh264 fail to keep up.

Any decisions about this are up to Nils.

Nils - any thoughts or has anything changed since last I looked at this?

Flags: needinfo?(ajones) → needinfo?(drno)

Folks, can be this patch included as a Tier-3 which means Mozilla will not use this code path but it will be used by distros which can't use ffmpeg by default like Fedora/RHEL?

Flags: needinfo?(drno)

I'm going to work on that. This is important to have it as Chromium already ships it and can play H.264 clips out of the box but Firefox fails so people are directed to use Chrome/Chromium.

The OpenH264 codec already supports the high profile and should be used only when system ffmpeg is missing and we're requested to play H.264 clips.

Assignee: jhorak → stransky
Status: RESOLVED → REOPENED
Type: defect → enhancement
Resolution: WONTFIX → ---

Also I think the final solution should be incorporated with Bug 1660336 as we may:

  1. use system ffmpeg first if VA-API is enabled and recent enough
  2. fall back to bundled (ffvpx) for VP9/8
  3. try system ffmpeg
  4. fallback to openh264 when system ffmpeg is missing
Status: REOPENED → ASSIGNED
  • Add empty EnableOpenH264Decoder() method to PlatformDecoderModule.
  • Implement it for ffvpx FFmpegDecoderModule module by mIsOpenH264Enabled attribute.

Depends on D89347

We already have a OpenH264 decoder via the GMP.

It can be used for HTML5 video so long as it's prefed on. The lack of support for the high profile was an issue in the past, we would need to update the GMP component.

However, this doesn't solve the case where OpenH264 is provided by Cisco and needs to be installed regardless, and on Android it appears we will no longer be able to install the OpenH264 plugin unless we use the playstore.

Status: ASSIGNED → RESOLVED
Closed: 8 years ago4 years ago
Resolution: --- → WONTFIX

(In reply to Martin Stránský [:stransky] from comment #29)

I'm going to work on that. This is important to have it as Chromium already ships it and can play H.264 clips out of the box but Firefox fails so people are directed to use Chrome/Chromium.

I don't believe that's the case.

Chrome can play h264 out of the box, but not Chromium.

The OpenH264 codec already supports the high profile and should be used only when system ffmpeg is missing and we're requested to play H.264 clips.

Then we need to update the various OpenH264 plugin and let the GMPDecoderModule be a default for video decoding
https://searchfox.org/mozilla-central/source/dom/media/platforms/PDMFactory.cpp#398-404

and for that to work the pref media.gmp.decoder.enabled needs to be turned on.
https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml#7149-7152

Attachment #9174174 - Attachment is obsolete: true
Attachment #9174173 - Attachment is obsolete: true
Attachment #9174171 - Attachment is obsolete: true
Attachment #9174170 - Attachment is obsolete: true
Attachment #9174169 - Attachment is obsolete: true
Attachment #9174168 - Attachment is obsolete: true
Attachment #9174172 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.