simulcast support for H264
Categories
(Core :: WebRTC: Audio/Video, enhancement, P3)
Tracking
()
People
(Reporter: bwc, Assigned: pehrsons)
References
(Blocks 1 open bug, Regressed 1 open bug)
Details
(Keywords: dev-doc-complete, webcompat:platform-bug, Whiteboard: [jitsi-meet])
Attachments
(10 files, 1 obsolete file)
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
Details | Review |
Comment 1•10 years ago
|
||
Comment 2•8 years ago
|
||
Comment 4•7 years ago
|
||
Depends on bug 1607238?
Comment hidden (advocacy) |
Comment 8•5 years ago
|
||
Agreed.
Comment 10•4 years ago
|
||
More accurate dependency: Bug 1654112.
Reporter | ||
Updated•4 years ago
|
Updated•3 years ago
|
Comment 11•2 years ago
|
||
"We have implemented to reject all Firefox from the conference, only Chrome and Safari allowed, and since then we had no problems."
What can I say? Sic transit gloria mundi.
Comment 12•2 years ago
|
||
Any update on this?
Thank you
Assignee | ||
Updated•8 months ago
|
Assignee | ||
Comment 13•8 months ago
|
||
Assignee | ||
Comment 14•8 months ago
|
||
Bitrate 0 could be configured when used with the SimulcastEncoderAdapter.
Assignee | ||
Comment 15•8 months ago
|
||
The GMP API of OpenH264 doesn't support configuring temporal layers.
Assignee | ||
Comment 16•8 months ago
|
||
Assignee | ||
Comment 17•8 months ago
|
||
It is unnecessary since bug 1932381 landed.
Assignee | ||
Comment 18•8 months ago
|
||
Assignee | ||
Comment 19•8 months ago
|
||
Assignee | ||
Comment 20•8 months ago
|
||
Assignee | ||
Comment 21•8 months ago
|
||
fakeopenh264 works by encoding some metadata and when decoding generating an
image from that metadata. It figures out what color to use for image generation
by averaging the color of all pixels in the encoder and putting that in the
payload.
To make VideoFrameEmitter work with fakeopenh264 we must therefore use a single
color across the entire canvas.
Assignee | ||
Comment 22•8 months ago
|
||
Upstream commit: https://webrtc.googlesource.com/src/+/4a6a7465d03f0b581122f11385aca87294eebd5d
In ParseNonParameterSetNalu check BitstreamReader::Ok before returning early
~BitstreamReader() DCHECKs that the last read has been verified, so all
paths where we may leave the slice_reader instance's scope early must be
guarded by an Ok().
Bug: None
Change-Id: Ic67f87c04d1f042392c1dd6a066fdccf26e19003
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/369540
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43463}
Updated•8 months ago
|
Assignee | ||
Updated•7 months ago
|
Updated•6 months ago
|
Assignee | ||
Comment 23•6 months ago
•
|
||
dev-doc-info |
Like bug 1692873 this needs a release note for developers. Something like: "WebRTC simulcast is now supported with H264 video."
Assignee | ||
Comment 24•6 months ago
|
||
Updated•6 months ago
|
Comment 25•6 months ago
|
||
Comment 26•6 months ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/0ec1f62c299a
https://hg.mozilla.org/mozilla-central/rev/0f26930f651d
https://hg.mozilla.org/mozilla-central/rev/5c59074b51bb
https://hg.mozilla.org/mozilla-central/rev/2873625fe17f
https://hg.mozilla.org/mozilla-central/rev/8d3ace737d57
https://hg.mozilla.org/mozilla-central/rev/d3b7681ac176
https://hg.mozilla.org/mozilla-central/rev/804e73d3b870
https://hg.mozilla.org/mozilla-central/rev/60b401685488
https://hg.mozilla.org/mozilla-central/rev/1c00fa1024ca
https://hg.mozilla.org/mozilla-central/rev/923c6123afcb
Updated•6 months ago
|
Comment 27•6 months ago
|
||
:pehrsons could you consider nominating this for a release note? (Process info)
Assignee | ||
Comment 28•6 months ago
|
||
Release Note Request (optional, but appreciated)
[Why is this notable]: Simulcast is needed by any video conferencing site that want to do large video calls, where peers are connected via a media server rather than directly to each other (peer to peer). H264 is the only other codec to be supported than VP8, and H264 simulcast is supported with hardware codecs on Android.
[Affects Firefox for Android]: Yes, in particular H264 is the only video codec that we have both simulcast and hardware support for.
[Suggested wording]: Firefox now supports sending multiple simultaneous versions of the same source over WebRTC, so called simulcast, with the H264 video codec. H264 is the second video codec after VP8 to be supported for sending simulcast.
[Links (documentation, blog post, etc)]: n/a
Comment 29•6 months ago
|
||
Thanks, added to the Fx136 nightly release notes, please allow 30 minutes for the site to update.
Keeping the relnote-firefox flag as ? to keep it on the radar for inclusion in the final Fx136 release notes.
Comment 30•5 months ago
•
|
||
The MDN WebRTC codecs guide has this note for H.264
Firefox for Android 68 and later do not support AVC (H.264) anymore.
This is due to a change in Google Play store requirements that prevent Firefox from downloading and installing the OpenH264 codec needed to handle H.264 in WebRTC connections. See <a href="https://support.mozilla.org/en-US/kb/firefox-android-openh264">this article on SUMO</a> for details.
Obviously no longer true, since this says we support it in hardware.
Can you tell me what version we started supporting it on Android again - is it this 74 as per https://bugzilla.mozilla.org/show_bug.cgi?id=1592140 ?
FYI I'm looking at the docs in https://github.com/mdn/content/pull/38079/files
It is quite likely that the page is out of date, so welcome any suggestions.
Assignee | ||
Comment 31•5 months ago
•
|
||
(In reply to Hamish Willee from comment #30)
The MDN WebRTC codecs guide has this note for H.264
Firefox for Android 68 and later do not support AVC (H.264) anymore.
This is due to a change in Google Play store requirements that prevent Firefox from downloading and installing the OpenH264 codec needed to handle H.264 in WebRTC connections. See <a href="https://support.mozilla.org/en-US/kb/firefox-android-openh264">this article on SUMO</a> for details.Obviously no longer true, since this says we support it in hardware.
Can you tell me what version we started supporting it on Android again - is it this 74 as per https://bugzilla.mozilla.org/show_bug.cgi?id=1592140 ?
Thanks for catching this! Indeed it should be working since bug 1592140.
FYI I'm looking at the docs in https://github.com/mdn/content/pull/38079/files
It is quite likely that the page is out of date, so welcome any suggestions.
I don't have much feedback to the docs on your patch, but perhaps we should mention the dependency descriptor RTP header extension, added in bug 1945261 to accompany mainly AV1 simulcast support. I'll explain it briefly here:
Older codecs like VP8 and VP9 included in their RTP payload a "payload header" with information on frame dependencies (keyframes have no dependencies, interframes typically depend on the previous frame, which depend on the previous, etc. to create a chain to the most recent keyframe). This is generally fine and SFUs can forward packets and switch simulcast layers without a receiver noticing. AV1 however doesn't have this information in the payload. Also, in scenarios involving end-to-end encryption (E2EE) the payload is encrypted and inaccessible to SFUs, for all codecs. The AV1 spec therefore defined a new RTP header extension called dependency descriptor. The RTP header (as opposed to the RTP payload header which is part of the payload) is not encrypted under E2EE. It being defined as a proper RTP header extension makes it suitable also for other codecs, and SFUs can then be made codec agnostic. Bug 1945261 added support for DD when doing simulcast with VP8, VP9 (which we don't do simulcast for by default yet, bug 1633876) and AV1. Bug 1947115 (OpenH264 on desktop) and bug 1947116 (Android H264 hw) adds support for it with our other codecs.
Comment 32•5 months ago
|
||
FF135 MDN docs for this can be tracked in https://github.com/mdn/content/pull/38079
@Andreas - thank you so much for the response. I have captured this in https://github.com/mdn/content/pull/38079 - you're welcome to review.
There wasn't a perfect place for the "DD" information so I have included it under the AV1 docs in the WebRTC codecs guide (because that is where it is in the spec). The section does note that this is generically useful and why.
Updated•5 months ago
|
Description
•