Closed Bug 1749046 (AudioEncoder) Opened 2 years ago Closed 2 months ago

[WebCodecs] Implement AudioEncoder

Categories

(Core :: Audio/Video: Web Codecs, task, P1)

task

Tracking

()

RESOLVED FIXED
126 Branch
Tracking Status
firefox126 --- fixed

People

(Reporter: chunmin, Assigned: padenot)

References

(Depends on 3 open bugs, Blocks 1 open bug)

Details

Attachments

(37 files)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
Component: Audio/Video → Audio/Video: Web Codecs
Alias: AudioEncoder
Depends on: 1860926

Depends on D192685

Depends on D197916

Depends on: 1876064
Depends on: 1876065
Depends on: 1876066

Sorry for the clang-format changes folded in.

This look like it's missing quite a few methods, but this is because the code is
shared and mostly identical in behaviour to the VideoEncoder class -- the
AudioEncoder class only implements the bits of behaviour that are
audio-specific. The audio-specific adjustments to the base class (which are not
numerous) are in the next patch of this series.

I found that useful when debugging, let's land it.

InitPromise now doesn't return the track type (since we're not using anything
related to tracks on the encode side).

Methods now either return a result or a nsresult.

Time base is handled in the child class (since it's going to be different from
audio and video).

It's very similar to the video counterpart, except:

  • Conversion to planar is done as needed, f32 is always used.
  • Most codec specific options are implemented later in the series.
  • The time base is always the sameple-rate, as this is what ffmpeg does
    internally, this limits conversions.
  • Codecs only accept a specific number of sample-frame per packet, this class
    repacketizes the input audio in the correct packet size (that can depend on the
    same rate, the codec, some other options).
  • The encoder delay isn't really handled, except all packets are shifted, and
    mOriginalPresentationWindows is set as it should so that no timing information
    is lost. This will be changed later.
Attachment #9376268 - Attachment description: Bug 1749046 - Remove expectations files for AudioEncoder WPTs. r?chunmin → Bug 1749046 - Adjust expectation files for AudioEncoder WPTs. r?chunmin
Depends on: 1883451
Assignee: cchang → padenot
Pushed by padenot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2610b665abfe
Add IsClosed to AudioData. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/60f8d9644c5d
Allow comparing audio config objects. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/98f8a6b55cb0
Make AudioData::ToString public. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/f31d976a1a57
Move some audio utilities to WebCodecUtils. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/00f57d5f27f6
Allow printing an AudioData for debugging purposes. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/8ef0b45aa258
Remove stray semicolon in AudioConverter.h. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b12d9c0f52f1
Extend the PEM interface to handle audio r=chunmin
https://hg.mozilla.org/integration/autoland/rev/22e7f89634b7
Allow converting all channels of an AudioData to f32 interleaved, but not from script. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/88b7fc2cbc1f
Allow printing an AudioDecoderConfigInternal, for logging. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/3de7e6e305e1
Interface and implementation for AudioEncoder. r=chunmin,webidl,smaug
https://hg.mozilla.org/integration/autoland/rev/b116ff85089d
Adjust EncoderTemplate to handle audio-specific things. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/50d78605b761
Count the number of encoded packets in logs. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/eabb710ecc1c
Factor out the code to set the description to use it on the audio side, in EncoderTemplate. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/ca880ba5c83d
Generalize EncoderTraits so that it works for audio. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/7770bc65067f
Add Opus-specific configuration options. r=chunmin,webidl,smaug
https://hg.mozilla.org/integration/autoland/rev/e1f13ef3badd
Vendor libopusenc.c and libvorbisenc.c and support files in ffvpx. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/dced90edf02c
Clarify that AudioData instances hold interleaved audio data. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/dce9cbf1e6f4
Add Vorbis to the codec list in PEM. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/47240f85df7c
Add more codecs to the PEM interface, even though they have no implementation yet. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/31ec05902d0a
Allow draining an AudioPacketizer. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/b05081eca4fe
Implement TimePacketizer. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/7936d27e4a10
Interface changes to FFmpegDataEncoder to prepare for FFmpegAudioEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/221953f41e1c
Implement FFmpegAudioEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/fe2e5e695048
Set the channel layout in FFmpegAudioEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/9de63c54266d
Add a way to iterate ffmpeg's list that end in 0 to FFmepgUtils.h. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/42d651b12660
Resampling input audio to the encoder sample-rate. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/c36ec1343c7d
Adjust expectation files for AudioEncoder WPTs. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/10a16e8a79ad
Put H264 enums in the correct header. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/e96c1bc9e530
Split EncoderConfig.h out of PlatformEncoderModule.h. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/92a04a3d9162
Expose more functions to set ffmpeg options. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/96806c2bdb29
Reflect Opus-specific configuration options to FFmpegAudioEncoder configuratin. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/022e47d17cca
Even when the configuration of an encoder wouldn't change because the config is identical, the decoder config must be emitted. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/a1c56f710998
Add a DTX flag to libopusenc.c. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/88078e99231d
Enable DTX and handle DTX packets in FFmpegAudioEncoder. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/2a446c2233a2
Move all ToString() methods on various objects to be nsCString containing utf-8. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/29994d907858
The Opus code supports bitrate in the [6,510]k range. r=chunmin
https://hg.mozilla.org/integration/autoland/rev/8b7f3601508a
1860925: apply code formatting via Lando
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/45377 for changes under testing/web-platform/tests
Attachment #9393569 - Attachment description: WIP: Bug 1749046 - Don't assert when checking audio encoding support on Windows. → Bug 1749046 - Don't assert when checking audio encoding support on Windows.
Pushed by padenot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a56c3a8cfb52
Don't assert when checking audio encoding support on Windows.

https://hg.mozilla.org/mozilla-central/rev/2610b665abfe
https://hg.mozilla.org/mozilla-central/rev/60f8d9644c5d
https://hg.mozilla.org/mozilla-central/rev/98f8a6b55cb0
https://hg.mozilla.org/mozilla-central/rev/f31d976a1a57
https://hg.mozilla.org/mozilla-central/rev/00f57d5f27f6
https://hg.mozilla.org/mozilla-central/rev/8ef0b45aa258
https://hg.mozilla.org/mozilla-central/rev/b12d9c0f52f1
https://hg.mozilla.org/mozilla-central/rev/22e7f89634b7
https://hg.mozilla.org/mozilla-central/rev/88b7fc2cbc1f
https://hg.mozilla.org/mozilla-central/rev/3de7e6e305e1
https://hg.mozilla.org/mozilla-central/rev/b116ff85089d
https://hg.mozilla.org/mozilla-central/rev/50d78605b761
https://hg.mozilla.org/mozilla-central/rev/eabb710ecc1c
https://hg.mozilla.org/mozilla-central/rev/ca880ba5c83d
https://hg.mozilla.org/mozilla-central/rev/7770bc65067f
https://hg.mozilla.org/mozilla-central/rev/e1f13ef3badd
https://hg.mozilla.org/mozilla-central/rev/dced90edf02c
https://hg.mozilla.org/mozilla-central/rev/dce9cbf1e6f4
https://hg.mozilla.org/mozilla-central/rev/47240f85df7c
https://hg.mozilla.org/mozilla-central/rev/31ec05902d0a
https://hg.mozilla.org/mozilla-central/rev/b05081eca4fe
https://hg.mozilla.org/mozilla-central/rev/7936d27e4a10
https://hg.mozilla.org/mozilla-central/rev/221953f41e1c
https://hg.mozilla.org/mozilla-central/rev/fe2e5e695048
https://hg.mozilla.org/mozilla-central/rev/9de63c54266d
https://hg.mozilla.org/mozilla-central/rev/42d651b12660
https://hg.mozilla.org/mozilla-central/rev/c36ec1343c7d
https://hg.mozilla.org/mozilla-central/rev/10a16e8a79ad
https://hg.mozilla.org/mozilla-central/rev/e96c1bc9e530
https://hg.mozilla.org/mozilla-central/rev/92a04a3d9162
https://hg.mozilla.org/mozilla-central/rev/96806c2bdb29
https://hg.mozilla.org/mozilla-central/rev/022e47d17cca
https://hg.mozilla.org/mozilla-central/rev/a1c56f710998
https://hg.mozilla.org/mozilla-central/rev/88078e99231d
https://hg.mozilla.org/mozilla-central/rev/2a446c2233a2
https://hg.mozilla.org/mozilla-central/rev/29994d907858
https://hg.mozilla.org/mozilla-central/rev/8b7f3601508a
https://hg.mozilla.org/mozilla-central/rev/a56c3a8cfb52

Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 126 Branch
Upstream PR merged by moz-wptsync-bot
Regressions: 1889318
Depends on: 1892785
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: