1673285 - Port libSoundTouch to RLBox

Status: RESOLVED FIXED

(Core :: Audio/Video, enhancement)

Description

[Evan Johnson]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0

Steps to reproduce:

This is the bug to track the process of porting libSoundTouch to RLBox. Currently it still uses the unsandboxed libSoundTouch, but now it uses the RLBox API so that we can easily switch between unsandboxed and sandboxed versions of the library.

Comment 1

[Evan Johnson]

Attachment: Bug 1673285 - Part 1: Retrofit AudioStream to use RLBoxed soundtouch.

Comment 2

[:Gijs (out for now; he/him)]

Why did you file this as a security bug?

Comment 3

[Evan Johnson]

I marked it as a security bug as it concerns the sandboxing (and therefore security) of a media library, although this is more of a security infrastructure issue than an actual vulnerability. If this is incorrect, it can be marked as a non-security bug.

Comment 4

[Deian Stefan]

I can't edit the security checkbox, but we don't need this to be a security bug. None of the other sandboxing bugs are.

Comment 5

[:Gijs (out for now; he/him)]

Unhiding per comment #3 / comment #4.

Comment 6

[Evan Johnson]

Attachment: Bug 1673285 - Make requested changes to libSoundTouch port

Depends on D94698

Comment 7

[Evan Johnson]

Attachment: Bug 1673285 - Remove old file

Depends on D99634

Comment 8

[Evan Johnson]

Attachment: Bug 1673285 - Fix build errors and clang format complaints

Depends on D99635

Comment 9

[Deian Stefan]

Seems like all the new patches should be squashes into a single patch (the original phab).

Comment 10

[Evan Johnson]

Attachment: Bug 1673285 - Update rlbox validators

Depends on D94698

Comment 11

[Deian Stefan]

Attachment: Bug 1673285 - Part 3: Add support for Wasm sandboxing soundtouch.

Depends on D94698

Comment 12

[Deian Stefan]

Attachment: Bug 1673285 - Part 5: Turn on Wasm sandboxing for soundtouch.

Depends on D109843

Comment 13

[Deian Stefan]

Attachment: Bug 1673285 - Part 2: Retrofit AudioDecoderInputTrack to use RLBoxed soudtouch.

Depends on D94698

Comment 14

[Deian Stefan]

Attachment: Bug 1673285 - Part 4: Export soundtouch symbols for windows builds.

Depends on D109843

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

Comment 16

[Willy R. Vasquez]

Attachment: Bug 1673285 - Retrofit AudioStream and AudioDecoderInputTrack to use RLBoxed SoundTouch. r=padenot,shravanrn,glandium,deian

Combines previous Patches 1-4, adapted to the current codebase.

Comment 17

[Willy R. Vasquez]

Attachment: Bug 1673285 - Enable Wasm sandboxing for SoundTouch. r=padenot,shravanrn,glandium,deian

Part of patch 4 and patch 5 from previous effort.

Depends on D183627

Comment 18

[Willy R. Vasquez]

Attachment: Bug 1673285 - Enable SIMD for Wasm sandboxed SoundTouch. r=padenot,shravanrn,glandium,deian

This patch enables SIMD in Wasm sandboxed SoundTouch by using
simd-everywhere to map SSE intrinsics to WASMSIMD intrinsics and
compiling with -msimd128 flag.

Depends on D183628

Comment 19

[Willy R. Vasquez]

These collection of patches apply the previous effort to the most recent Mozilla codebase, and also enable SIMD support for Wasm sandboxed libSoundTouch.

For

Comment 20

[Willy R. Vasquez]

(continuing the previous message)

For completeness, I'm copying over https://bugzilla.mozilla.org/show_bug.cgi?id=1829765#c11 :

---

We have some SoundTouch numbers we can now share [1].
​
We calculated these numbers by following the methodology described in [2,3]:

1. Open [4] and click 'set rate and play'. This will start playing a sound at 0.25x speed. Each time we click 'set rate and play' the speed will increase by 1x, up to 3.25x.
2. Begin collecting Profiling information with Firefox Profiler
3. Let the audio play completely at each speed, then click 'set rate and play' to go to the next speed.
4. Once it stops playing, stop Profiling.
5. Use [5] to plot Audio Callback values and get statistics.
   ​

Note that Native+SIMD is the default in Firefox.
​
Results:

 **Mean time (s)**
          No SIMD |   SIMD
        -------------------
 Native | 0.03382 | 0.01486 |
        |-------------------|
  Wasm  | 0.04419 | 0.01745 |
        -------------------
**Median time (s)**
          No SIMD |   SIMD
        -------------------
 Native | 0.03304 | 0.01386 |
        |-------------------|
  Wasm  | 0.04165 | 0.01473 |
        -------------------

The above Mean time numbers show that an RLBoxed SoundTouch with SIMD (0.01745s) gives us a 2.5x speedup compared to an RLBoxed SoundTouch without SIMD (0.04419s). Previous results compared native+SIMD (0.01486s) to RLBoxed SoundTouch without SIMD (0.04419s) giving an almost 200% overhead. Compared to native+SIMD (0.01486s), RLBoxed SoundTouch with SIMD (0.01745s) has a 17% overhead. The linked to Google docs [1] has some more details.
​
[1] New results: https://docs.google.com/document/d/19LtpeSWA6ghfAEGw1lN8im2QPMc7wL2W7LusXPzfg18/edit
[2] Methodology: https://blog.paul.cx/post/profiling-firefox-real-time-media-workloads/#the-new-solution
[3] Previous results: https://docs.google.com/document/d/1h4UsI_9bXrMS3jy1dtu83GLwdi7xB3bNYrWh2MdyEes/edit
[4] Benchmark website: https://ritter.vg/misc/ff/soundtouch-perf.html
[5] Audio callback plotting plugin: https://github.com/padenot/fx-profiler-audio-cb

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f66c9e9aad96
Retrofit AudioStream and AudioDecoderInputTrack to use RLBoxed SoundTouch. r=padenot,glandium
https://hg.mozilla.org/integration/autoland/rev/f28e1fe0311f
Enable Wasm sandboxing for SoundTouch. r=glandium
https://hg.mozilla.org/integration/autoland/rev/6c2e3da74c76
Enable SIMD for Wasm sandboxed SoundTouch. r=glandium

Comment 22

[Sandor Molnar[:smolnar]]

Backed out for causing build bustages in dom/media/AudioStream.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/273aa7310e552441853148728a6d1945ccb991cd

Push with failures

Failure log -> ERROR - /builds/worker/checkouts/gecko/dom/media/AudioStream.cpp:497:10: error: duplicated 'if' condition [-Werror=duplicated-cond]

Comment 23

[Mike Hommey [:glandium]]

Can you fix the problem with AudioStream.cpp and also update rlfox_wasm2c_sandbox to https://github.com/PLSysSec/rlbox_wasm2c_sandbox/commit/0938ae96c5467bdf2210cf21be47ae576cbcc3cd ?

Comment 24

[Willy R. Vasquez]

Fixed, and see https://bugzilla.mozilla.org/show_bug.cgi?id=1850471

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9cde99c7ec6a
Retrofit AudioStream and AudioDecoderInputTrack to use RLBoxed SoundTouch. r=padenot,glandium
https://hg.mozilla.org/integration/autoland/rev/ac7b4c876c2c
Enable Wasm sandboxing for SoundTouch. r=glandium
https://hg.mozilla.org/integration/autoland/rev/0e326fb37f8c
Enable SIMD for Wasm sandboxed SoundTouch. r=glandium

Comment 26

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/9cde99c7ec6a
https://hg.mozilla.org/mozilla-central/rev/ac7b4c876c2c
https://hg.mozilla.org/mozilla-central/rev/0e326fb37f8c