2040494 - Collapse NEON WebAudio fingerprints

Status: ASSIGNED

(Core :: Web Audio, enhancement)

Description

[Tom Ritter [:tjr]]

It looks like the work in Bug 2036977 collapsed x64-64 with a FMA3 CPU into the x86-64 without FMA3 (plus x86) bucket, but not aarch64 (NEON instructions). (I lack such a machine to test locally.)

This bug is about investigating that.

Comment 1

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:karlt, could you have a look please?

For more information, please visit BugBot documentation.

Comment 2

[Karl Tomlinson (:karlt)]

Chrome seems to have a similar difference: https://issues.chromium.org/issues/40160543

Comment 3

[Tom Ritter [:tjr]]

Attachment: WIP: Bug 2040494 - Add scalar-only ffvpx FFT path, gated by two RFPTargets.

WebAudio output exposed to script via destination buffer rendering
and AnalyserNode.getFloatFrequencyData() differs by 1-10 ULP per
sample between x86_64 and aarch64. Two ffvpx non-determinisms drive
that drift:

1. libavutil/tx_template.c computes FFT twiddle-factor tables via
   raw libm cos/sin at av_tx_init() time. glibc, musl, and Darwin
   libm each ship their own cos/sin approximations (sin and cos are
   not IEEE-754 correctly-rounded), so the tables differ by a few
   ULP across libcs. Route the cos/sin calls in tx_template.c
   through Mozilla's bundled fdlibm via a file-scoped #include and
   macro redirect. fdlibm is a static Library, so adding it to
   libmozavutil's USE_LIBS links its object files directly into the
   shared library without any cross-DSO symbol lookup.

2. Even with deterministic twiddle tables, ffvpx dispatches FFT
   execution to arch-specific assembly:
   libavutil/x86/tx_float.asm (AVX2/SSE3) and
   libavutil/aarch64/tx_float_neon.S (NEON). The two kernels are
   independently hand-written and order their butterfly multiply-add
   operations differently, producing residual 1-10 ULP output
   divergence. Add AV_TX_MOZ_SCALAR_ONLY as a Mozilla-only AVTXFlags
   bit. When passed to av_tx_init(), the codelet selection loop in
   ff_tx_init_subtx() rejects any codelet declaring an arch-specific
   cpu_flags, leaving only the scalar C implementations eligible.
   The flag propagates through recursive ff_tx_init_subtx() calls so
   RDFT's sub-FFT is also forced to scalar. The flag is stripped
   from req_flags before the existing req_flags-vs-codelet match
   check so it does not interfere.

The scalar codelet path is gated by two new RFPTargets so callers
opt in only where the cross-architecture bit-identity matters and
the perf cost is acceptable:

• RFPTarget::WebAudioFFT (ID 81) — PeriodicWave (oscillator
  wavetable band-limiting) and AnalyserNode (getFloatFrequencyData
  output). Both nodes feed the User Characteristics audio
  fingerprint render and the Signifyd tracker's
  floatFrequencyDataSum fingerprint. FFTs happen at oscillator
  construction and rAF rate respectively, so the scalar slowdown
  is small.

• RFPTarget::WebAudioConvolverFFT (ID 82) — ConvolverNode
  (partitioned convolution). The FFT is invoked in the per-block
  hot loop; the scalar path carries a measurable per-block CPU
  cost (~50% on x86 Linux, ~25% on Windows, ~0% on Apple Silicon
  observed in the Raptor webaudio benchmark). Convolver is not on
  the known fingerprint surface, so this target is separated to
  let consumers opt in only when broader defense is warranted.

AudioContext captures both booleans on the main thread at
construction and exposes them via ShouldUseScalarFFT() and
ShouldUseScalarConvolverFFT(). Each FFTBlock construction site
reads the appropriate accessor and forwards the boolean through
FFTBlock's new constructor parameter. The BasicWaveFormCache,
WebCore::PeriodicWave factories, Reverb, ReverbConvolver,
ReverbConvolverStage, and FFTConvolver pipe the bool through their
constructors so audio-thread call sites can read a snapshot that
the main thread captured at AudioContext construction.

Verified Linux x86_64 opt vs macOS aarch64 opt produce identical
output for all 5000 samples across sine (no FFT), sawtooth,
triangle, and triangle+DynamicsCompressor+Analyser pipelines when
both RFP targets are enabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <[e-mail]>

Comment 4

[Tom Ritter [:tjr]]

Attachment: WIP: Bug 2040494 - Add WebAudio fingerprint cross-architecture mochitest.

Adds dom/media/webaudio/test/test_audioFingerprint_neonCollapse.html,
a regression test for the cross-architecture WebAudio fingerprint
bit-identity property. The test reproduces the
UserCharacteristicsPageService audio fingerprint render byte-for-byte
and probes four configurations to localize any future divergence:

sine_osc - ComputeSine + fdlibm_sinf, no PeriodicWave/FFT
sawtooth_osc - PeriodicWave + FFT (ComputeCustom)
osc - triangle PeriodicWave + FFT (ComputeCustom)
comp - triangle + DynamicsCompressor + Analyser pipeline
(matches usercharacteristics.js byte-for-byte)

For each probe the test emits a headline sum-of-absolute-values
integer (matching FP1), a Float32-bit-pattern FNV-1a 64-bit hash over
the same 500-sample window, and the bit-pattern of every individual
sample to BUG2040494-prefixed lines. Comparing the log lines between
runs on different architectures localizes any divergence to a specific
oscillator type and sample index.

Notes for future maintainers:

• dump() rather than info() is used because mochitest's info()
  output is filtered from live_backing.log on passing tests in CI,
  while dump() always reaches stdout.
• A pure-JS BigInt FNV-1a is used rather than crypto.subtle because
  mochitests run on http://mochi.test:8888 (insecure context) where
  crypto.subtle is unavailable.
• The test body is wrapped in try/catch/finally with
  SimpleTest.finish() in the finally block so any thrown error in
  the render still terminates the test cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <[e-mail]>

Comment 5

[Tom Ritter [:tjr]]

Alright, I dug into this quite a bit deeper. Using try, I was eventually able to make things bit-identical. The differences were arising from two places:

1. The use of trig functions setting up FFT tables. This was fixed fairly easily by using fdlibm functions
2. The use of optimized assembly in FFT paths

There is a no-assembly path that resolves the differences; obviously, it's slower.

The performance impact of this is measured here: https://perf.compare/compare-results?baseRev=f877796d2a4ec9b855b6a9c83762429e097cbe94&baseRepo=try&newRev=49a1833b6a5e4d44b8980edf7d4b372712e04cfe&newRepo=try&framework=13

In short, the Convolution reverb test takes the biggest hit: 40-80% regression. Most of other things see no change (some even see a small improvement.) Periodic Wave with Automation has a 10% regression.

Now it just so happens that all of the WebAudio fingerprint techniques don't actually use Convolution; so I've made a patch that can separately bypass the optimized assembly paths for Convolver vs the rest. This would let us enable WebAudioFFT in FPP (and later, potentially in bSFPP which is ETP Standard/default) and leave WebAudioConvolverFFT, off by default.

I've posted example patches - if this seems like an alright path forward, I will clean them up, including write a local patch file for ffvpx/libavutil and documenting it. (I might also try to add this to Updatebot and update it in general, unless that sounds like a bad idea.)

Comment 6

[Tom Ritter [:tjr]]

Attachment: WIP: Bug 2040494 - Add gtest measuring scalar-vs-SIMD FFT throughput in FFTBlock.

Adds dom/media/gtest/TestFFTBlockScalarPerf.cpp that times forward +
inverse RDFT throughput at size 8192 (the PeriodicWave default) using
av_tx_init() directly with two flag settings:

flags = 0 (arch-native SIMD; AVX2/SSE3 on
x86_64, NEON on aarch64)
flags = AV_TX_MOZ_SCALAR_ONLY (scalar C codelets only)

2000 iterations per pass, 3 passes per direction, taking the minimum
of each set to suppress CI host jitter. Logs a single line:

[FFTBlockScalarPerf] size=8192 iters=N simd=Xms scalar=Yms scalar/simd=Zx

The test is not a CI gate; it does not assert a specific ratio. The
log line is for human review when comparing runs across architectures
to decide whether the scalar default in FFTBlock remains acceptable
for production WebAudio paths. A companion BothPathsRoundTrip test
verifies that both flag settings correctly recover an arbitrary input
through forward + inverse RDFT.

Observed on Try push 1924964:

Linux x86_64 debug: simd ~67ms, scalar ~135ms (scalar/simd 2.00x)
macOS aarch64 opt: neon ~30ms, scalar ~54ms (scalar/simd 1.75x)

Apple Silicon scalar is faster in absolute terms than Linux SIMD.
Estimated production CPU impact at 44.1 kHz with 512-sample blocks
(~86 FFTs/sec, dominant ConvolverNode case) is ~1 ms/sec at 1.75x,
well below an audio-glitch threshold.

Co-Authored-By: Claude Opus 4.7 (1M context) <[e-mail]>

Comment 7

[Tom Ritter [:tjr]]

Attachment: WIP: Bug 2040494 - Run the webaudio Raptor benchmark on Apple Silicon macOS.

The browsertime-benchmark config has a per-platform subtest list and
the (macosx1400|macosx1500).shippable. matcher catches Apple Silicon
Mac hardware. That matcher's subtest list previously included
JetStream 2/3, Speedometer 2/3, and MotionMark 1.3 but not webaudio,
so the aarch64 macOS pool has had no WebAudio CPU perf coverage. The
remaining desktop platforms (Linux x86_64, Intel macOS, Windows x86_64)
all run webaudio through the catch-all default: subtest list, but
that list does not apply to macosx1400/macosx1500 because the
arch-specific matcher fires first.

Add [webaudio, wa] to the aarch64 subtest list so the benchmark runs
on Apple Silicon. The matching tier, run-on-projects (already
[mozilla-central] for macosx1400/1500), max-run-time (default 900s,
sufficient since webaudio is short), and mozharness extra-options
(picked up by the existing macosx1400/1500.shippable. mozharness
branch) all already accommodate the addition.

This is needed for Bug 2040494: PeriodicWave / Convolver / AnalyserNode
in WebAudio dispatch through the scalar (non-SIMD) FFT codelets in
ffvpx so cross-architecture output is bit-identical. Without aarch64
perf coverage we have no way to measure the NEON-vs-scalar gap in a
real WebAudio pipeline; the existing gtest microbenchmark covers raw
FFT throughput only.

Co-Authored-By: Claude Opus 4.7 (1M context) <[e-mail]>

Comment 8

[Tom Ritter [:tjr]]

Attachment: WIP: Bug 2040494 - Run the webaudio Raptor benchmark on Android.

The mobile browsertime kind (taskcluster/kinds/browsertime/mobile.yml)
previously scheduled only Speedometer 2/3, MotionMark 1.3, JetStream
2/3, and Unity-WebGL on Android. The webaudio benchmark had never run
on Android in CI even though it is part of the standard desktop
browsertime suite.

This matters for Bug 2040494: Android aarch64 uses the same
libavutil/aarch64/tx_float_neon.S NEON FFT kernel that macOS aarch64
does, so the scalar-vs-NEON perf delta from the
AV_TX_MOZ_SCALAR_ONLY change applies on Android too. Without Android
coverage we have no signal whether the scalar FFT default in FFTBlock
introduces a regression on the aarch64 hardware Firefox ships to
users.

Adds:

• taskcluster/kinds/browsertime/mobile.yml — a new
  browsertime-benchmark-webaudio-mobile task block, modeled on
  browsertime-benchmark-unity-webgl-mobile. fenix at tier 2,
  other apps tier 3, scheduled mozilla-central on the
  android-hw-a55 device pool.

• testing/raptor/raptor/tests/benchmarks/webaudio-mobile.toml —
  Raptor benchmark config for the mobile apps (fenix, geckoview,
  chrome-m), mirroring webaudio.toml (which is desktop-only) with
  the same preferences and 5-cycle page setup, following the
  jetstream2 / jetstream2-mobile naming convention.

• testing/raptor/raptor/raptor.toml — include the new mobile toml.

The legacy 32-bit Android armv7 path uses ffvpx's C scalar codelets
already (media/ffvpx/libavutil/arm/ contains no tx_float*.S), so this
change is most relevant on aarch64-class Android hardware.

Co-Authored-By: Claude Opus 4.7 (1M context) <[e-mail]>