2043895 - mediaCapabilities.decodingInfo is still slow

Status: UNCONFIRMED

(Core :: Audio/Video, defect, P2)

Description

[bugzilla.mozilla.org.wireless475]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Gecko/20100101 Firefox/152.0

Steps to reproduce:

After triaging an issue for playback starting much slower on Firefox than Chromium with Shaka player. I arrived at https://bugzilla.mozilla.org/show_bug.cgi?id=1676902, which seems to be closed despite the performance still not being usable in some cases.

Quick test:

(async () => {
    let N = 30;
    let ms = t => t.toFixed(2);
    let base = {
        type: 'media-source',
        video: {
            contentType: 'video/mp4; codecs="avc1.640028"',
            width: 1920, height: 1080, framerate: 30,
            bitrate: 5_000_000,
        },
    };

    // 1. cold: first call after page load (RDD warmup included)
    let t0 = performance.now();
    await navigator.mediaCapabilities.decodingInfo(structuredClone(base));
    console.log(`cold:     ${ms(performance.now() - t0)} ms`);

    // 2. uncached: vary bitrate per call
    t0 = performance.now();
    for (let i = 0; i < N; i++) {
        let c = structuredClone(base);
        c.video.bitrate = 5_000_000 + i + 1;
        await navigator.mediaCapabilities.decodingInfo(c);
    }
    console.log(`uncached: ${ms((performance.now() - t0) / N)} ms/call  (N=${N})`);

    // 3. cached: same config repeatedly
    let fixed = structuredClone(base);
    fixed.video.bitrate = 7_500_000;
    await navigator.mediaCapabilities.decodingInfo(fixed);  // prime
    t0 = performance.now();
    for (let i = 0; i < N; i++) {
        await navigator.mediaCapabilities.decodingInfo(fixed);
    }
    console.log(`cached:   ${ms((performance.now() - t0) / N)} ms/call  (N=${N})`);
})();

Actual results:

Chromium results:

cold:     1.60 ms
uncached: 0.09 ms/call  (N=30)
cached:   0.07 ms/call  (N=30)

Firefox Nightly Version 153.0a1 (2026-05-30) (64-bit) results:

cold:     16.00 ms
uncached: 18.87 ms/call  (N=30)
cached:   17.33 ms/call  (N=30)

Expected results:

Firefox should have more comparable performance. In some cases playback was delayed by tens of seconds because of the volume of available stream types that had to be checked.

Comment 1

[media-triaging-bot]

Thanks for the detailed benchmark and for tracking this back to bug 1676902.

I can confirm the cost and I have looked into where it comes from. For a non-DRM video configuration, navigator.mediaCapabilities.decodingInfo() currently instantiates a real MediaDataDecoder for each query -- it creates the decoder, calls Init(), reads the hardware-acceleration status, then shuts it down. On Windows the H.264 decoder is a Media Foundation decoder set up across the RDD/utility process, which dominates the ~17 ms you measured, and the per-call work is serialized, so Shaka's many parallel probes queue up. The result is not cached, which is why your cached run costs the same as uncached. (Bug 1676902's earlier improvement was on the codec-agnostic/WebM path, so it never touched this Windows H.264 instantiation cost.)

The key observation: of the three result fields, supported already comes from a cheap in-process capability lookup and smooth is a constant -- only powerEfficient currently needs the live decoder, and only to read its hardware-accelerated flag. The WebRTC branch of this same API already computes powerEfficient without instantiating a decoder. So the proposed fix is:

1. Compute powerEfficient from the capability/HW-decode signal instead of spinning up and tearing down a decoder, mirroring the WebRTC path. (One dependency: in the content process the remote support query does not yet distinguish HW vs SW decode -- tracked by bug 1754239 -- so the HW-accel signal has to come from the same source the WebRTC path uses, or ride on that bug.)
2. Add a configuration-keyed result cache as a complementary safety net, so repeated identical probes (exactly Shaka's pattern) are effectively free.

This is a real performance problem worth fixing. Ashley (:az) — adding a needinfo to see whether you'd like to take this, since it sits squarely in your Media Capabilities area. If you don't have the bandwidth, we can route it to :alwu instead. Keeping S3/P2 — functional, but a meaningful startup-latency regression vs. Chromium on Windows DASH/Shaka.