Open Bug 1832276 Opened 1 year ago Updated 11 months ago

VP9 SVC only lowest layer is decoded with MediaDataDecoder

Categories

(Core :: WebRTC: Audio/Video, defect, P3)

Product:
Core
Component:
WebRTC: Audio/Video
Version:
Firefox 112
Type:
defect

Tracking

()

People

(Reporter: florian.limberger, Assigned: pehrsons)

References

(Depends on 1 open bug)

Details

Attachments

(1 file)

webrtc test setup where sdp must be copied manually for call setup
33.82 KB, text/html
Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0

Steps to reproduce:

Stream a VP9 SVC video stream with scalability mode L2T2 (640x480,320x240) via webrtc from goole-chrome (112.0.5615.165) to firefox.

Actual results:

Firefox only decodes the lowest scalability layer (i.e. 320x240).

Expected results:

Decodes 640x480 (i.e. combines both layers).

The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Graphics
Product: Firefox → Core
Component: Graphics → WebRTC: Audio/Video

Thanks for reporting! Even though we don't support webrtc-svc I agree being able to receive the full stream would be useful.

Do you have a page to reproduce this issue I could use to debug? Also, on a hunch, does toggling the media.navigator.mediadatadecoder_vpx_enabled pref have anything to say here?

Flags: needinfo?(florian.limberger)

Indeed, setting media.navigator.mediadatadecoder_vpx_enabled to false does fix the problem. So it seems to be an issue related to hardware acceleration, right? Please let me know, if you still need a page for reproduction.

Flags: needinfo?(florian.limberger)

Ok, this means that the problem is somewhere in our MediaDataDecoder stack. If your machine has a vp9 hardware decoder it could be specific to that, but it could perhaps be more generic and apply to the software decoder too. I'd have to debug (thus reproduce) to know for sure.

If you can create a page for reproduction reasonably easy that would be great. Since we don't support encoding svc it gets a bit tricky to test in-tree.

Severity: -- → S4
Status: UNCONFIRMED → NEW
Ever confirmed: true
Priority: -- → P3
Summary: VP9 SVC only lowest layer is decoded. → VP9 SVC only lowest layer is decoded with MediaDataDecoder

I'll provide a page for reproduction next week.

Checking for another decoder here, could you set media.navigator.mediadatadecoder_vpx_enabled back to true and then set both media.ffvpx.enabled and media.ffmpeg.enabled to false? This will use our own libvpx decoder rather than libwebrtc's, and so should let us know whether it's a decoder issue or something earlier in the pipe.

In order to reproduce, use the following approach:

  • Open the file in firefox and chrome
  • Start with chrome first
  • Hit "Start media"
  • Select "vp9 profile-0" and L2T2 for SVC
  • Hit "Start connection"
  • Then hit "Copy offer"
  • Open file in firefox
  • Hit "Start media"
  • Paste SDP from clipboard into right text-box
  • Hit "Apply offer"
  • Hit "Copy answer"
  • Go back to chrome and Paste answer into right text-box
  • Hit "Apply answer"

Now the webrtc-call should be setup. Chrome generates an SVC video stream which consists of a 320x240 and a 640x480 layer. As already described if the flag media.navigator.mediadatadecoder_vpx_enabled is set to true, one will see the 320x440 stream only.
In order to verify, go to "about:webrtc" and check the "Video Frame Statistics". Please note, that the Synthetic-Videostream will only generate frames, if the chrome browser has the focus.

(In reply to Andreas Pehrson [:pehrsons] from comment #6)

Checking for another decoder here, could you set media.navigator.mediadatadecoder_vpx_enabled back to true and then set both media.ffvpx.enabled and media.ffmpeg.enabled to false? This will use our own libvpx decoder rather than libwebrtc's, and so should let us know whether it's a decoder issue or something earlier in the pipe.

Had no effect.

The recording shows weird timestamps being extrapolated, as if we're not able to process/decode frames fast enough. It might be because of the non-opt debug (i.e. slow) build, because logging in nightly does not indicate the same symptom.

Testing with the prefs again (a few more needed to be flipped than those mentioned in comment 6) I get:

[1] Per chromium, Apple's VideoToolbox needs superframes to be assembled so perhaps we need a similar filter.
[2] I'm not sure what ffmpeg expects here but it contains a similar bitstream filter to the one above, that for ffvpx we do not compile.

Depends on: 1837228
See Also: → 1858904
Depends on: 1859880
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: