1766037 - H264 codec isn't available in fresh profiles

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P3)

Description

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Error: Couldn't find offset 0 of codec H264 while looking for offset 0 in sdp

While trying to repro various issues around WebRTC using H264, I came across this error a lot. I tried doing a mozregression on it, since it appeared to either happen every time or not at all, but it seems arbitrary as none of the ranges make sense.

Maybe this is could be some unreported failure of failing to pull down the openH264 plugin, since mozregression I believe starts from the same profile starting point by default? But aren't we supposed to be using MS

STRs (I used a Logitech camera which gave me a 1920x1080 camera source. YMMV):

1. Open https://jsfiddle.net/jib1/02j8q6h9/show (leave ☑h264 checked)
2. Hit the gUM! button (and allow camera)

Expected results:

• two videos appear (receiver and sender)

Actual result:

 Error: Couldn't find offset 0 of codec H264 while looking for offset 0 in
sdp:
v=0 o=mozilla...THIS_IS_SDPARTA-99.0 6807228323487549301 0 IN IP4 0.0.0.0 s=- t=0 0 a=fingerprint:sha-256 E5:A3:84:D9:A0:FD:2F:5D:76:B1:8F:1C:A6:27:B8:EC:D4:94:43:FF:02:05:0B:31:F2:87:9D:93:00:84:13:EC
a=group:BUNDLE 0
a=ice-options:trickle
a=msid-semantic:WMS *
m=video 9 UDP/TLS/RTP/SAVPF 120 124 121 125
c=IN IP4 0.0.0.0
a=sendrecv
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
a=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:124 apt=120
a=fmtp:121 max-fs=12288;max-fr=60
a=fmtp:125 apt=121
a=ice-pwd:11cba9cf8403403446cf3db31730584b
a=ice-ufrag:31cb9670
a=mid:0
a=msid:- {36d47d96-b735-4d9a-9edd-904d17d15f69}
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:120 transport-cc
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:121 transport-cc
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:120 VP8/90000
a=rtpmap:124 rtx/90000
a=rtpmap:121 VP9/90000
a=rtpmap:125 rtx/90000
a=setup:actpass
a=ssrc:1870026881 cname:{58bd6b8c-668b-434b-af3b-02d002e28a80}
a=ssrc:4270559034 cname:{58bd6b8c-668b-434b-af3b-02d002e28a80}
a=ssrc-group:FID 1870026881 4270559034

Only observed on Windows 10.

Comment 1

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Note the part of the fiddle producing the error is the following. Including it here to clarify this is the local offer:

    const offer = await pc1.createOffer();
    await pc1.setLocalDescription(offer);
→   offer.sdp = preferCodec(offer.sdp, h264.checked? "H264" : "VP8");

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #0)

Maybe this is could be some unreported failure of failing to pull down the openH264 plugin, since mozregression I believe starts from the same profile starting point by default? But aren't we supposed to be using MS

According to bug 1741244 we're supposed to "use Microsoft Media Foundation H.264 video encoder for WebRTC"... Does that mean we're using it for decoding as well, or are we still reliant on OpenH264 for that? John, do you know?

Comment 2

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Although both encoder and decoder are needed here since this is a local loop, so maybe my question is moot. In any thoughts on why this is happening like this is appreciated.

Comment 3

[John Lin [:jhlin][:jolin]]

I couldn't reproduce it locally on my win10 machine. Could you please turn on the PlatformEncoderModule log module and look for MFTEncoder? It should show if the platform encoder is available or not. Thanks!

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

The fiddle attempts to munge SDP to prefer H264, which involves looking for the H264 payload type, and when it isn't there (as shown in comment 0), it errors before ever establishing a connection, so no MFTEncoder messages ever get logged then. Is there any other info I should look at?

Note this is happening most reliably for me using MozRegression GUI.

In contrast, in case it is helpful, on a successful (non-MozRegressionGUI) run, it logs:

[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::PopulateEncoderInfo: <ENC> [H.264] H264 Encoder MFT
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::PopulateEncoderInfo: <ENC> [VP9] VP9VideoExtensionEncoder
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::PopulateEncoderInfo: <ENC> [VP8] VP9VideoExtensionEncoder
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Request platform encoder for video/vp8, bitRate=1200000 bps, frameRate=60
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: D/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Init encode, mimeType video/vp8, mode N/A
[Child 3224: MediaPEncoder #1]: D/PlatformEncoderModule MFTEncoder::EnumEncoders: Don't use HW encoder when win32k locked down.
[Child 3224: WebrtcWorker #2]: V/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Encode frame, type 3 size 2073600
[Child 3224: WebrtcWorker #2]: V/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Encode frame, type 4 size 2073600
[Child 3224: WebrtcWorker #2]: V/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Encode frame, type 4 size 2073600
[Child 3224: MediaSupervisor #1]: V/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Received encoded frame, nums 1 width 1920 height 1080
[Child 3224: MediaSupervisor #1]: V/PlatformEncoderModule WebrtcMediaDataEncoder=2520f84f300, Send encoded image

Comment 5

[Jim Mathies [:jimm]]

Jeff repro'd this on a M1 Mac, Jib on Windows.

Comment 6

[Jim Mathies [:jimm]]

I can't get Firefox to trigger download and install of OpenH264 through webrtc usage, or the install is so slow that it takes a long time. Also there is no UI as we have with the Widevine CDM. This is a poor user experience.

STR:

1. create a fresh profile through about:profiles
2. launch a browser in that profile
3. open about:addons / plugins
4. open about:profiles, and open the profile folder
5. navigate to https://jsfiddle.net/jib1/02j8q6h9/show, click gUM!, approve camera access

Result:
Error: Couldn't find offset 0 of codec H264 while looking for offset 0 in sdp, and there's no receiver video
Check about:addons, the install is pending.
Check the profile folder, no gmp plugin folder is created.

After two or three restarts and about five minutes, everything work.

Comment 7

[Jim Mathies [:jimm]]

Other notes -

I tried again and enabled gmp logging via the media.gmp.log.level=5 setting. Nothing showed up related to updates. I restarted and checked the log again, still nothing, and openh264 download wasn't triggered. Letting the browser sit for a bit to see if idle triggers it.

Comment 8

[Jim Mathies [:jimm]]

After about a minute of idle time, the plugin came down and everything works.

There are two bugs here - 1) attempted use doesn't trigger the download, and 2) no UI.

My bet is this has been this way for ages so not a recent regression.

Media team has a backlogged project revolving around improving gmp plugin UI that would help here.