FF34 breaks Video with H264 on some sites that worked with FF33

RESOLVED INVALID

Status

()

defect
P1
critical
RESOLVED INVALID
5 years ago
5 years ago

People

(Reporter: ehugg, Unassigned)

Tracking

unspecified
mozilla34
x86
macOS
Points:
---

Firefox Tracking Flags

(firefox34 wontfix, firefox35 affected, firefox36 unaffected, firefox37 affected)

Details

Reporter

Description

5 years ago
Two reports of using H264 that worked in FF33 that shows a black screen on the reciever end for FF34.

1. Project Squared - web.projectsquared.com
2. Report from aslan.l@amaryllo.eu pasted to mozilla-dev mailing list - full text pasted below.

The pc_test.html loopback test works, but something has changed and hopefully there is a way they can adapt these sites.
Reporter

Comment 1

5 years ago
Posted on dev-media mailing list by aslan.l:

I posted an issue about using H264 stream on FF 33 before (https://groups.google.com/forum/#!searchin/mozilla.dev.media/h264/mozilla.dev.media/Hbh3P1qGW9Y/Y4Ji2T6w6awJ), and that's is solved by everyone's help! Thanks.
But.... when FF is upgraded to 34, this is happened again.. (OMG).

So I come here again, and hope anyone can give me help~ thanks.
Here is my test environment:

    My Device <---H264---> FF 34 (No video)
    My Device <---H264---> FF 33 (Has video)

And here are offer and answer:

Offer(FF to My Device)
v=0
o=Mozilla-SIPUA-34.0 14896 0 IN IP4 0.0.0.0
s=SIP Call
t=0 0
a=ice-ufrag:867d948c
a=ice-pwd:12e3c549e7c1aa5b62d19c5ba31d8a90
a=fingerprint:sha-256 B9:4B:EE:66:4F:B6:A6:FD:3E:78:43:36:9A:FE:E6:FD:B1:A4:50:82:B8:8C:A0:02:7E:E3:66:FC:66:57:99:B5
m=audio 9 RTP/SAVPF 109 9 0 8 101
c=IN IP4 0.0.0.0
a=rtpmap:109 opus/48000/2
a=ptime:20
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=setup:actpass
a=rtcp-mux
m=video 9 RTP/SAVPF 120 126 97
c=IN IP4 0.0.0.0
a=rtpmap:120 VP8/90000
a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=42e01f;packetization-mode=1
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42e01f
a=recvonly
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=setup:actpass
a=rtcp-mux

Answer(My Device to FF)
v=0
o=- 3637702335846616252 2 IN IP4 127.0.0.1
s=-
t=0 0
a=msid-semantic: WMS stream_label
m=audio 1 RTP/SAVPF 0 8 101
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:dcIFiIhOR+fZHFY9
a=ice-pwd:mLqKK3C5It1/Z0L+SWWYrj+y
a=fingerprint:sha-1 6C:FF:C0:2F:CA:C8:FD:44:88:DE:DB:2E:09:9D:1A:50:D8:B9:FD:D8
a=setup:active
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=sendrecv
a=rtcp-mux
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=ssrc:2594247746 cname:g07iau/woWmpLmF5
a=ssrc:2594247746 msid:stream_label audio_label
a=ssrc:2594247746 mslabel:stream_label
a=ssrc:2594247746 label:audio_label
m=video 1 RTP/SAVPF 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:OVnd8UkxhD6jqb4W
a=ice-pwd:8bvK+cEwVy/UPE9f+Qv/LIJc
a=fingerprint:sha-1 6C:FF:C0:2F:CA:C8:FD:44:88:DE:DB:2E:09:9D:1A:50:D8:B9:FD:D8
a=setup:active
a=mid:video
a=sendonly
a=rtcp-mux
a=rtpmap:126 H264/90000
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=ssrc:3920187810 cname:g07iau/woWmpLmF5
a=ssrc:3920187810 msid:stream_label video_label
a=ssrc:3920187810 mslabel:stream_label
a=ssrc:3920187810 label:video_label

Has there any strange things that might cause this issue?
[Tracking Requested - why for this release]: regression in 34
Byron -- Can you take a look at the SDP in Comment 1 and comment if you see anything that's a problem?  

Also, can you work with Ethan and Jesup to get more details from Cisco on what problem they are seeing?  (I want to verify that the problem Cisco is reporting is the same as the one reported in the dev-media post.)
Flags: needinfo?(docfaraday)
Severity: normal → critical
Priority: -- → P1
Target Milestone: --- → mozilla34
Reporter

Comment 4

5 years ago
This can be analyzed on web.projectsquared.com by creating two accounts and connecting them with a video call.  Looks like it works fine on FF33 and on FF34 it connects but shows blank video.  Not sure it's related to the SDP since the data seems to flow according to about:webrtc.
The striking thing about that answer SDP is the lack of an fmtp attribute. It is likely that we're interpreting the lack of fmtp as packetization-mode=0, which is of course not what was offered on payload type 126.
Flags: needinfo?(docfaraday)
Reporter

Comment 6

5 years ago
I set the pref "media.navigator.video.preferred_codec" to 126 which makes it prefer H264 in negotiations and ran Firefox Hello, verified that it used H264, and the video worked fine in both directions, so the ability to do H264 is working.
Reporter

Comment 7

5 years ago
Using web.projectsquared.com.

about:webrtc from successfull FF33 call:
http://pastebin.mozilla.org/7724786

about:webrtc from call on FF34 which has a blank recv screen:
http://pastebin.mozilla.org/7724799
Does audio work on that squared call either? Because maybe what we're actually seeing here is a transport setup interop problem, since that changed somewhat in 34.
Flags: needinfo?(ethanhugg)

Comment 9

5 years ago
Audio works on my side, but Video doesn't. Here are some information about audio and video in about:webrtc :

inbound_rtp_audio_1
A/V sync: 0 ms Jitter-buffer delay: 229 ms
Local: 10:45:25 GMT+0800 inboundrtp SSRC: 3275023840 Received: 2162 packets (380.04 Kb) Lost: 0 Jitter: 0
Remote: 10:45:46 GMT+0800 outboundrtp SSRC: 3275023840 Sent: 2068 packets (323.13 Kb)
inbound_rtp_video_2
Decoder: Avg. bitrate: 0.00 Mbps (0.00 SD) Avg. framerate: 0.00 fps (0.00 SD) Discarded packets: 0
Local: 10:45:25 GMT+0800 inboundrtp SSRC: 0 Received: 3112 packets (3004.44 Kb) Lost: undefined Jitter: undefined

So FF34 is actually receiving data from my device, but Decoder doesn't work. Here is the full information https://mega.co.nz/#!VJFAyLKY!mRc2qN6l1DpGk11wBJnfEtq3OlNpu2naAePEmDvBT80 . 

In addition, I think "packetization-mode=0" is not the problem. Because I also try to change my code to let my device return "a=rtpmap:97 H264/90000" in answer, not works too.
Reporter

Comment 10

5 years ago
I heard no audio on the FF34 side although it was sending OK.  I built a debug version of mozilla-release and put a nspr log of signaling:6,GMP:5,webrtc_trace:65535,timestamp here:
https://drive.google.com/folderview?id=0B3jogUA4XhcTOE9ubDBsb044VjA&usp=sharing
shared with docfaraday and can share with anyone else who wishes to take a look.  It's too big to upload here.
Flags: needinfo?(ethanhugg)

Comment 11

5 years ago
I build an Android App for this issue. The source code of this Android App can be found on https://code.google.com/p/webrtc/source/browse/#svn%2Ftrunk%2Ftalk%2Fexamples%2Fandroid%253Fstate%253Dclosed . And the apk file can be downloaded from https://mega.co.nz/#!FI0XDYxI!4S0zdBC94Qv_jIu2tFBPmx8gCPm_VtUVRI97YYkITxI .

Here explain how to use:

Phone side
1. Install this apk on your Android Phone
2. Open AppRTC application, enter your room number at the end of url. For example, https://raymondwebrtc.appspot.com/?r=5555 . (room number can be any digital number as you want, here I set it as 5555 )
3. Click "Go" button

PC side
1. Open your FF34, and enter the url as the previous mentioned. Example, https://raymondwebrtc.appspot.com/?r=5555 . Only need do this.

By the previous example, PC will send offer to phone, and phone will return answer to FF34. That doesn't work. But......

If you open FF34 and connect to a new room, such as https://raymondwebrtc.appspot.com/?r=6666, and then open AppRTC application to connect this room, that works!!!!

So.. I find:
FF34 ==Offer==> Phone ==Answer==> FF34  (Not work)
Phone ==Offer==> FF34 ==Answer==> Phone (Work)

Hope this can help for fixing this issue.
(In reply to aslan.l from comment #9)
> Audio works on my side, but Video doesn't. Here are some information about
> audio and video in about:webrtc :
> 
> inbound_rtp_audio_1
> A/V sync: 0 ms Jitter-buffer delay: 229 ms
> Local: 10:45:25 GMT+0800 inboundrtp SSRC: 3275023840 Received: 2162 packets
> (380.04 Kb) Lost: 0 Jitter: 0
> Remote: 10:45:46 GMT+0800 outboundrtp SSRC: 3275023840 Sent: 2068 packets
> (323.13 Kb)
> inbound_rtp_video_2
> Decoder: Avg. bitrate: 0.00 Mbps (0.00 SD) Avg. framerate: 0.00 fps (0.00
> SD) Discarded packets: 0
> Local: 10:45:25 GMT+0800 inboundrtp SSRC: 0 Received: 3112 packets (3004.44
> Kb) Lost: undefined Jitter: undefined
> 
> So FF34 is actually receiving data from my device, but Decoder doesn't work.
> Here is the full information
> https://mega.co.nz/#!VJFAyLKY!mRc2qN6l1DpGk11wBJnfEtq3OlNpu2naAePEmDvBT80 . 
> 
> In addition, I think "packetization-mode=0" is not the problem. Because I
> also try to change my code to let my device return "a=rtpmap:97 H264/90000"
> in answer, not works too.

That link is not working for me. This is just a small PDF, just attach to the bug.

Comment 13

5 years ago
Hi all,

I find the cause of this problem, but I haven't any idea how to patch it. 
Firstly, in ViEReceiver::ReceivePacket function, it will always return false at here:

rtp_payload_registry_->GetPayloadSpecifics(header.payloadType,
                                                  &payload_specific)

Because the variable "payload_type_map_" only has two codecs, h264 and vp8. However, the problem is the id of the payload type of H264 is 97, not 126. In sdp, FF34 tells it provides two types of H264 type, 97 and 126, but 126 is not registered in payload_type_map_. That why FF34 can not play video from my device, because my device send type is 126, not 97.

On my side, if I set payload type is 97 both in sdp and rtp header, and then FF34 works. However, the problem still exists: Why the payload type(126) isn't registered in payload_type_map_ , but it exists in sdp. 

Anyone can help it?
Reporter

Comment 14

5 years ago
(In reply to aslan.l from comment #13)
> Hi all,
> 
> I find the cause of this problem, but I haven't any idea how to patch it. 
> Firstly, in ViEReceiver::ReceivePacket function, it will always return false
> at here:
> 
> rtp_payload_registry_->GetPayloadSpecifics(header.payloadType,
>                                                   &payload_specific)
> 
> Because the variable "payload_type_map_" only has two codecs, h264 and vp8.
> However, the problem is the id of the payload type of H264 is 97, not 126.
> In sdp, FF34 tells it provides two types of H264 type, 97 and 126, but 126
> is not registered in payload_type_map_. That why FF34 can not play video
> from my device, because my device send type is 126, not 97.
> 
> On my side, if I set payload type is 97 both in sdp and rtp header, and then
> FF34 works. However, the problem still exists: Why the payload type(126)
> isn't registered in payload_type_map_ , but it exists in sdp. 
> 
> Anyone can help it?


Aslan,

projectsquared.com does not have this problem with 97/126. Any chance on this last test the SDP answer listed both 126 and 97?  That sounds like this bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1073475
For now any answer needs to list only one.

Comment 15

5 years ago
Here is the part of the answer from projectsquared.com

a=rtpmap:126 H264/90000
a=fmtp:126 PROFILE=0;LEVEL=0;profile-level-id=42000D;packetization-mode=1;level-asymmetry-allowed=1;max-mbps=27600;max-fs=920;max-dpb=891;max-br=1000;parameter-add=1;usedtx=0;stereo=0;useinbandfec=0;cbr=0

And the answer from my device is 

a=rtpmap:126 H264/90000

No matter projectsquared.com or mime, the answer only lists one video format, H264/126. In addition, packetization-mode isn't a point of this issue, because the answer form my device doesn't response packetization-mode to FF34.

Need more investigate on this issue.
(In reply to aslan.l from comment #15)

> And the answer from my device is 
> 
> a=rtpmap:126 H264/90000
> 
> No matter projectsquared.com or mime, the answer only lists one video
> format, H264/126. In addition, packetization-mode isn't a point of this
> issue, because the answer form my device doesn't response packetization-mode
> to FF34.

That's an invalid response, even if it used to work.  The default for packetization-mode is 0.  126 with packetization-mode 1 does not match 126 with packetization mode 0 - the numbers are not how it matches, it actually is supposed to match on packetization-mode.  (You can answer an offer of m=video .... 123 with m=video ... 99 (or any other number) for H.264 IF the packetization modes match (and if mode=2, dpb and there are a few other rules for that IIRC - but mode 2 is never used for interactive).

Also: you must send the correct format that you accepted - and if you said 126 with no fmtp, you *really* accepted 97 (mode 0) and MUST send 97 (and will get 126 mode 0).  This can be very confusing (though correct).  Don't do this to yourself....

Comment 17

5 years ago
Yes, I just realized I shall accept "97" not "126 without packetization-mode" when this issue happened on FF34. But still can not explain why projectsquared.com doesn't work. Only thing we know that the problem in projectsquared.com is not the same as mine.
Reporter

Comment 18

5 years ago
projectsquared.com works well on FF34 on certain networks so it probably has a different problem than yours.
Reporter

Comment 19

5 years ago
I don't think there's a Firefox bug here.   Squared has been patched by fixing their JS.  The timing of WebRTC events is different between FF33 and FF34 and it exposed a race condition in their JS code.  The fix should be pushed out to their production soon.

From comments 13-17 I believe that Aslan's problem was different and was related to not putting packetization-mode=1 in the answer.

So closing this bug as invalid.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID
Since this is invalid, I'm clearing the tracking flags so it doesn't mess up searches.
You need to log in before you can comment on or make changes to this bug.