Cannot communicate audio in a P2P call from Taipei's Mozilla Guest network to a home network

RESOLVED DUPLICATE of bug 950660

Status

()

Core
WebRTC: Networking
RESOLVED DUPLICATE of bug 950660
5 years ago
3 years ago

People

(Reporter: jsmith, Unassigned)

Tracking

29 Branch
ARM
Gonk (Firefox OS)
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
Build Environment 1:

Build Date - 1/12/2014 Buri 1.4
Network - Mozilla-G wifi network in Taipei

Build Environment 2:

Build Date - 1/16/2014 Buri 1.3
Network - Home wifi network in US

STR

1. Have BE 1 go to apprtc.appspot.com, accept permissions
2. Have BE 2 go to the apprtc URL seen in BE 1, accept permissions
3. Try talking back & forth vocally

Expected

Audio should be sent & received between each peer.

Actual

The call successfully establishes, but audio cannot be sent between each peer.

Note - working on getting logs. If this needs to be reproduced again, then have one of the Taipei developers do an audio call to Naoki Hirata over Taipei's Mozilla-G wifi network to Naoki's wifi network.
(Reporter)

Updated

5 years ago
blocking-b2g: --- → 1.3?
(Reporter)

Comment 1

5 years ago
One correction - the Taipei's wifi network is Mozilla Guest, not Mozilla-G. I forgot we don't support connecting to Mozilla-G & Mozilla wifi networks on FxOS right now.
Summary: Cannot communicate audio in a P2P call from Taipei's Mozilla-G network to a home network → Cannot communicate audio in a P2P call from Taipei's Mozilla Guest network to a home network

Comment 2

5 years ago
Steven, please check this problem first
Flags: needinfo?(slee)
(Reporter)

Comment 3

5 years ago
Created attachment 8361495 [details]
BE 1 Wireshark Trace

First example of a trace from BE 1
Created attachment 8361497 [details]
BE 2 Wireshark Trace

Here's the other side's Wireshark Trace log.
(Reporter)

Updated

5 years ago
Attachment #8361495 - Attachment description: BE 1 Wireshark Trace (Example 1) → BE 1 Wireshark Trace
(Reporter)

Comment 5

5 years ago
FWIW - Steven mentioned in person that audio communication works fine on Taipei Guest --> Taipei Guest. He's going to dive into the wireshark trace to analyze the bug more.

Comment 6

5 years ago
(In reply to Jason Smith [:jsmith] from comment #5)
> FWIW - Steven mentioned in person that audio communication works fine on
> Taipei Guest --> Taipei Guest. He's going to dive into the wireshark trace
> to analyze the bug more.
I tried on desktop browser and device under Taipei Guest and worked fine. I will try both on devices. If there is any updates, I will post it. BTW, I checked the wireshark log and found that there are only stun packets.
Flags: needinfo?(slee)
Note from email this was only attempting audio as the 1.3 device doesn't support video.  When the taipei side was at a hotel, it works.  This implies it's related to the network characteristics of the Taipei-Guest network's firewall.

It looks like naoki's side got STUN responses.  I can't find any indication that the "callingside" trace (taipei) ever got *any* response to STUN requests.  Note that I might not see/understand TURN-TCP, though I don't see any TCP SYN packets after taipei starts sending STUN UDP.

nicer logs (R_LOG_LEVEL etc) would be more useful, plus logs from nspr for signaling:5,mtransport:5 would be good too.

Does this reproduce with one end (either one) being a desktop running 29?  Both ends being a desktop?  Both tests would be good first-level checks of the network before getting too deep into whether it's a B2G issue.  In either case, logs from the desktop(s) would help a lot in debugging this and a lot easier to get.

Note that Taipei Guest -> Taipei-Guest are very likely to work.

See https://wiki.mozilla.org/Media/WebRTC/Logging
(Reporter)

Comment 8

5 years ago
(In reply to Randell Jesup [:jesup] from comment #7)
> Note from email this was only attempting audio as the 1.3 device doesn't
> support video.  When the taipei side was at a hotel, it works.  This implies
> it's related to the network characteristics of the Taipei-Guest network's
> firewall.

To clarify - the hotel testing was done in North Carolina in which each peer was on the same network. I haven't tested a call to Naoki from the Taipei hotel.

> 
> It looks like naoki's side got STUN responses.  I can't find any indication
> that the "callingside" trace (taipei) ever got *any* response to STUN
> requests.  Note that I might not see/understand TURN-TCP, though I don't see
> any TCP SYN packets after taipei starts sending STUN UDP.
> 
> nicer logs (R_LOG_LEVEL etc) would be more useful, plus logs from nspr for
> signaling:5,mtransport:5 would be good too.
> 
> Does this reproduce with one end (either one) being a desktop running 29? 
> Both ends being a desktop?  Both tests would be good first-level checks of
> the network before getting too deep into whether it's a B2G issue.  In
> either case, logs from the desktop(s) would help a lot in debugging this and
> a lot easier to get.

I'll work with Naoki on this next week to do desktop --> fxos in each direction.

I'll leave qawanted on here to see if this bug reproduces from doing a desktop to fxos call from Taipei Guest to the home network. If it reproduces there, then we'll work on getting the NSPR logs from the desktop machine.
Keywords: qawanted

Comment 9

5 years ago
I tried that both fxos and desktop are under Taipei-Guest, the connection is fine. BTW, how can I use apprtc and construct audio only connection? My test is to force the media type to fake on fxos.
One more question, could we connect 2 peers between Mozilla-Guest and Mozilla?
(In reply to StevenLee[:slee] from comment #9)
> I tried that both fxos and desktop are under Taipei-Guest, the connection is
> fine. 

Right - testing from within the same firewall doesn't test any significant part of ICE or TURN, and with this sort of failure that's the first place to look (and if I read things correctly, all UDP STUN requests failed).  If TCP turn was working on both of those devices, it *should* have worked; I didn't see it try in the wireshark, but I may have missed it.

jsmith's test within the same hotel network may or may not have used ICE/TURN - some hotel networks like the one we had at the Sunnyvale work week block peer-2-peer connections within the firewall, even if they use the external IP (hairpinning).

> BTW, how can I use apprtc and construct audio only connection? My test
> is to force the media type to fake on fxos.

You have to deny access to video I think (for 1.4 or desktop).  It's not designed for audio-only calls, so it's painful.

> One more question, could we connect 2 peers between Mozilla-Guest and
> Mozilla?

In Mountain View?  I assume so, at least using TURN. I'm not there so I haven't done the test myself, but I'm certain it's been done.
(Reporter)

Comment 11

5 years ago
Btw - one idea I was thinking that would help with debugging is to have Steven do an audio P2P call to Naoki sometime next week with Steven's device setup for whatever debug logging he needs. That would help figure out what's not working here.
Important note: B2G does not currently have support for TURN TCP.
See https://bugzilla.mozilla.org/show_bug.cgi?id=950660
If you are behind a firewall that blocks UDP,
B2G will fail and desktop will work. 
(this is not a blocker since we shipped FF for a long time without
TURN TCP). Based on Randell's analysis that sounds like the problem.
(Reporter)

Comment 13

5 years ago
(In reply to Eric Rescorla (:ekr) from comment #12)
> Important note: B2G does not currently have support for TURN TCP.
> See https://bugzilla.mozilla.org/show_bug.cgi?id=950660
> If you are behind a firewall that blocks UDP,
> B2G will fail and desktop will work. 
> (this is not a blocker since we shipped FF for a long time without
> TURN TCP). Based on Randell's analysis that sounds like the problem.

Just because we shipped desktop with the problem doesn't necessarily mean it's okay to ship FxOS with the problem. When desktop initially shipped, TURN wasn't supported in P2P. Now, TURN is supported & is actively used on the web in P2P apps, including on apprtc, which is a key demo application. We also know that TURN TCP landed in Gecko 28 in bug 906968, so this code path exists in FxOS 1.3. Additionally, we have at least two content partners looking to build two communication style apps on FxOS, so we have be conscious that P2P communication works extremely well to get partner buy in to build those apps. On that regard, I see that this problem is a content blocker for 1.3 with severe impact, given the fact that audio communication is failing & the fact this impacts a key demo application.

Note - one thing that would be interesting to look into is to figure out if the patch "turn off TURN TCP for gonk" isn't having negative impact to cause audio communication to fail here on FxOS.
(In reply to Jason Smith [:jsmith] from comment #13)
> (In reply to Eric Rescorla (:ekr) from comment #12)
> > Important note: B2G does not currently have support for TURN TCP.
> > See https://bugzilla.mozilla.org/show_bug.cgi?id=950660
> > If you are behind a firewall that blocks UDP,
> > B2G will fail and desktop will work. 
> > (this is not a blocker since we shipped FF for a long time without
> > TURN TCP). Based on Randell's analysis that sounds like the problem.
> 
> Just because we shipped desktop with the problem doesn't necessarily mean
> it's okay to ship FxOS with the problem.

It's not a problem. It's a missing feature.


> When desktop initially shipped,
> TURN wasn't supported in P2P. Now, TURN is supported & is actively used on
> the web in P2P apps, including on apprtc, which is a key demo application.
> We also know that TURN TCP landed in Gecko 28 in bug 906968, so this code
> path exists in FxOS 1.3.

Not quite. Part of the code path exists but because of the E10S split,
we need to add *extra* code to make TURN TCP work and that code isn't complete
(it's in the bug I pointed at in c12).


> Additionally, we have at least two content partners
> looking to build two communication style apps on FxOS, so we have be
> conscious that P2P communication works extremely well to get partner buy in
> to build those apps. On that regard, I see that this problem is a content
> blocker for 1.3 with severe impact, given the fact that audio communication
> is failing & the fact this impacts a key demo application.

Well, audio communication is hardly going to work any better if we turn
it off entirely. Note that even with TURN TCP there are still going to
be network environments where WebRTC fails (because they block TURN)
and where yet more aggressive proxying is needed. Moreover, consider
that if we had simply slipped TURN TCP to FF 30 we wouldn't even
be having this conversation.


> Note - one thing that would be interesting to look into is to figure out if
> the patch "turn off TURN TCP for gonk" isn't having negative impact to cause
> audio communication to fail here on FxOS.

See above.
(Reporter)

Comment 15

5 years ago
(In reply to Eric Rescorla (:ekr) from comment #14)
> > Additionally, we have at least two content partners
> > looking to build two communication style apps on FxOS, so we have be
> > conscious that P2P communication works extremely well to get partner buy in
> > to build those apps. On that regard, I see that this problem is a content
> > blocker for 1.3 with severe impact, given the fact that audio communication
> > is failing & the fact this impacts a key demo application.
> 
> Well, audio communication is hardly going to work any better if we turn
> it off entirely. Note that even with TURN TCP there are still going to
> be network environments where WebRTC fails (because they block TURN)
> and where yet more aggressive proxying is needed. Moreover, consider
> that if we had simply slipped TURN TCP to FF 30 we wouldn't even
> be having this conversation.

In that case - What we probably need to better understand here is to get a picture of the scope of wifi network environments that will be impacted by this bug that FxOS supports. 

Note - I know we support a more limited scope of wifi networks (e.g. we don't support WPA-Enterprise in FxOS) in comparison to Desktop.

I'll do some wifi tests across different networks that I have available to see if this reproduces in other situations with apprtc.
Well, I can tell you the properties: any network that blocks UDP more or less
entirely. Rather than going full AppRTC, I would just take a copy of
the turn unittest program (media/mtransport/tests) and run it
in the target network environment. If it succeeds, you should be good
to go. Otherwise not. If there is an environment where turn_unittest works
on both sides and apprtc does not, that's most likely some other kind of
defect
(Reporter)

Comment 17

5 years ago
Okay - what I think we can do here is evangelize to partners that their P2P apps using TURN will only work on networks that do not have UDP packets blocked. So I'll clear nomination here & add a dependency to bug 950660.
blocking-b2g: 1.3? → ---
Depends on: 950660
Keywords: qawanted
SGTM
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 950660
You need to log in before you can comment on or make changes to this bug.