1039188 - PeerConnection is not successfully established if devices are in different networks

Status: RESOLVED DUPLICATE of bug 1042345

(Firefox OS Graveyard :: Gaia::Loop, defect)

Description

[Maria Angeles Oteo (:oteo)]

Attachment: logcat.txt

When trying to establish a Loop call between two different FxOS Devices the media streams are successfully exchanged if they are in the same network but not if they are in different ones. 

Steps to reproduce:

 - Two devices with FxOS Loop one logged in with IDA and another with IDB
 - Select in Device A the option to call IDB via loop (e.g. via Address Book)
 - After a while trying to establish the connection both parties get the message " The stream was unable to connect due to a network error. Make sure your connection isn't blocked by a firewall."

Having a look at the logs, it seems the call establishment process is finished correctly (we got to state {"messageType":"progress","state":"connected”} ) and that the ICE servers are also passed correctly in the RTCPeerConnection (attaching the logs to this bug) but ICE does not complete successfully.

Comment 1

[Maria Angeles Oteo (:oteo)]

Attachment: stderr.txt

Comment 2

[Maria Angeles Oteo (:oteo)]

Attachment: nsprlog_signaling.txt

Comment 3

[Daniel Coloma:dcoloma]

Adding need info to :bwc as he told me via IRC he could have a look at the logs...

Comment 4

[Byron Campen [:bwc]]

So, definitely seeing that TURN is failing outright; are we sure that the credentials are ok? Can we get a PCAP of this somehow? Also seeing some interesting errors regarding the SDP that I haven't seen before (OnSdpParseError SDP Parse Error:  key-salt error decoding buffer: Base64 Bad Data). This looks like code that is supposed to parse a=crypto lines, but I don't see any such lines in the logcat. A p2p call using apprtc does not have any a=crypto lines either, and I do not see this error logging.

Also, seeing lots of "CallScreen: Error when parsing message SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data" in the logcat, but I'm not familiar with the software the generates these.

Comment 5

[:shell escalante]

Hi Rob, Wanted to reach out to you for guidance on who is the best person to engage with at TokBox for this cross-network call connection issue?

Comment 6

[Rob Hainer]

Song is probably the best person run point on this. He can pull in others as need be. Is

Comment 7

[Rob Hainer]

Is this issue unique to FFOS? Have you tried establishing a connection across the two networks using the Loop client in the browser?

Comment 8

[Maire Reavy [:mreavy]]

We're able to do calls in Loop Desktop across networks, and we're able to do WebRTC calls across networks on FxOS outside of Loop.  See Comment 4 for Byron's initial analysis.  Could the credentials be wrong?  Or could there be a problem in calling the SDK on the mobile Loop app?

Comment 9

[ggb]

:bwc Is FFOS supporting TURN TCP?  I see errors only for the UDP one (port 3478) and no logs for the TCP one (port 443).

Comment 10

[Byron Campen [:bwc]]

I do not think that we have TCP support in NrSocketIpc (eg; e10s):

http://dxr.mozilla.org/mozilla-central/source/media/mtransport/nr_socket_prsock.cpp#1029

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

See bug 1039655 for TURN TCP support for FxOS.  The usecase for this is where the phone is behind a firewall that totally blocks UDP, which is rare in residential situations and uncommon in offices.  Using TCP for media is always a poor substitute for UDP due to how packet loss and congestion windows are handled.  It's basically a solution-of-last-resort. 

Until it is supported in FxOS, a user would need to disconnect from the WiFi that's blocking UDP (carriers generally allow UDP).

Comment 12

[Daniel Coloma:dcoloma]

This seems not to be a network specific/configuration issue as it is happening in any network we are testing: WiFi, ethernet, 3G... Furthermore, cross-network calls worked in those networks in the past with previous versions of FFOS and Tokbox v2.2.5.

Comment 13

[ggb]

I only see "host" candidates for that session in the signaling in our server logs.   

In stderr.txt it says "No STUN servers specified" while in logcat.txt it shows the iceServers that we are passing to the PeerConnection:


Creating peer connection config "{"iceServers":[{"url":"turn:turn802-lhr.tokbox.com","credential":"4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"},{"url":"stun:turn802-lhr.tokbox.com","credential":null,"username":null},{"url":"turn:turn802-lhr.tokbox.com:443?transport=tcp","credential":"4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"}]}"

Any idea on where those iceServer could be lost?  Or what's the meaning of "No STUN/TURN servers specified"?

I'm very suspicious about the log "WARNING: pipe error (169): Connection reset by peer"  in stderr.txt.

Comment 14

[ggb]

:dcoloma any chance you can test new version of Opentok SDK with old version of FFOS or viceversa to try to isolate the problem? 

Thx a lot

Comment 15

[Daniel Coloma:dcoloma]

(In reply to ggb from comment #14)
> :dcoloma any chance you can test new version of Opentok SDK with old version
> of FFOS or viceversa to try to isolate the problem? 
> 
> Thx a lot

Yep, the problem also occurred with SDK v2.2.5 with current FFOS. Testing with an old version of FFOS is going to bit more challenging as many permissions we are using have recently changed, I'll give it a try.

Comment 16

[Byron Campen [:bwc]]

(In reply to ggb from comment #13)
> I only see "host" candidates for that session in the signaling in our server
> logs.   
> 
> In stderr.txt it says "No STUN servers specified" while in logcat.txt it
> shows the iceServers that we are passing to the PeerConnection:
> 
> 
> Creating peer connection config
> "{"iceServers":[{"url":"turn:turn802-lhr.tokbox.com","credential":
> "4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-
> V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-
> 9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"},{"url":"stun:
> turn802-lhr.tokbox.com","credential":null,"username":null},{"url":"turn:
> turn802-lhr.tokbox.com:443?transport=tcp","credential":
> "4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-
> V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-
> 9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"}]}"
> 
> Any idea on where those iceServer could be lost?  Or what's the meaning of
> "No STUN/TURN servers specified"?
> 
> I'm very suspicious about the log "WARNING: pipe error (169): Connection
> reset by peer"  in stderr.txt.

   We aren't trickling relayed candidates because our allocation requests aren't working, so they're never gathered.

Comment 17

[ggb]

(In reply to Byron Campen [:bwc] from comment #16)
> (In reply to ggb from comment #13)
> > I only see "host" candidates for that session in the signaling in our server
> > logs.   
> > 
> > In stderr.txt it says "No STUN servers specified" while in logcat.txt it
> > shows the iceServers that we are passing to the PeerConnection:
> > 
> > 
> > Creating peer connection config
> > "{"iceServers":[{"url":"turn:turn802-lhr.tokbox.com","credential":
> > "4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-
> > V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-
> > 9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"},{"url":"stun:
> > turn802-lhr.tokbox.com","credential":null,"username":null},{"url":"turn:
> > turn802-lhr.tokbox.com:443?transport=tcp","credential":
> > "4uan1zh8kV6cTcaSTOlfrHGhsq0=","username":"1405582735:1.2_MX40NDY2OTEwMn5-
> > V2VkIEp1bCAxNiAwMDozODo0MyBQRFQgMjAxNH4wLjgyODA3NTM1flB-.2e02b9db-66e3-481f-
> > 9f04-04afeed8b3cb.22666160-e1e2-4023-a3e7-8f296b3708c6"}]}"
> > 
> > Any idea on where those iceServer could be lost?  Or what's the meaning of
> > "No STUN/TURN servers specified"?
> > 
> > I'm very suspicious about the log "WARNING: pipe error (169): Connection
> > reset by peer"  in stderr.txt.
> 
>    We aren't trickling relayed candidates because our allocation requests
> aren't working, so they're never gathered.

But the log says "No STUN servers specified".   Is it correct from your understanding?  We are passing a list of iceServers with 1 STUN server and 2 TURN servers (UDP + TCP).

Comment 18

[José Antonio Olivera Ortega [:jaoo]]

(In reply to ggb from comment #14)
> :dcoloma any chance you can test new version of Opentok SDK with old version
> of FFOS or viceversa to try to isolate the problem? 

Daniel and myself have been poking around and we have found the root cause (at least something really fishy). Did a quick test with a simple demo app between two browsers in the desktop in different networks and the call worked. Ran that demo in FxOS browser (the device browser) with two devices in different networks and worked as well. The demo was using the API key, session ID, and token I got from the code page at [1]. What really surprised us is that the demo stopped working when we changed the API key, session ID, and token to the ones provided by the Loop server when making a call. After changing them we started hitting
the network error (see [2]).

[1] http://tokbox.com/opentok/tutorials/hello-world/js/demo.html
[2] E/GeckoConsole( 3432): Content JS ERROR at TB.js:913 in generateLoggingMethod/<: OT.exception :: title: Connection Failed (1013) msg: Subscriber PeerConnection Error: The stream was unable to connect due to a network error. Make sure your connection isn't blocked by a firewall.

Comment 19

[José Antonio Olivera Ortega [:jaoo]]

Attachment: 1039188-demo-working-logs.txt

Some logs capture while the demo was working (using API key, session ID, and token from the demo at [1]).

[1] http://tokbox.com/opentok/tutorials/hello-world/js/demo.html

Comment 20

[José Antonio Olivera Ortega [:jaoo]]

Attachment: 1039188-demo-notworking-logs.txt

Some logs captured while didn't work at all (using API key, session ID, and token we got from the Loop server).

Comment 21

[Byron Campen [:bwc]]

(In reply to ggb from comment #17)
> (In reply to Byron Campen [:bwc] from comment #16)
> 
> But the log says "No STUN servers specified".   Is it correct from your
> understanding?  We are passing a list of iceServers with 1 STUN server and 2
> TURN servers (UDP + TCP).

    The logging is just wrong, sadly. This is what is logged when the set of stun servers is not configured in the (singleton) registry. This makes sense when you're running a SIP UA or similar, but not when each ice_ctx might have a completely different set of STUN/TURN servers. For our case, we configure the set of STUN/TURN servers after the ice_ctx is created on a case-by-case basis.

Comment 22

[Daniel Coloma:dcoloma]

(In reply to José Antonio Olivera Ortega [:jaoo] from comment #18)
> (In reply to ggb from comment #14)
> > :dcoloma any chance you can test new version of Opentok SDK with old version
> > of FFOS or viceversa to try to isolate the problem? 
> 
> Daniel and myself have been poking around and we have found the root cause
> (at least something really fishy). Did a quick test with a simple demo app
> between two browsers in the desktop in different networks and the call
> worked. Ran that demo in FxOS browser (the device browser) with two devices
> in different networks and worked as well. The demo was using the API key,
> session ID, and token I got from the code page at [1]. What really surprised
> us is that the demo stopped working when we changed the API key, session ID,
> and token to the ones provided by the Loop server when making a call. After
> changing them we started hitting
> the network error (see [2]).
> 
> [1] http://tokbox.com/opentok/tutorials/hello-world/js/demo.html
> [2] E/GeckoConsole( 3432): Content JS ERROR at TB.js:913 in
> generateLoggingMethod/<: OT.exception :: title: Connection Failed (1013)
> msg: Subscriber PeerConnection Error: The stream was unable to connect due
> to a network error. Make sure your connection isn't blocked by a firewall.

In case anyone one the scenario it's quite easy, two FirefoxOS devices in different networks.

If both devices open the following URL in the browser: 

http://dcoloma.github.io/browser2browser/index.html

the call is established.

If both devices open the following URL in the browser:

http://dcoloma.github.io/browser2browser/index.html?creds=fxosloop

the call is not established

The only difference between both URLs is the APIKey, session id and token used to establish the connection.

Comment 23

[ggb]

The fxosloop session is P2P while the other one is MCU based.   The MCU one works because when you have an MCU in the middle you usually don't need TURN.     Can you retest with P2P sessions in both cases?

(In reply to Daniel Coloma:dcoloma from comment #22)
> (In reply to José Antonio Olivera Ortega [:jaoo] from comment #18)
> > (In reply to ggb from comment #14)
> > > :dcoloma any chance you can test new version of Opentok SDK with old version
> > > of FFOS or viceversa to try to isolate the problem? 
> > 
> > Daniel and myself have been poking around and we have found the root cause
> > (at least something really fishy). Did a quick test with a simple demo app
> > between two browsers in the desktop in different networks and the call
> > worked. Ran that demo in FxOS browser (the device browser) with two devices
> > in different networks and worked as well. The demo was using the API key,
> > session ID, and token I got from the code page at [1]. What really surprised
> > us is that the demo stopped working when we changed the API key, session ID,
> > and token to the ones provided by the Loop server when making a call. After
> > changing them we started hitting
> > the network error (see [2]).
> > 
> > [1] http://tokbox.com/opentok/tutorials/hello-world/js/demo.html
> > [2] E/GeckoConsole( 3432): Content JS ERROR at TB.js:913 in
> > generateLoggingMethod/<: OT.exception :: title: Connection Failed (1013)
> > msg: Subscriber PeerConnection Error: The stream was unable to connect due
> > to a network error. Make sure your connection isn't blocked by a firewall.
> 
> In case anyone one the scenario it's quite easy, two FirefoxOS devices in
> different networks.
> 
> If both devices open the following URL in the browser: 
> 
> http://dcoloma.github.io/browser2browser/index.html
> 
> the call is established.
> 
> If both devices open the following URL in the browser:
> 
> http://dcoloma.github.io/browser2browser/index.html?creds=fxosloop
> 
> the call is not established
> 
> The only difference between both URLs is the APIKey, session id and token
> used to establish the connection.

Comment 24

[ggb]

(In reply to Byron Campen [:bwc] from comment #21)
> (In reply to ggb from comment #17)
> > (In reply to Byron Campen [:bwc] from comment #16)
> > 
> > But the log says "No STUN servers specified".   Is it correct from your
> > understanding?  We are passing a list of iceServers with 1 STUN server and 2
> > TURN servers (UDP + TCP).
> 
>     The logging is just wrong, sadly. This is what is logged when the set of
> stun servers is not configured in the (singleton) registry. This makes sense
> when you're running a SIP UA or similar, but not when each ice_ctx might
> have a completely different set of STUN/TURN servers. For our case, we
> configure the set of STUN/TURN servers after the ice_ctx is created on a
> case-by-case basis.

Understood.   Any way to debug why the srflx&relay candidates gathering is failing?  (I don't even have a FFOS)    Do you think the WARNING pipe error could be the culprit?

Comment 25

[Daniel Coloma:dcoloma]

(In reply to ggb from comment #23)
> The fxosloop session is P2P while the other one is MCU based.   The MCU one
> works because when you have an MCU in the middle you usually don't need
> TURN.     Can you retest with P2P sessions in both cases?
> 

So I assume that P2P/MCU is decided based on the credentials, so I would need some alternative credentials to the FFOS ones to launch the session as P2P, right?

Comment 26

[Daniel Coloma:dcoloma]

So, based on comment 23 I've tried to focus on the ffos credentials scenario: 
http://dcoloma.github.io/browser2browser/index.html?creds=fxosloop

Basically, if both devices are in the same network, the communication is always established.

If devices are in different networks (e.g. one in 3G and another one in WiFi) I got the following results:

FirefoxOS & FirefoxOS -> It does not work (network error)
FirefoxOS & Firefox -> It does not work (network error)
Firefox & Firefox -> It works 

So it seems there is something different between FirefoxOS (or FFOS Browser) and Firefox behaviour as the networks we are using are the same.

Comment 27

[Daniel Coloma:dcoloma]

Attachment: logcat-P2P_credentials

Comment 28

[Daniel Coloma:dcoloma]

(In reply to Daniel Coloma:dcoloma from comment #25)
> (In reply to ggb from comment #23)
> > The fxosloop session is P2P while the other one is MCU based.   The MCU one
> > works because when you have an MCU in the middle you usually don't need
> > TURN.     Can you retest with P2P sessions in both cases?
> > 
> 
> So I assume that P2P/MCU is decided based on the credentials, so I would
> need some alternative credentials to the FFOS ones to launch the session as
> P2P, right?

Nevermind, we have used some other credentials (P2P ones we believe [1]) and with those ones the call across different networks works, it still fails when using the FxOS Loop ones. Can we do anything else to help you debugging this? Uploaded Attachment #8459609 [details]

[1]https://github.com/dcoloma/browser2browser/blob/gh-pages/js/startup.js#L18

Comment 29

[José Antonio Olivera Ortega [:jaoo]]

(In reply to ggb from comment #23)
> The fxosloop session is P2P while the other one is MCU based.   The MCU one
> works because when you have an MCU in the middle you usually don't need
> TURN.     Can you retest with P2P sessions in both cases?

Gave a try with the demo at [1] -FxOS Loop client app using P2P credentials(see [2])- and It worked. If we are not wrong there is no MCU in the middle. Could you verify it please? Thanks!

[1] https://github.com/jaoo/firefoxos-loop-client/commits/1039188
[2] https://github.com/jaoo/firefoxos-loop-client/commit/c9ecc3ec10e5c05c041a4d83de6b7b6f467ab9cc#diff-e5742477c728dce6482735bac5c05a40R72

Comment 30

[Nils Ohlmeier [:drno]]

Here is some feedback from TokBox I got today:
- You should defiantly use only the 2.2.6 SDK (do not use the 2.2.5 SDK any more)
- This commit https://github.com/jaoo/firefoxos-loop-client/commit/c9ecc3ec10e5c05c041a4d83de6b7b6f467ab9cc#diff-e5742477c728dce6482735bac5c05a40R72 looks like you are hard coding tokens into the app. This token is only valid for 24 hours and will stop working tomorrow. Tokens are not suppose to be hard coded in app, but be used dynamically on a per session base.

Comment 31

[Daniel Coloma:dcoloma]

(In reply to Nils Ohlmeier [:drno] from comment #30)
> Here is some feedback from TokBox I got today:
> - You should defiantly use only the 2.2.6 SDK (do not use the 2.2.5 SDK any
> more)
> - This commit
> https://github.com/jaoo/firefoxos-loop-client/commit/
> c9ecc3ec10e5c05c041a4d83de6b7b6f467ab9cc#diff-
> e5742477c728dce6482735bac5c05a40R72 looks like you are hard coding tokens
> into the app. This token is only valid for 24 hours and will stop working
> tomorrow. Tokens are not suppose to be hard coded in app, but be used
> dynamically on a per session base.

I think there is some confusion here so let me try to clarify. We are using some test apps in order to narrow down what is happening, in those apps we are hardcoding tokens, but in the real app [1] (where the bug is also occurring) we should be getting dynamically the tokens. Additionally me migrated that app to 2.2.6 around one week ago, but this issue occurred with both TB versions (2.2.5 and 2.2.6) 

[1] https://github.com/mozilla-b2g/firefoxos-loop-client

Comment 32

[José Antonio Olivera Ortega [:jaoo]]

(In reply to Nils Ohlmeier [:drno] from comment #30)
> Here is some feedback from TokBox I got today:

> - This commit
> https://github.com/jaoo/firefoxos-loop-client/commit/
> c9ecc3ec10e5c05c041a4d83de6b7b6f467ab9cc#diff-
> e5742477c728dce6482735bac5c05a40R72 looks like you are hard coding tokens
> into the app. 

This won't land at all. It seems there was a bit of misunderstanding about comment 29.

> This token is only valid for 24 hours and will stop working
> tomorrow. Tokens are not suppose to be hard coded in app, but be used
> dynamically on a per session base.

This was just a test guys. We wanted to give a try with other API key, session ID and token values than the ones the Loop client gets from the Loop server.

Comment 33

[Daniel Coloma:dcoloma]

(In reply to Daniel Coloma:dcoloma from comment #28)
> (In reply to Daniel Coloma:dcoloma from comment #25)
> > (In reply to ggb from comment #23)
> > > The fxosloop session is P2P while the other one is MCU based.   The MCU one
> > > works because when you have an MCU in the middle you usually don't need
> > > TURN.     Can you retest with P2P sessions in both cases?
> > > 
> > 
> > So I assume that P2P/MCU is decided based on the credentials, so I would
> > need some alternative credentials to the FFOS ones to launch the session as
> > P2P, right?
> 
> Nevermind, we have used some other credentials (P2P ones we believe [1]) and
> with those ones the call across different networks works, it still fails
> when using the FxOS Loop ones. Can we do anything else to help you debugging
> this? Uploaded Attachment #8459609 [details]
> 
> [1]https://github.com/dcoloma/browser2browser/blob/gh-pages/js/startup.js#L18

ggb has told us offline there is a bug in TB console and all the sessions are generated as MCU :( so the tests we did to compare what happens with other credentials are not valid. 

So, the only fishy information so far is what we described in comment 26:

With a session generated for FFOS, if devices are in different networks (e.g. one in 3G and another one in WiFi) we got the following results:

FirefoxOS Browser & FirefoxOS Browser -> It does not work (network error)
FirefoxOS Browser & Firefox Desktop -> It does not work (network error)
Firefox Desktop & Firefox Desktop -> It works 

So I think the key thing is identifying what is different between FFOS (or FFOS Browser) and FF Desktop that makes it work in Desktop to Desktop communication.

Comment 34

[song]

Just in case you don't already have this information, here is how to generate a p2p session id: https://github.com/opentok/opentok-node#creating-sessions

Comment 35

[Randell Jesup [:jesup] (needinfo me)]

Ok - we're going to get some detailed logs and captures at Mozilla tomorrow.   In the morning EU time, if you could get us about:webrtc data for the desktop tests that worked (preferably in the same set of networks that fail for the app) that might help.   (turn on the Connection logging, then Select All, Copy and save into a file.)

A Wireshark/etc WiFi sniff/capture might also be of use, or something sniffing all the traffic upstream of the AP.  Also, a wireshark capture for a FxOS<->Desktop (on the Desktop) and the about:webrtc (on the Desktop) would help a lot.

Thanks!  We'll get the bottom of this.

Comment 36

[Daniel Coloma:dcoloma]

Attachment: Test Environment Description

Comment 37

[Daniel Coloma:dcoloma]

Attachment: log-DesktopToDesktop-DesktopWifiA.txt

Desktop-to-Desktop, successful call, about:webrtc information WiFi A

Comment 38

[Daniel Coloma:dcoloma]

Attachment: log-DesktopToDesktop-DesktopWifiB.txt

Desktop-to-Desktop, successful call, about:webrtc information WiFi B

Comment 39

[Daniel Coloma:dcoloma]

Attachment: log-DesktopToFFOS-DesktopWifiA.txt

Desktop-to-FFOS, failed call, about:webrtc information WiFi A (Desktop)

Comment 40

[Daniel Coloma:dcoloma]

Attachment: log-DesktopToFFOS-FFOSWifiB.txt

Desktop-to-FFOS, failed call, logcat information WiFi B (FFOS)

Comment 41

[Nils Ohlmeier [:drno]]

(In reply to Daniel Coloma:dcoloma from comment #33)
> FFOS Browser) and FF Desktop that makes it work in Desktop to Desktop
> communication.

The most important difference is missing TURN TCP support in FFOS as per bug 1039655.

If the networks you are testing in do block UDP that would explain why you are seeing the difference between Firefox Desktop vs. FFOS.

Comment 42

[Byron Campen [:bwc]]

Also, if UDP is blocked, it would explain the inability to gather any relay candidates. And I'm not seeing any hint of server reflexive candidates in the log attached in comment 40. Can we get DEBUG ICE logging (ie; R_LOG_LEVEL=7) from the phone? A pcap would also be extremely useful.

Comment 43

[Lucas Adamski [:ladamski]]

Mozilla QA is concerned that this bug is blocking full functional testing for the webRTC / Loop APIs, which in turn is preventing them from declaring FC for OS 2.0.  Can we get an assessment whether we can fix this in 2.0, and conversely if we don't is the impact tolerable.

Comment 44

[Eric Rescorla (:ekr)]

I understand that Nils diagnosed this as a failure of the DNS resolver code in
B2G. I will take a look tonight after I get b2g rebuilt. However, it's important
to distinguish between Loop and the platform APIs. Modulo then problem being
trivial, it's going to be a lot easier for us/TB to fix Loop by injecting numeric
IP addresses into the JS. This won't fix the platform, however.

Comment 45

[Maire Reavy [:mreavy]]

Loop is the focus for v2.0.  For Loop, we can make sure that the server provides numeric IPs and/or the app can lookup and cache the IP address of each STUN/TURN server using an external service.

For other apps, here are some possible solutions we've been brainstorming:
*) provide a lookup-and-cache module that apps can include
*) Specify at least one STUN (and TURN) server numerically (perhaps only to B2G <2.1 clients)
*) We can configure a backup STUN server (our default STUN server) and if we get a list of all-FQDN servers, add our server to the list so STUN-resolvable solutions work.
*) reach out to providers to encourage them to specify servers as IPs per above (again, it can be a backup done only for FxOS <2.1 devices)

In the meantime, we'll investigate a root cause fix with the networking team.

Comment 46

[Eric Rescorla (:ekr)]

(In reply to Maire Reavy [:mreavy] (Plz needinfo me) from comment #45)
> Loop is the focus for v2.0.  For Loop, we can make sure that the server
> provides numeric IPs and/or the app can lookup and cache the IP address of
> each STUN/TURN server using an external service.
> 
> For other apps, here are some possible solutions we've been brainstorming:
> *) provide a lookup-and-cache module that apps can include
> *) Specify at least one STUN (and TURN) server numerically (perhaps only to
> B2G <2.1 clients)
> *) We can configure a backup STUN server (our default STUN server) and if we
> get a list of all-FQDN servers, add our server to the list so
> STUN-resolvable solutions work.

Note that this will still not work with TURN unless we also supply
a TURN server

> *) reach out to providers to encourage them to specify servers as IPs per
> above (again, it can be a backup done only for FxOS <2.1 devices)
> 
> In the meantime, we'll investigate a root cause fix with the networking team.

Comment 47

[Randell Jesup [:jesup] (needinfo me)]

A quick temporary workaround would be to hack in adding a numeric IP STUN (and maybe TURN) setup (nslookup on tokbox's servers for now probably, or the old default Mozilla STUN server).  I don't know if you can override "new mozRTCPeerConnection(...)" such that you can avoid editing the SDK.

Comment 48

[Byron Campen [:bwc]]

Did any of these failure logs have NSPR_LOG_MODULES=mtransport:6 in them? There are a few places where we'd log something if DNS resolution did not work, but I'm not seeing this in any of the log files.