1020812 - [Tarako] FxOS sometimes cannot connect to Marketplace

Status: RESOLVED WONTFIX

(Marketplace Graveyard :: General, defect)

Description

[William Hsu [:whsu]]

Attachment: Marketplace_Log_20140605

* Description:
  This bug happened on Tarako device and cannot be 100% reproduced.( Reproduce rate: 1/10)
  After you enable WIFI and try to launch Marketplace, you will see that FxOS stays on launch page.
  Attaching the log and demo video.
  - Log:   Marketplace_Log_20140605
  - Video: https://www.youtube.com/watch?v=akVQ-SSEIAU&feature=youtu.be

* Reproduction steps:
  1. Flash a new/fresh tarako build on Tarako device
  2. Enable WIFI and connect to available network on FTU
  3. Tap "Marketplace" to launch marketplace

* Expected result:
  FxOS can launch Marketplace

* Actual result:
  FxOS stays on launch page

* Build information:( V1.3T - Tarako )
 - Gaia     c7d82808c695537fc258accb4217c58eb8160c48
 - Gecko    https://hg.mozilla.org/releases/mozilla-b2g28_v1_3t/rev/acc261c856a3
 - BuildID  20140604164001
 - Version  28.1

Comment 1

[Joe Cheng [:jcheng] (please needinfo)]

ni? Wil

Comment 2

[William Hsu [:whsu]]

Might it cause by WIFI connection.
There was something wrong.

-----------------------------------------------------------------------------------------------------
I/Gecko   (   86): -*- WifiWorker component: State change: ASSOCIATED -> COMPLETED
W/WifiHW  (   86): wifi_command: ENABLE_NETWORK 0
D/wpa_supplicant(  352): CTRL_IFACE: ENABLE_NETWORK id=0
W/WifiHW  (   86): wifi_command: SIGNAL_POLL
D/wpa_supplicant(  352): nl80211: survey data missing!
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 ssid
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='ssid'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 bssid
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='bssid'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'bssid'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 psk
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='psk'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'psk'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 wep_key0
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='wep_key0'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'wep_key0'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 wep_key1
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='wep_key1'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'wep_key1'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 wep_key2
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='wep_key2'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'wep_key2'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 wep_key3
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='wep_key3'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'wep_key3'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 wep_tx_keyidx
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='wep_tx_keyidx'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 priority
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='priority'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 key_mgmt
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='key_mgmt'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 scan_ssid
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='scan_ssid'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 disabled
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='disabled'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 identity
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='identity'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'identity'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 password
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='password'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'password'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 auth_alg
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='auth_alg'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 phase1
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='phase1'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'phase1'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 phase2
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='phase2'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'phase2'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 eap
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='eap'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'eap'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 pin
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='pin'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'pin'
W/WifiHW  (   86): wifi_command: GET_NETWORK 0 pcsc
D/wpa_supplicant(  352): CTRL_IFACE: GET_NETWORK id=0 name='pcsc'
D/wpa_supplicant(  352): CTRL_IFACE: Failed to get network variable 'pcsc'
I/Gecko   (   86): -*- WifiWorker component: Firing connectionInfoUpdate: ({signalStrength:-55, relSignalStrength:100, linkSpeed:1, ipAddress:""})
E/slog    (  100): check_available_volume: statfs return err! [2]

Comment 3

[Henry Chang [:hchang]]

What's wrong with this log?

Comment 4

[Vincent Chang[:vchang][changyihsin]]

I am wondering if you can access the internet using others app, eg. browser at that time. It helps to double confirm wifi connection status.

Comment 5

[Wil Clouser [:clouserw]]

We've seen situations where the device claims to be on the network but it can't get anywhere.  In that case the Marketplace isn't showing the offline page but network requests don't work.  We also couldn't reliably reproduce.  Testing other apps (as in comment 4) would support this theory.

Comment 6

[Wil Clouser [:clouserw]]

For some background:  

If the Marketplace is online it will display the home page (cached locally) and fire network requests for the icons (loaded lazily after the page is shown).

If the Marketplace is offline it will display a splash screen letting the user know they need to be online to use the Marketplace.

There are long standing bugs (years) preventing the Marketplace from reliably asking the device whether it is online or offline (see bug 756364 and bug 654579).  To work around that, we attempt to open a socket to our CDN and if it fails we assume we are offline:

>        try {
>            var host = (new URL(settings.cdn_url)).host;
>            var port = 80;
>            var socket = navigator.mozTCPSocket.open(host, port);
>        } catch (e) { 
>            ...
>        }
>
>        socket.onerror = function(e) {
>            offline(socket, def);
>        };

I suspect this is hanging around that detection, but unreliable network connections are going to be tough for us to do something with.  ni?doug to see if he has a better recommendation for testing if we're offline.

Comment 7

[Joe Cheng [:jcheng] (please needinfo)]

reproduce rate seems low. let's not block the release with this bug bug continue to find out more. thanks

Comment 8

[William Hsu [:whsu]]

(In reply to Vincent Chang[:vchang] from comment #4)
> I am wondering if you can access the internet using others app, eg. browser
> at that time. It helps to double confirm wifi connection status.
Yes, I saw Joe can google something via browser while the Marketplace is stuck on loading page.
So, the network should be valid and connected.

One more question, what can I do for you if I encounter this problem again?
I have attached the log on the bug. But, it seems useless.

Comment 9

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Having the logcat will help because it's showing the ip address and if you have wifi connection. ( esp if you have wifi debugging pref turned on )

Noting where in the timestamp in the logcat you launched marketplace or did something will help.
Checking to make sure browser does load a webpage also helps after this occurs: https://marketplace.firefox.com/  [ ie checking if it's not a dns server issue ]

Wil might be able to provide more.  
Note : VT found that when you turn on airplane mode while Marketplace is already up, can give the wifi error exception.

Comment 10

[Wil Clouser [:clouserw]]

log annotations would be good.  Some networking creativity on the wifi access point could be done where packets to that hostname are blackholed or there is massive packet loss or something in an effort to reproduce.  Krupa is the only one I know who has had a similar situation and it couldn't be reproduced.

Comment 11

[William Hsu [:whsu]]

Thanks for the helpful information.
I will try to reproduce it and dump all WIFI log to see if I can find some weird phenomenons.

Needinfo myself ~ (whsu)

Comment 12

[Jason Duell]

> a better recommendation for testing if we're offline.

We could keep some flag set for some short time period after each time we do successful network I/O, and just assume we're still online during that (if the connection to the CDN is a bad latency hit that might reduce it) but that will if anything make the reliability of online/offline detection sketchier.  

You could launch two different HTTP connections and set online=true if either of them succeed.  (If they're launched in parallel this actually reduces your avg latency while increasing your chance of success).  I don't know if it would make sense to stagger the requests' asyncOpen slightly (might help with transient network failure).

Patrick may have ideas here.

Comment 13

[Patrick McManus [:mcmanus]]

:bagder is working on exatcly this class of problem (my network changed, my network is unreliable, my network woke up, etc..)

Comment 14

[Daniel Stenberg [:bagder]]

The ground work for the network-changed fixes are done as part of Bug 939318, but the FxOS-specific work that needs to be done for this to really fix the FxOS situation is filed as a separate issue in Bug 1008091.

Comment 15

[William Hsu [:whsu]]

Hi, Everyone,

I seem to be able to reproduce this problem.

I suppose that the network was unstable while I encountered this issue.
So, I use a WIFI hotspot to assist my test.
Finally, the result is same as we encountered.

Here are my test steps:
1. Prepare a WIFI hotspot
2. Tarako device connects to hotspot
3. Launch the marketplace on Tarako device, and turn off the outgoing network of hotspot at the same time.
4. Turn on outgoing network of hotspot again
5. Using browser to check the network.
6. If the network is valid, launching the Marketplace again.
* Demo video: http://youtu.be/Er2JlJaZBJA

If you think foregoing symptom caused by different root causes, I would like to submit the other bug to trace it.
Thanks! Have a nice day!

Comment 16

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

removing qawanted, William provided STRs.

Comment 17

[David Durst [:ddurst]]

Comment 15 STR show the same symptom, but not the same root cause (the initial STR don't have the user manually disabling the outbound network connection).

Initial STR are still the low frequency occurrences; we could use more log data to analyze.

The STR in Comment 15 manually introduce an unreliable network connection.
In Comment 6, the method we use to detect the network connection is detailed.
In Comment 14, a bug is referenced that, when solved, would be an alternate way for us to detect the network connection (though it would not necessarily mitigate the STR in Comment 15).

I'm going to refer to Comment 7 and close this as WONTFIX until we have more detail as to what's happening (Comment 9 and Comment 10) or a fix (like Comment 14) that is shown to solve the problem, whatever it is.