952464 - [emulator][mochitest] allow changing network connection on emulator mochitest

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Emulator, defect)

Description

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: minimal reproducing test case

The iframe.src = "some-page-url" stop functioning after enabling RIL data connection. In this test case, iframe1 is loading test_iframe1.html and change to test_iframe2.html after turning on 3G data. However, we can only found the log for test_iframe1.html. Removing the enableDataConnection() will make the entire test set finish correctly without blocking.

TEST PROCEDURE
run |./mach mochitest-remote Harness_sanity/|

EXPECTED
all test cases are executed.

ACTUAL
test cases cannot continue after test_breaker.html, and do not see the "test_iframe2 is loaded" message in console.

Comment 1

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

@vicamo has some patches for completely emulating 3G data connection on emulator. These patches are aimed to fix the no default route issue on emulator. However, the bug still exist after manually applying those patches. So, this issue is probably not causing by DNS or network. Need some one to take look at document loading process.

Here is the step-by-step guide for patching the emulator 3G network issue. (thanks to @vicamo!). With this patch you can browsing web page on Browser app inside the emulator after turning on 3G.
#### 1 ####
pull two PRs (external/qemu, hardware/ril)in bug 871475.

#### 2 ####
pull the PR (.repo/manifest) in bug 951053
$ ./repo sync external/iproute2

#### 3 ####
# edit system/core/rootdir/etc/init.goldfish.sh and insert following line after line 2
# Setup networking when boot starts
fconfig eth0 10.0.2.15 netmask 255.255.255.0 up
ip route add default via 10.0.2.2 dev eth0 metric 2

#### 4 ####
$ rm out/target/product/generic/system/etc/init.goldfish.sh
$ ./build.sh

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

This is the bug specifically about the problems we had with loading the next test in bug 950317.

I'm not 100% sure that schien is correct that DNS/network can be the issue, as I don't know how the mochitests feed the tests from the framework to the device

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

Adding a few more people who might have ideas about what's up with the b2g mochitest framework here

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Randell Jesup [:jesup] from comment #2)
> I'm not 100% sure that schien is correct that DNS/network can be the issue,

"can't be the issue", not can.

Comment 5

[Andrew Halberstadt [:ahal]]

So in the test case the test is waiting for a message from the parent process before continuing. This could actually be bug 951895. Though if this is the case, then test_breaker.html wouldn't be executed either.. Do you see 'enable radio' in the logs? If not try applying the patch from 951895.

Comment 6

[Andrew Halberstadt [:ahal]]

(In reply to Randell Jesup [:jesup] from comment #2)
> I'm not 100% sure that schien is correct that DNS/network can be the issue,
> as I don't know how the mochitests feed the tests from the framework to the
> device

To answer this question, we just serve all the tests on the host, load up SimpleTest and pass a manifest in via url parameter. SimpleTest then just loads each test one at a time in an iframe.

Comment 7

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

(In reply to Andrew Halberstadt [:ahal] from comment #5)
> So in the test case the test is waiting for a message from the parent
> process before continuing. This could actually be bug 951895. Though if this
> is the case, then test_breaker.html wouldn't be executed either.. Do you see
> 'enable radio' in the logs? If not try applying the patch from 951895.
Not necessary, this test case is intend to remove all the dependency from other bug. The message we are waiting for is send from the BreakerLoadChromeScript.js which we load for this test case. I see all the log I printed in test_breaker.html on my site.

Comment 8

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Andrew Halberstadt [:ahal] from comment #6)
> (In reply to Randell Jesup [:jesup] from comment #2)
> > I'm not 100% sure that schien is correct that DNS/network can be the issue,
> > as I don't know how the mochitests feed the tests from the framework to the
> > device
> 
> To answer this question, we just serve all the tests on the host, load up
> SimpleTest and pass a manifest in via url parameter. SimpleTest then just
> loads each test one at a time in an iframe.

"loads each tests" - loads from where?  That's where installing a new data connection could fubar things...

Comment 9

[Andrew Halberstadt [:ahal]]

The tests are served with xpcshell/httpd.js here:
http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#112

We use a PAC url to proxy requests to the proper location on the host:
http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/mozprofile/permissions.py#310
http://mxr.mozilla.org/mozilla-central/source/build/pgo/server-locations.txt

Here is where the test is loaded:
http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/TestRunner.js#325

Comment 10

[Andrew Halberstadt [:ahal]]

So to summarize, we've figured out how to enable 3g in the mochitests by synthesizing the message-listener-ready event, but once we do so, all subsequent mochitests fail to run. The host serves the tests and the emulator uses a special ip (10.0.2.2) which resolves to 127.0.0.1 on the host [1]. The theory is that enabling data somehow messes with the network and the emulator can no longer reach the tests on the host.

[1] http://developer.android.com/tools/devices/emulator.html#networkaddresses

Comment 11

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

After setup 3G data, ProxyAutoConfig::GetProxyForURI() is never invoked before DNS resolution. The PAC file we used in mochitest should redirect http://mochi.test:8888 to an local IP address. I think that's the reason we saw DNS error for http://mochi.test.

Comment 12

[Andrew Halberstadt [:ahal]]

So could we listen for ondatachange and then invoke it manually, possibly in b2g_start_script again?

Comment 13

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

I'm not an expert of PAC so I'm not sure if it is normal discarding PAC setting after network change.

@Patrick, mochitest uses PAC file for translating the fake domain name to local IP address. B2G emulator seems discarding the PAC setting after we manually change the network interface in testcase. Should we manually restore the PAC setting after changing network interface?

Comment 14

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Ok, now I got the whole picture of this bug. The root cause is that the pref "network.proxy.type" is been reset to PROXYCONFIG_SYSTEM after turning on 3G data [1]. Mochitest framework uses user-defined PAC file for mapping mochi.test to local IP address, and overwrite the "network.proxy.type" to PROXYCONFIG_PAC in user preference. Changing proxy type will make Gecko discard the user-defined PAC setting, which will break the mochi.test name lookup.

[1] http://dxr.mozilla.org/mozilla-central/source/dom/system/gonk/NetworkService.js#329

Comment 15

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: manually revert proxy type

Document loading will success if we manually set "network.proxy.type" back to 2 after turning on 3G data.

Comment 16

[Andrew Halberstadt [:ahal]]

Comment on attachment 8358338 [details] [diff] [review]
manually revert proxy type

Review of attachment 8358338 [details] [diff] [review]:
-----------------------------------------------------------------

\o/

Would listening to ondatachange work instead of the timeout?

Comment 17

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

(In reply to Andrew Halberstadt [:ahal] from comment #16)
> Would listening to ondatachange work instead of the timeout?
There will be 3 datachange events being fired and proxy.type will also being set multiple times. I should reach out to @vicamo for a better timing instead of timeout.

Comment 18

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Comment on attachment 8358338 [details] [diff] [review]
manually revert proxy type

Hi @vicamo, I found there will be 3 datachange events being fired after turning on 3g data on emulator. Therefore, I cannot use ondatachange for reverting proxy.type setting. Could you point out a better timing for reverting proxy.type?

Comment 19

[Vicamo Yang [:vicamo][:vyang]]

Comment on attachment 8358338 [details] [diff] [review]
manually revert proxy type

Review of attachment 8358338 [details] [diff] [review]:
-----------------------------------------------------------------

Listening to datachange and check "mozMobileConnections[N].data.connected" would be the most correct way.

Comment 20

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

After having a quick chat with @ahal and @vchang, we could modify NetworkService.js to restore the system initial proxy setting instead of clear all the user settings [1].

[1] http://dxr.mozilla.org/mozilla-central/source/dom/system/gonk/NetworkService.js?from=NetworkService.js#329

Comment 21

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: revert-proxy-setting-in-network-service.patch

Revert proxy settings to system initial values instead of default values.

Comment 22

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

@ahal, you can try revert-proxy-setting-in-network-service.patch and see if it solves the problem while running mochitest on real device.

Comment 23

[Andrew Halberstadt [:ahal]]

David Clarke is the one working on the device mochitests (bug 915810). He had already hardcoded the pref as a work-around, but it seems like we are still hitting another unrelated problem. I think he's rebuilding with your patch applied now just to see.

Comment 24

[[e-mail] [:onecyrenus]]

@schien, i just tested this patch, and it works from my perspective.  Your patch doesn't apply cleanly, could you respin it. 

the other problems i'm seeing aren't related to this patch.

Comment 25

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: revert-proxy-setting-in-network-service.patch

I forgot that I reverted my repository back to last Friday because of other issues. Rebase to latest m-c.

Comment 26

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: Part 2 - gonk-network-testcase.patch

add mochitest case to dom/system/gonk/tests/mochitest

Comment 27

[Vincent Chang[:vchang][changyihsin]]

Comment on attachment 8359629 [details] [diff] [review]
revert-proxy-setting-in-network-service.patch

Review of attachment 8359629 [details] [diff] [review]:
-----------------------------------------------------------------

The patch looks good. But I feel that I am not familiar with mochitest, I would like to redirect the review to vicamo.

Comment 28

[Vincent Chang[:vchang][changyihsin]]

Comment on attachment 8359629 [details] [diff] [review]
revert-proxy-setting-in-network-service.patch

Review of attachment 8359629 [details] [diff] [review]:
-----------------------------------------------------------------

The patch looks good. But I feel that I am not familiar with mochitest, I would like to redirect the review to vicamo.

Comment 29

[Vincent Chang[:vchang][changyihsin]]

Comment on attachment 8360270 [details] [diff] [review]
Part 2 - gonk-network-testcase.patch

Review of attachment 8360270 [details] [diff] [review]:
-----------------------------------------------------------------

Redirect to vicamo :-)

Comment 30

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

try result here: https://tbpl.mozilla.org/?tree=Try&rev=0ae00c23a7b1

Comment 31

[Randell Jesup [:jesup] (needinfo me)]

When combined with bug 950317, test_switch_dataConnection.html doesn't exit (see try results in that bug)

Comment 32

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: Part 1 - revert-proxy-setting-in-network-service.patch

Update the reviewer info on hg commit log.

Comment 33

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: Part 2 - gonk-network-testcase.patch, v2

Reset data connection before test and update the reviewer info in hg commit log.

Let's see how it works with WebRTC mochitest enabled.
https://tbpl.mozilla.org/?tree=Try&rev=c71da9db09d0

Comment 34

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Comment on attachment 8360844 [details] [diff] [review]
Part 1 - revert-proxy-setting-in-network-service.patch

Remove review request because this patch might store wrong proxy setting as system initial value.