1751357 - Wdspec tests on Android inappropriately make use of FirefoxAndroidBrowser class instead of NullBrowser

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

When you try to run the webdriver tests from the web-platform-tests on your local machine via mach wpt you can notice that wptrunner starts the test_runner application on it's own and not utilizing geckodriver:

 1:43.81 adb INFO Setting test_root to /data/local/tmp/test_root
 1:49.23 INFO STDOUT: timed out waiting for profiles.ini
 1:49.39 adb INFO Granting important runtime permissions to org.mozilla.geckoview.test_runner
 1:50.68 adb INFO launch_application: am start -W -n org.mozilla.geckoview.test_runner/org.mozilla.geckoview.test_runner.TestRunnerActivity -a android.intent.action.MAIN --es env0 MOZ_CRASHREPORTER=1 --es env1 MOZ_CRASHREPORTER_NO_REPORT=1 --es env2 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env3 MOZ_HIDE_RESULTS_TABLE=1 --es env4 MOZ_IN_AUTOMATION=1 --es env5 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --es env6 R_LOG_LEVEL=6 --es env7 R_LOG_DESTINATION=stderr --es env8 R_LOG_VERBOSE=1 --es env9 MOZ_PROCESS_LOG=/var/folders/q9/vhvs7hf57lgbbvkylx6cdp_w0000gn/T/tmpn54gdlegpidlog --es env10 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env11 STYLO_THREADS=1 --es arg0 -no-remote --es arg1 -profile --es arg2 /data/local/tmp/test_root/profile --es arg3 --marionette --es arg4 about:blank --ez use_multiprocess True
 1:52.08 INFO Starting runner
 1:54.15 pid:67448 Full command: /Users/henrik/code/gecko/target/debug/geckodriver --marionette-port 2828 --host 127.0.0.1 --port 55421 -vv

For WebDriver tests the wptrunner should never launch the application itself.

Note that this is causing side-effects with bug 1699988 when geckodriver hasn't removed the geckoview config file in a former mach wpt run and then the application is just crashing.

James, do you know where this might happen? If not I'll have to dig into. Thanks.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The problematic code is actually here:

https://searchfox.org/mozilla-central/rev/c1598f6d3edad19ccc53f53ab045d1d29835e1dd/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox_android.py#240-254

So it's not only that the test runner gets started by wpt itself but also that port forwards and reverse are set.

James, I assume this might require a maybe larger refactoring of the code and maybe it's something you could take care of?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As mentioned by James on Element:

The class you pointed at should never be created for wdspec tests, and by the time we create it we definitely do know which kind of tests will be run.

You need to change the configuration to match e.g. firefox: https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox.py#43-44

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Simply switching to the NullBrowser doesn't work. While we have all the appropriate preferences used in the GeckoView profile any navigation command fails because requests for web-platform.test URLs error out.

As James mentioned he can take a look at this after he is done with the refactoring for:
https://github.com/web-platform-tests/wpt/pull/29089

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Everything as discovered on this bug should actually be fixed with the downstream sync we are waiting for to happen over on bug 1712540.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jgraham, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Wdspec tests on Android make use of the FirefoxAndroidWdSpecBrowser class now:

https://searchfox.org/mozilla-central/rev/855b2af753bc01bf27fc1b9f2d0184a0a440773d/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox_android.py#25-26
https://searchfox.org/mozilla-central/rev/855b2af753bc01bf27fc1b9f2d0184a0a440773d/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox_android.py#307-359

As such there is nothing left to do here.