1853120 - WebDriver browser configuration (like environment variables) are not forwarded to the pytestrunner

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Environment variables are currently not forwarded to the pytestrunner. As such our Mozilla specific tests do not run with the MOZ_DISABLE_NONLOCAL_CONNECTIONS environment variable set and causing crashes when trying to get async events working with Wdspec (see bug 1852243).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is actually not only for environment variables but basically related to a lot of the browser configuration arguments.

For the public WdSpec tests the webdriver binary gets started by the FirefoxWdSpecBrowser (WebDriverBrowser) class.

But for Mozilla specific tests we start another instance of geckodriver via the GeckoDriver fixture. And here we would have to pass the extra information like environment settings via the pytestrunner and WdspecExecutor.

It looks like that we might have to subclass WdspecExecutor to have a WdspecFirefoxExecutor class specific for Firefox (similar for Android), so that we can pass all the arguments from the FirefoxWdSpecBrowser class as well. Otherwise we miss a lot of those arguments and start Firefox with incorrect settings.

Given that this sounds like a larger refactoring I would like to hear the feedback from James, or let it get discussed in next weeks triage meeting. Maybe we can find a quick solution for environment variables?

Comment 2

[James Graham [:jgraham]]

Attachment: Bug 1853120 - Pass the environment in to wdspec configuration

This allows it to be used in fixtures that directly control processes.

Comment 3

[James Graham [:jgraham]]

I imagine something like the patch attached will work? I haven't extensively tested it though (and for some reason try didn't run any jobs).

Comment 4

[James Graham [:jgraham]]

https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=Pc8DVyfQTUafrwUlUW9tvw.0&revision=7897f3711204e2735d5cf35d9cba90b10b5edb3c is now running and we do indeed get some new crashes with this patch due to passing in the disabled network connections.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1853120 - [wdspec] Update Mozilla specific test "test_profile_root" to use wpt default preferences.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to James Graham [:jgraham] from comment #4)

https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=Pc8DVyfQTUafrwUlUW9tvw.0&revision=7897f3711204e2735d5cf35d9cba90b10b5edb3c is now running and we do indeed get some new crashes with this patch due to passing in the disabled network connections.

The problem here was with the profile root test, where geckodriver itself creates a new profile but doesn't have a reference to the default profile. As such only the recommended preferences are set and not those from the wpt's user.js file. The extra revision that I just pushed fixes the problem.

Comment 7

[James Graham [:jgraham]]

Attachment: Bug 1853120 - [wdspec] Make Mozilla specific profile fixtures for getting argument values more robust

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43c06d6ece96
[wdspec] Update Mozilla specific test "test_profile_root" to use wpt default preferences. r=webdriver-reviewers,jgraham
https://hg.mozilla.org/integration/autoland/rev/cd110eb5314b
Pass the environment in to wdspec configuration r=webdriver-reviewers,jgraham
https://hg.mozilla.org/integration/autoland/rev/6ce94024661c
[wdspec] Make Mozilla specific profile fixtures for getting argument values more robust r=whimboo,webdriver-reviewers

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/43c06d6ece96
https://hg.mozilla.org/mozilla-central/rev/cd110eb5314b
https://hg.mozilla.org/mozilla-central/rev/6ce94024661c

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/42196 for changes under testing/web-platform/tests

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1853120 - [wpt] Fix formatting and spelling errors in pytestrunner script.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a0949ab56be3
[wpt] Fix formatting and spelling errors in pytestrunner script. r=jgraham,Sasha

Comment 13

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/a0949ab56be3

Comment 14

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot