1483559 - [wdspec] Add tests for GeckoExtensionCommand's

Status: NEW

(Remote Protocol :: Marionette, enhancement, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

To make sure that with the Serde conversion on bug 1396821 no regressions are getting introduced, we should have wdspec tests for all the GeckoExtensionCommand's which are:

>    GetContext,
>    SetContext(GeckoContextParameters),
>    XblAnonymousChildren(WebElement),
>    XblAnonymousByAttribute(WebElement, AttributeParameters),
>    InstallAddon(AddonInstallParameters),
>    UninstallAddon(AddonUninstallParameters)

It would also be good to have unit tests for geckodriver to make sure that serialization and deserialization works well with Serde.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Selenium pytest tests

Attached you can find the Selenium tests which I used to verify the correct behavior for the Serde transition, and which we want as wdspec tests.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Moving to p2 because no activity for at least 24 weeks.
See How Do You Triage for more information

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I won't have the time to continue on this particular bug.

Comment 4

[Kriti Singh]

Hi Henrik. I was thinking of taking up this issue. Could you give me some pointers on how to get started? Thanks!

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sorry, I'm currently not working on this project. Andreas would be able to help you here.

Comment 6

[Andreas Tolfsen ❲:ato❳]

Hi Kriti,

If this is for the Outreachy application I would advise that you
submit your final application. I’ve closed project proposal for
more applications.

If you still want to work on this I can give some guidance and
pointers, but I will mostly be away for the next two weeks.

Comment 7

[Kriti Singh]

Thanks Andreas for letting me know so. Irrespective, I would like to solve the issue. Thank you!

Comment 8

[Andreas Tolfsen ❲:ato❳]

wdspec is a Web Platform Test (WPT) test type used for testing
conformance to the WebDriver standard. geckodriver extends the
base WebDriver command endpoints with those listed by whimboo in
comment #0. You can see the URL routes in command::extension_routes()
in geckodriver.

Since they are non-standard extension commands to WebDriver, the
tests for them can’t be put in the upstream WPT repo. For tests
that are Mozilla specific we have the folder
testing/web-platform/mozilla, where we can place Mozilla-specific
WPT tests which use the same tools and test harnesses as the upstream
tests. We have some examples of Mozilla-specific wdspec tests under
testing/web-platform/mozilla/tests/webdriver.

This bug is about testing the public-facing APIs of the geckodriver
extension commands in a similar fashion. Some context information
about the different commands:

• GetContext/SetContext(GeckoContextParameters) is a getter/setter
  for the Marionette context, which determines where all subsequent
  commands are evaluated. Commands are run in "content" context
  by default, but this can be switched to "chrome" to allow
  retrieval of chrome UI elements, and scripts to be evaluated in
  the system privileged sandbox.

• XblAnonymousChildren(WebElement) and
  XblAnonymousByAttribute(WebElement, AttributeParameters) are
  scheduled for removal and are very complicated. I would suggest
  delay these.

• InstallAddon(AddonInstallParameters) and
  UninstallAddon(AddonUninstallParameters) do as their names
  suggest, allow installing and uninstalling web extensions. The
  install command takes either a path to an extension on the file
  system, or a Base64 encoded blob of the .xpi file. To test this
  you will need to provide a simple web extension along with the
  test.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Please note that it would also be great to have Rust unit tests under geckodriver/src which verify the serialization/deserialization with Serde. Those would look similar to the WebDriver command unit tests.

Comment 10

[Andreas Tolfsen ❲:ato❳]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #9)

Please note that it would also be great to have Rust
unit tests under geckodriver/src which verify the
serialization/deserialization with Serde. Those would look similar
to the WebDriver command unit tests.

Could you update your comment #0 to that effect? I wonder if it
makes sense to track both these things—wdpsec and Rust unit
tests—separately, as blockers for this bug?

Comment 11

[Kriti Singh]

Hi :ato, I had written some starter tests for Get/Set Context but upon running them using ./mach wpt <path_to_test_file>, they gave a TIMEOUT, I initially thought it was my code but even the screenshot.py in the take_full_screenshot gives the same result. I suppose there is some other way to run python wdspec tests but can't find it in docs. Could you help me with that? Thanks.

Comment 12

[Andreas Tolfsen ❲:ato❳]

I’ve not experienced that before. This is what I get when I run
the take_full_screenshot/screenshot.py test on my system:
https://gist.github.com/andreastt/6b9b464d49ea3b478ac49dd365d07e93

Can you post the output of the command, including any stacktraces?
If it still fails, try the command with --webdriver-arg="-vv" as
an argument to get more fine-grained output from geckodriver and
Firefox.

Comment 13

[Kriti Singh]

I tried with the flag --webdriver-arg="-vv" but the results are same. Here are the logs. https://pastebin.com/UbmgSuiJ
Thanks.

Comment 14

[Andreas Tolfsen ❲:ato❳]

Hm, I don’t know about this. I’ve filed
https://bugzilla.mozilla.org/show_bug.cgi?id=1541812.