1865353 - Enable web-platform-tests with Wayland

Status: NEW

(Firefox Build System :: Task Configuration, task)

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Hi Andrew, is there any timeline when one will try to run the web-platform tests via Wayland? Could we maybe get a try build with all the wpt tests running to see if any failures exist specifically to Wayland that we would have to address? Thanks!

Comment 2

[Andrew Halberstadt [:ahal]]

Hey Henrik, apologies I'm not actively working on this. I should have removed myself as assignee.

But yes, it's possible to get some try pushes going. You can delete this line to enable them:
https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/web-platform.yml#89

Alternatively you can use ./mach try fuzzy --worker-override to override the existing worker alias to the wayland pool (I don't recall the exact string needed here though).

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thank you for the details! Removing just a line is simple enough to get started. I pushed a try build for all web-platform tests to just get an overview of possible blocking issue. If it turns out that tests pass most of the time I'll trigger another try build with a couple of retries to rule out intermittently failing tests.

https://treeherder.mozilla.org/jobs?repo=try&revision=66ac70688ab752238583175b75a76124c0ec3c18

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As the try build shows the different handling of window positioning / sizing with Wayland causes timeouts in Marionette for the WebDriver:SetWindowRect command. There is bug 1857571 to get this fixed.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I actually wonder about side-effects that we could expect especially for reftests when we cannot resize the window that is used for testing. By default we have a size of 800x600 but that would no longer be the case.

James, do you have had any situation like that in the past when a different window size caused issues?

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Note that we could actually set the MOZ_ENABLE_WAYLAND=0 environment variable to run the tests on X11 (XWayland). Maybe that's what we should do with web-platform tests to not have to run two configurations?

Comment 7

[James Graham [:jgraham]]

I think with wayland we can set the window size when it's opened, but we can't move the window to a specified position. Unfortunately WebDriver assumes that if you can do one you can do the other, so wdspec tests are pretty broken, but I hope reftests should be fine.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to James Graham [:jgraham] from comment #7)

I think with wayland we can set the window size when it's opened, but we can't move the window to a specified position.

It would be great if that's possible. We would have to check as best on bug 1857571. Sadly the new tab/window methods do not allow to pass neither width nor height.

Unfortunately WebDriver assumes that if you can do one you can do the other, so wdspec tests are pretty broken, but I hope reftests should be fine.

Should we maybe upgrade first to Ubuntu 22.04 with a Python 3.10 (right now we have 3.8) installation, and afterward check for Wayland? This would potentially reduce other possible side-effects that we haven't noticed yet.

Comment 9

[James Graham [:jgraham]]

I agree that updating workers first and enabling wayland second seems like it reduces risk. On the other hand I'm not sure how much advantage there is to updating workers on their own; if wpt is supposed to keep working with Python 3.8 for as long as mach does it in theory doesn't allow us to upgrade the Python version sooner. So I think this would only really make sense in the context of trying to get testing onto wayland.

(I also think we should consider updating WebDriver for this "you can set the window size but not position thing, and we should certainly ensure it works in BiDi).