Open Bug 1852986 Opened 2 years ago Updated 2 months ago

Enable Marionette unit tests with Wayland

Categories

(Testing :: Marionette Client and Harness, task, P3)

Product:
Testing
Component:
Marionette Client and Harness
Type:
task
Points:
3

Tracking

(Not tracked)

People

(Reporter: whimboo, Unassigned)

References

(Depends on 2 open bugs, Blocks 1 open bug)

Details

(Whiteboard: [webdriver:m19])

Attachments

(2 obsolete files)

Similar to bug 1850549 for Firefox UI tests it would be great to get the Marionette (Mn) tests running on Wayland as well. This would allow us to have an environment which is much closer to our WebDriver users in real life.

There are test failures visible for those tests that move the window. Hereby the minimal x position of the window now seems to be 26px. I wonder if with Ubuntu 22.04 (and Wayland) we have a left menu visible, or do we still turn this off?

Andrew, do you know?

Flags: needinfo?(ahal)
Flags: needinfo?(stransky)

I do see that locally:
./mach test testing/marionette/harness/marionette_harness/tests/unit/test_window_rect.py
Looks like Wayland issue.

Assignee: nobody → stransky
Component: Task Configuration → Widget: Gtk
Flags: needinfo?(stransky)
Priority: -- → P2
Product: Firefox Build System → Core
Flags: needinfo?(ahal) → needinfo?(stransky)

Martin, can you please file a new bug specific to this Wayland issue? This bug is really to get the Marionette job running under Wayland. Please mark this bug being blocked by the Wayland bug. Thanks.

Component: Widget: Gtk → Task Configuration
Product: Core → Firefox Build System
Assignee: stransky → nobody
Priority: P2 → --
Assignee: nobody → stransky
Status: NEW → ASSIGNED
Flags: needinfo?(stransky)

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #4)

Martin, can you please file a new bug specific to this Wayland issue? This bug is really to get the Marionette job running under Wayland. Please mark this bug being blocked by the Wayland bug. Thanks.

Okay, filed as Bug 1857571

Comment on attachment 9357108 [details]
Bug 1852986 [Linux] Add moz:wayland system capability for tests r?ahal

Revision D190317 was moved to bug 1857571. Setting attachment 9357108 [details] to obsolete.

Attachment #9357108 - Attachment is obsolete: true

Comment on attachment 9357110 [details]
Bug 1852986 [Linux] Don't test window positioning on Wayland r?ahal

Revision D190318 was moved to bug 1857571. Setting attachment 9357110 [details] to obsolete.

Attachment #9357110 - Attachment is obsolete: true

Hm, it is unfortunate that Wayland doesn't support resizing and positioning a window. It will put us into trouble if Wayland would be the only WindowManager that we could use in CI for testing. We definitely cannot loose the test coverage for positioning and resizing windows on Linux. Andrew, will there be an optional non-Wayland test machine be available for testing?

Assignee: stransky → nobody
Status: ASSIGNED → NEW
Flags: needinfo?(ahal)

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #10)

Hm, it is unfortunate that Wayland doesn't support resizing and positioning a window. It will put us into trouble if Wayland would be the only WindowManager that we could use in CI for testing. We definitely cannot loose the test coverage for positioning and resizing windows on Linux. Andrew, will there be an optional non-Wayland test machine be available for testing?

Window resizing is supported of course. You set window size and wayland compositor places it automatically on screen. It can be moved by user etc. but application itself can't change toplevel window position.

Popup windows can be positioned relatively to its parent so menu/popups works as usual. Popup constraints (place popup inside screen area) is managed by wayland by extra call where relative popup coordinates is send to application after placement request.

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #10)

Hm, it is unfortunate that Wayland doesn't support resizing and positioning a window. It will put us into trouble if Wayland would be the only WindowManager that we could use in CI for testing. We definitely cannot loose the test coverage for positioning and resizing windows on Linux. Andrew, will there be an optional non-Wayland test machine be available for testing?

Yeah, I mean at least for the foreseeable future. As long as a significant portion of users are using X11, I imagine we'll keep a test platform around for them. I guess eventually the number of X11 users will be too small to justify spending CI dollars on, but I can't imagine that happening anytime soon.

Flags: needinfo?(ahal)

(In reply to Andrew Halberstadt [:ahal] from comment #12)

Yeah, I mean at least for the foreseeable future. As long as a significant portion of users are using X11, I imagine we'll keep a test platform around for them. I guess eventually the number of X11 users will be too small to justify spending CI dollars on, but I can't imagine that happening anytime soon.

That means that we could run Mn jobs by default with the Wayland worker, and maybe on mozilla-central or very rarely on autoland with the X11 worker?

Sasha, I assume we also need to block this bug on both other issues that you noticed, is that correct?

Flags: needinfo?(aborovova)

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #14)

Sasha, I assume we also need to block this bug on both other issues that you noticed, is that correct?

I don't think so. From what I can see, we don't have the marionette tests for maximization and minimization of the window (at least there are no related failures).

Flags: needinfo?(aborovova)

Alright. I just pushed a try build to see how it actually looks like with Wayland enabled:
https://treeherder.mozilla.org/jobs?repo=try&revision=d7d6eb0c94c3165b70ba573caf827779b0c8c931

Depends on: 1924781
Depends on: 1924784

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #16)

Alright. I just pushed a try build to see how it actually looks like with Wayland enabled:
https://treeherder.mozilla.org/jobs?repo=try&revision=d7d6eb0c94c3165b70ba573caf827779b0c8c931

It shows two perma failures that need to be addressed before we can enable Mn Wayland jobs by default.

Depends on: 1923833

Lets actually move this bug into the Marionette component so that we can better track this. I've also pushed a new try build to check if something has changed since we tried the last time about 9 months ago.

https://treeherder.mozilla.org/jobs?repo=try&revision=5f205683ed7f9d06232d6dd987ebf170d537da40

Component: Task Configuration → Marionette Client and Harness
Product: Firefox Build System → Testing
Points: --- → 3
Priority: -- → P3
Summary: Enable Marionette tests with Wayland → Enable Marionette unit tests with Wayland
Whiteboard: [webdriver:m17]
No longer depends on: 1924781, 1924784
See Also: → 1977223

The following try build shows that we are blocked on setting the correct window size for reftests:
https://treeherder.mozilla.org/logviewer?job_id=522917744&repo=try&task=TJD5oGHwQ9ixbpoULd9rXA.0

Depends on: 1982745

Could you run that with MOZ_LOG="Widget:5 WidgetScreen:5" env variable? We should compare requested and real sizes. It may be related to used screen size or possible titlebar or so.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #20)

Could you run that with MOZ_LOG="Widget:5 WidgetScreen:5" env variable? We should compare requested and real sizes. It may be related to used screen size or possible titlebar or so.

Yeah, Emilio found a bug and provided a patch on that other bug already. So the only remaining issue for us before we can enable Wayland is the window restore logic when a window got minimized. I hope that I can find some time soon.

(In reply to Henrik Skupin [:whimboo][⌚️UTC+2] from comment #21)

(In reply to Martin Stránský [:stransky] (ni? me) from comment #20)

Could you run that with MOZ_LOG="Widget:5 WidgetScreen:5" env variable? We should compare requested and real sizes. It may be related to used screen size or possible titlebar or so.

Yeah, Emilio found a bug and provided a patch on that other bug already. So the only remaining issue for us before we can enable Wayland is the window restore logic when a window got minimized. I hope that I can find some time soon.

Can you point me to that issue? I'm afraid Wayland app can't itself switch from minimized state unless it has any window opened with active focus - then xdg_activation token can be used to transfer focus.

This is bug 1923833 which is on the dependency list of this bug, and you already helped us over there. But I sadly haven't had any time yet to dig into due to other priorities.

Sure, will look at it.

Whiteboard: [webdriver:m17] → [webdriver:m19]
Depends on: 1991123
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: