Add Ci jobs for Linux 64 to run Marionette and WebDriver tests under xvfb
Categories
(Remote Protocol :: Agent, task)
Tracking
(Not tracked)
People
(Reporter: whimboo, Unassigned)
References
Details
We got an issue report for WebDriver that the Set Window Rect command doesn't work with Firefox 104 onward (see bug 1789823). This specific case only happens when the tests are run under xvfb. Given that we do not have such a job at all in Taskcluster we missed that particular regression.
Given that quite a lot of web developers might run Selenium tests under xvfb we should add new CI jobs for both Marionette and Wdspec to run all of the tests under xvfb as well.
Joel, are you aware of any limitations by the that configuration? Or would it be enough to just set the need-xvfb flag in the Taskcluster job definition? We may have to adjust our mozharness scripts for sure to change the command to xvfb-run ....
| Reporter | ||
Comment 1•4 years ago
|
||
And to add, we should run these tests with a window manager like fluxbox being installed. Otherwise features like maximize and fullscreen might not necessarily work due to missing OS support.
| Reporter | ||
Comment 2•4 years ago
|
||
Lets wait until bug 1789823 has been fixed so that we know which requirements are actually necessary for such a job.
| Reporter | ||
Comment 3•4 years ago
|
||
Hi Joel, could you please have a look at comment 0? Would such a config be possible in our CI including fluxbox as simple window manager?
In Selenium CI we will do the following:
- name: Install Fluxbox
run: sudo apt-get -y install fluxbox
[..]
- name: Start Xvfb
run: Xvfb :99 &
- name: Start Fluxbox
run: fluxbox -display :99 &
Comment 4•4 years ago
|
||
we do run our CI with xvfb and a proper window manager, many tests run full screen. Are webdriver tests headless?
I don't know much about linux window managers, I assume this would be ok. Would this be a change for all the linux CI testing? We are looking to replace our CI with ubuntu 22.04, both X11 and Wayland (not XWayland though)- our first attempt will be using a VM (not docker image) as it will support audio/video/window manager much better than a docker container.
| Reporter | ||
Comment 5•4 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC -0800) from comment #4)
we do run our CI with xvfb and a proper window manager, many tests run full screen. Are webdriver tests headless?
Do you mean that we run all the tests with Xvfb and a window manager or only these which actually explicitly require Xvfb?
WebDriver tests would have to be best run with a normal window manager, headless and if doable via Xvfb + window manager. But if we always do the latter our coverage should be fine then.
I don't know much about linux window managers, I assume this would be ok. Would this be a change for all the linux CI testing? We are looking to replace our CI with ubuntu 22.04, both X11 and Wayland (not XWayland though)- our first attempt will be using a VM (not docker image) as it will support audio/video/window manager much better than a docker container.
This is only for Wdspec given that we should make sure to not regress anything for our test audience. Most of the time tests are run headless and with a default window manager, but in some cases it also still happens with xvfb.
Comment 6•4 years ago
|
||
there are a few jobs that don't run with xvfb, these are build jobs or release tasks. window manager is set by default for all docker tests:
https://searchfox.org/mozilla-central/source/taskcluster/gecko_taskgraph/transforms/job/mozharness_test.py#148
even headless tasks have xvfb/window manager.
| Reporter | ||
Comment 7•4 years ago
|
||
Ok, thanks Joel! That means that this bug is basically invalid.
Updated•4 years ago
|
Description
•