1801347 - wayland for docker-worker testers

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Posix OS, enhancement)

Description

[:dhouse]

spike/research to enable testing Firefox with wayland on linux hosts

Comment 1

[:dhouse]

https://bugzilla.mozilla.org/show_bug.cgi?id=1613689 - When migrating testers from AWS to GCP, we first attempted using monopacker to build the host image the same as we did/do for builders. It did not work and so we imported the disk image from AWS, and we were left on Ubuntu 14.04 for the host.
Current testing running in docker-worker on this host image uses xvfb to provide x11.

Comment 2

[:dhouse]

https://bugzilla.mozilla.org/show_bug.cgi?id=1756660 started as a spike to try getting wayland to work on a GCP VM without a gpu, with GCP's virtual display, or fully-headless.
tldr: wayland on Ubuntu currently needs a gpu. The wayland+weston source starting end of Sept'22 worked without a gpu.

In testing, I was only able to get X11 to work with the gcp virtual display and not wayland.
Following that, I found I could not get wayland to run headless (both with virtual display enabled and disabled) under Ubuntu releases (attempted with 22,20,18). In every attempt, I had failures or failover to X11. Searching and attempts included Gnome (mutter wayland drops to x11 I think because of no gpu), xpra, xvfb/xephyr replacements, and docker containerizing of gui apps like x11docker that suggests wayland provision but are limited the same as direct on-host wayland to require a gpu. Finally, building from the (end of Sept'22) wayland+weston source, weston ran on wayland and fully headless on gcp.
Following the x11docker wiki, https://github.com/mviereck/x11docker/wiki/How-to-provide-Wayland-socket-to-docker-container, the headless wayland worked for a container image to use the running external wayland (tested with x11docker/cinnamon and downloading+executing firefox nightly).

Comment 3

[:dhouse]

Attachment: Dockerfile.build

Comment 4

[:dhouse]

Because our test hosts are running Ubuntu1404, setup of wayland on the host seemed too difficult (old system libraries) and running in a container alongside the test container seemed reasonable.
[attached dockerfile for building wayland+weston from gitlab.freedesktop]

Comment 5

[:dhouse]

Attachment: Dockerfile.serve

Comment 6

[:dhouse]

Attachment: Dockerfile.test

Comment 7

[:dhouse]

example test:

sudo docker build -t weston -f Dockerfile.build .
sudo docker build -t waytest -f Dockerfile.serve .
sudo docker build -t tester -f Dockerfile.test .
sudo rm -rf waysocket; mkdir -p waysocket; chmod 0700 waysocket; sudo chown 0:0 waysocket; sudo docker run --rm -t -d --env XDG_RUNTIME_DIR=/tmp/wayland --env WESTON_BACKEND=rdp --env WAYLAND_DEBUG=1 --env MESA_DEBUG=1 --env EGL_LOG_LEVEL=error --publish 13389:3389 --volume $PWD/waysocket:/tmp/wayland waytest 
curl -L -X GET "https://download.mozilla.org/?product=firefox-nightly-latest-ssl&os=linux64&lang=en-US" -o firefox.tar.bz2
tar xjf firefox.tar.bz2
waysock=1; sudo docker run --rm -ti -e XDG_RUNTIME_DIR=/tmp -e XDG_SESSION_TYPE=wayland -e WAYLAND_DISPLAY=wayland-${waysock} -e GDK_BACKEND=wayland -v $PWD/waysocket/wayland-${waysock}:/tmp/wayland-${waysock} --user=0: --volume ${PWD}/firefox/:/var/tmp/firefox/ --env MOZ_ENABLE_WAYLAND=1 tester /var/tmp/firefox/firefox about:support

output from final container:

ALSA lib seq_hw.c:466:(snd_seq_hw_open) open /dev/snd/seq failed: No such file or directory
...

connecting with rdp from remote shows firefox displaying about:support, scrolling down wayland is listed as window protocol, mesa llvmpipe as gpu #1

Comment 8

[:dhouse]

Attachment: Dockerfile.serve

Comment 9

[:dhouse]

Attachment: Dockerfile.test

Comment 10

[:dhouse]

Attachment: 2022-10-11-133414_weston-ff-nightly107(1).png

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

:dhouse, given https://searchfox.org/mozilla-central/source/taskcluster/docker/ubuntu1804-test/Dockerfile, the above files seem to be lacking. I assume the .test is meant for a test image, I am not sure though- are these dockerfiles related? Can you build a file that works with with taskcluster so we can work on greening up tests?

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

also this reference scripts which do a lot with the config, such as:
https://searchfox.org/mozilla-central/source/taskcluster/scripts/tester/test-linux.sh

^ this script does a lot with xwindows setup, and I don't see anything in the attached files that indicate these scripts are ran/referenced/updated.

Comment 13

[:dhouse]

The attached dockerfile's are standalone to test no-gpu wayland like in https://bugzilla.mozilla.org/show_bug.cgi?id=1801347#c7

We can set up the software-rendered wayland like this as an alternative path to the xwindows setup in-tree. I haven't touched those in-tree docker files or scripts before, but it would be possible.

Alternatively, wayland could be provided with cloud instances that have gpu's or with hardware such as the moonshots. What do you think? That may be more reliable than the llvmpipe/software opengl from this experiment.

Comment 14

[Andrew Erickson [:aerickson]]

We've implemented Wayland in a VM (via Virtualbox). Spike complete.

See Bug 1756660 for more info.