1917216 - Extend headless Puppeteer Wd job to also run on Windows and MacOS

Status: ASSIGNED

(Remote Protocol :: Agent, enhancement, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We have official support for Puppeteer via WebDriver BiDi. As such we should test Puppeteer on all desktop platforms. Right now it's only done for Linux because there was a limitation to this platform in the early days for source-test jobs. Meanwhile source-test jobs like python-test run on all these platforms. So we should take a look and also extend Puppeteer Wd jobs to Windows and MacOS.

This work should be done before we start testing in headful mode (bug 1669435).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I would like to propose this bug as addition for M13.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I'm trying to get the changes into our Taskcluster configuration for Puppeteer jobs but it turns out to be a bit hard. Mostly because source-tests are currently not really run on other platforms than Linux with a Firefox binary as dependency.

Here are the issues that I have:

• I was able to add all the builds as dependencies and specified the binary archives within a fetches block (under a by-platform section). That downloads the builds and automatically extracts them. But sadly the reference to the extracted binary that can be launched is not known, and it differs between platforms. I assume that there is no environment variable or such that gets set for the binary location?
• With specifying extract: false for the build fetches I tried to extract the archives myself in the run section by using mozinstall. But here we do not have a reference to the installer file as well (I filed bug 1926592 for that). I can use a workaround to specify it myself, but also with doing that I have trouble to get mozinstall running on at least Linux because of issues with the custom installed version of Python. Maybe I would have to create our own venv just for the installation? If yes, I could use export FIREFOX_BINARY=$(mozinstall GECKO_INSTALLER_URL --destination=$MOZ_FETCHES_DIR) to get the binary exported. Here I would wish that Taskcluster would do that automatically. Would that be possible? I cannot find the place of code that is used to extract/install the build archives. Where are those?

Here a latest try build for trying to get the binary installed on Linux:
https://treeherder.mozilla.org/jobs?repo=try&revision=99066cfe84f86dfed7c4caf3b6d44cbc39b9b669

Andrew, do you have some suggestions? Thanks!

Comment 3

[Andrew Halberstadt [:ahal]]

The code to extract the fetch is here:
https://searchfox.org/mozilla-central/rev/18f09bdf36a62ea7079c018301f1d257f71f655b/third_party/python/taskcluster_taskgraph/taskgraph/run-task/fetch-content#383

We shouldn't add any Gecko or Firefox specific logic to that script though. I'm not really sure what you mean by "Taskcluster would do that automatically".

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrew Halberstadt [:ahal] from comment #3)

The code to extract the fetch is here:
https://searchfox.org/mozilla-central/rev/18f09bdf36a62ea7079c018301f1d257f71f655b/third_party/python/taskcluster_taskgraph/taskgraph/run-task/fetch-content#383

Is that really the place where the binary archive or DMG is extracted? I miss the part for DMG actually.

We shouldn't add any Gecko or Firefox specific logic to that script though. I'm not really sure what you mean by "Taskcluster would do that automatically".

So it was told to us in the early days that the Puppeteer job should run as a source test job given that this is the future and we should not start using mozharness for it as well. It was fine for us given that we didn't know about the limitations for other platforms by then. But now trying to extend the job to Windows and MacOS as well shows a couple of issues. So is that no longer true that source test job is right here and we have to switch to a normal test job, including the creation of a mozharness script + all the packaging steps? This will be quite a bit of work.

Or what is your suggestion to get the binary archives extracted (including DMG) and have a reference to the actual binary to start. I only know mozinstall which is capable of unextracting on all platforms and also returning the full path to the binary. It would be great to just have to specify the build fetches and then have an eg. environment variable that can be used in the run section to reference the path of the binary.