1912292 - Run puppeteer tests in several chunks

Status: RESOLVED FIXED

(Remote Protocol :: Agent, task, P3)

Description

[Julian Descottes [:jdescottes]]

At the moment puppeteer tests run in single chunks. As the test suite grows bigger we had to use several workarounds to keep the gzipped log size < 5MB (eg Bug 1911382).

We should update the tests to allow to run them in chunks on CI.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

On GitHub the Puppeteer repository uses the shard feature via GitHub Actions. But that's something that we can not use and mocha.js doesn't have it natively.

We should check if our chunking algorithm for other test suites would help here as well.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I've asked Alex and he pointed me to: https://github.com/puppeteer/puppeteer/blob/main/.github/workflows/ci.yml#L203

So it's basically a command like:

npm run test -- --shard '1-2'

to run the first half of tests.

All dependencies are getting installed automatically when Puppeteer is installed.

So that seems to be pretty easy to implement.

Comment 3

[Alexandra Borovova [:Sasha]]

Hi Andrew,
I'm trying to add chunking to the puppeteer source test job, but it doesn't seem to work. I've tried adding chunks property in the various places, but the schema validation would always fail (e.g. here) and since we don't have the chunking in any other source test jobs, I started doubting that it actually works at all for this type. Is that so, or am I just doing something wrong?

Comment 4

[Andrew Halberstadt [:ahal]]

Yes, the chunking you are thinking of is specific to the test kind. That said, Taskgraph has built-in chunking transforms that you can use now. See:
https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/transforms/chunking.html

You'll need to add this transform to the kind.yml first.

Comment 5

[Alexandra Borovova [:Sasha]]

Attachment: Bug 1912292 - Run puppeteer tests in several chunks.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/116ba4014954
Run puppeteer tests in several chunks. r=webdriver-reviewers,whimboo,ahal

Comment 7

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/116ba4014954