1626089 - Run with multiple processes in CI

Status: RESOLVED FIXED

(Testing :: web-platform-tests, enhancement)

Description

[James Graham [:jgraham]]

This should allow a speedup at the cost of perhaps introducing some instability. It doesn't work on android since we can only run a single copy of the geckoview test app.

Comment 1

[James Graham [:jgraham]]

Attachment: Bug 1626089 - Run two wpt processes in CI,

This runs wpt with --processes=2 in CI, excluding wdspec tests where this
causes obvious test failures and android where we can't have more than a
single instance. This should provide some speedup at the cost of possibly
increased instability. So following landing this it will be necessary to
look out for an increased rate of intermittents.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0eea7f51b16a
Run two wpt processes in CI, r=jmaher

Comment 3

[Noemi Erli[:noemi_erli]]

Backed out changeset 0eea7f51b16a (Bug 1626089) for causing web platform failures in bundle.https.html CLOSED TREE

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=d5f19f6bda1bd5231252a5d812c55dadad1f16f4&revision=0eea7f51b16a43ed2b00dbd943debb9c83465c51&selectedJob=295629689

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=295629689&repo=autoland&lineNumber=6334

Backout: https://hg.mozilla.org/integration/autoland/rev/9af589864188fc4061a020c45914f54fa9f67b0d

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

this seems to be a failuree specifically on windows 7 shippable:
TEST-UNEXPECTED-FAIL | /webrtc/protocol/bundle.https.html | bundles on the first transport and closes the second - assert_not_equals: got disallowed value undefined
TEST-UNEXPECTED-OK | /webrtc/protocol/bundle.https.html | expected TIMEOUT

I couldn't see anything in the log file that was an indicator of what went wrong or what is different from the normal timeout.

Comment 5

[James Graham [:jgraham]]

Looks like that test is already marked as intermittent on some Windows configurations, so I think it might be enough to mark it as always intermittent on windows opt.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7b7fbd8c6c36ed9043311240922377bf08ce6d20 for a try push with that change.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c4b80e083d0b
Run two wpt processes in CI, r=jmaher

Comment 7

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/c4b80e083d0b

Comment 8

[James Graham [:jgraham]]

Attachment: Bug 1626089 - Disable multiple processes for fission builds,

Running this made intermittents too frequent to manage on this platform.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e4034d6f39e5
Disable multiple processes for fission builds, r=jmaher

Comment 10

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/e4034d6f39e5

Comment 11

[Andreea Pavel [:apavel]]

I've tested backing this out on try, the regressions here do not show up anymore, however, there is this new failure: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception&revision=e28069b1e1cb95c5866d149cc0d5ae8e6b2674ba&selectedJob=297299804

Should we move on with backing this out on autoland?

Additional fixes for the regression don't seem to work in some cases: https://bugzilla.mozilla.org/show_bug.cgi?id=1626943#c5

Comment 12

[Andreea Pavel [:apavel]]

Given that the failure rate in the regression bugs has dropped, the backout is no longer necessary?