1566182 - Annotate mochitests that fail with Fission enabled

Status: RESOLVED FIXED

(Testing :: Mochitest, task, P2)

Description

[Kris Maglione [:kmag]]

This will allow us to enable running Fission mochitests on normal integration branches, and to see actual regressions from unexpected failures that would otherwise be lost in the noise.

The tests will be annotated with fail-if = fission annotations, which will need to be removed as soon as a test starts passing with Fission enabled so that it can't regress.

Comment 1

[Kris Maglione [:kmag]]

Attachment: Bug 1566182: Annotate mochitests that fail with Fission enabled. r=mccr8

My preference was to annotate most of the failing tests with fail-if so that
if they start passing, the fail-if needs to be removed and they need to keep
passing. That doesn't work for tests that timeout, or which trigger failures
from their cleanup functions, however, so those tests need skip-if. And tests
with fail in their cleanup functions likely leave the browser in an
inconsistent state for subsequent tests, anyway, so really should be skipped
regardless.

There are some remaining tests which still fail because of crashes. I chose
not to skip them here, but to fix the crashes in separate bugs instead.

Comment 2

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bcdcbeb490e137f4d093f63394080917d6cd7ca4
Bug 1566182: Annotate mochitests that fail with Fission enabled. r=mccr8

Comment 3

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/bcdcbeb490e1