1694828 - Enable Fission mochitest-media tests on more platforms

Status: RESOLVED FIXED

(Testing :: Mochitest, task)

Description

[Chris Peterson [:cpeterson]]

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

• debug and opt builds
  • Where “shippable” opt builds available, we prefer to test shippable opt builds on central and plain opt builds on autoland.
• autoland and central
• We prefer to run tests with Fission + WebRender (“qr”) where available.

Add platforms:

• linux1804-64-qr/opt on autoland (we’re already testing opt on central; we need opt on autoland)
• linux1804-64-asan/opt
• linux1804-64-tsan/opt
• windows10-64-qr/debug (we’re already testing debug on central; we need debug on central)

Remove platforms:

• linux1804-64 opt fission without qr (because we already test linux1804-64 opt fission+qr)
• linux1804-64-qr/debug fission without qr (because we already test linux1804-64-qr/debug fission+qr)
• windows10-64 opt fission without qr (because we already test windows10-64 opt fission+qr)

Comment 1

[Alastor Wu [:alwu]]

Is there any media related failures that causes not being able to run some test tasks on certain platforms? I checked the doc in the comment1, but I couldn't find that. Feel free to ping/NI me if you need media team to look any failure test. Thank you.

Comment 2

[Chris Peterson [:cpeterson]]

(In reply to Alastor Wu [:alwu] from comment #1)

Is there any media related failures that causes not being able to run some test tasks on certain platforms? I checked the doc in the comment1, but I couldn't find that. Feel free to ping/NI me if you need media team to look any failure test. Thank you.

We are currently running the mochitest-media tests with Fission on Linux (test-linux1804-64-qr/debug-mochitest-media-fis-e10s). I don't know if there are mochitest-media failures with Fission on other platforms.

If you or @ahal write the patch to enable mochitest-media for these additional platforms and see any new failures in a Try run, I can file new bugs for those individual test failures. We can skip those individual tests and and land the patch to enable the other tests in this bug. As each individual test gets fixed in those new bugs, the engineer fixing that test can re-enable it in those bugs.

What do you recommend?

Comment 3

[Alastor Wu [:alwu]]

Sounds good to me. Once Fission team encounters any new error during enabling tests on new platform, feel free to NI me.

Comment 4

[Chris Peterson [:cpeterson]]

Assigning remaining "Enable Fission tests on more platforms" bugs to ahal.

Comment 5

[Andrew Halberstadt [:ahal]]

This bug has been added to releng's backlog and will be picked up sometime in the next few weeks. I'll likely end up taking this, but unassigning myself for now in case someone else ends up helping out.

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694828 - [ci] mochitest-media: Stop running fission tasks without webrender, r?jmaher

These will run with webrender enabled instead. This removes the following tasks
on central:

< test-linux1804-64-shippable/opt-mochitest-media-fis-e10s
< test-linux1804-64/debug-mochitest-media-fis-e10s
< test-windows10-64-shippable/opt-mochitest-media-fis-e10s

It does not remove any tasks on autoland.

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694828 - [ci] mochitest-media: Enable webrender opt linux64 and windows10 on autoland, r?jmaher

These will run with webrender enabled instead. This adds the following
configurations on autoland:

test-linux1804-64-qr/opt-mochitest-media-fis-e10s
test-windows10-64-qr/opt-mochitest-media-fis-e10s

Depends on D110157

Comment 8

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694828 - [ci] mochitest-media: Enable linux64 asan with fission, r?jmaher

This enables the following tasks on autoland and central:

test-linux1804-64-asan/opt-mochitest-media-fis-e10s

Depends on D110158

Comment 9

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694828 - [ci] mochitest-media: Enable linux64 tsan with fission

This enables the following tasks on autoland and central:

test-linux1804-64-tsan/opt-mochitest-media-fis-e10s

Depends on D110159

Comment 10

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694828 - [ci] mochitest-media: Enable windows10 debug with fission, r?jmaher

Depends on D110160

Comment 11

[Andrew Halberstadt [:ahal]]

Overall looks pretty green save for an issue with Windows 10 debug and some tsan intermittents (though that platform is normally flaky, so hard to say if this is fission or just normal):
https://treeherder.mozilla.org/jobs?repo=try&revision=4810aaf69994f94d946527036635ce2c24ec7f41

I structured the series in a way that the green stuff can all go ahead and land and I'll see if we can get green on central via disabling tomorrow.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b5dcbd5f683e
[ci] mochitest-media: Stop running fission tasks without webrender, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/a2eebc3bc2fa
[ci] mochitest-media: Enable webrender opt linux64 and windows10 on autoland, r=jmaher

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2656b8cce382
[ci] mochitest-media: Enable linux64 asan with fission, r=jmaher

Comment 14

[Andrew Halberstadt [:ahal]]

Hey Chris, fyi I only had to disable one test to get Windows 10 debug green:
https://hg.mozilla.org/try/rev/254ded54737a882577b6ffaafd8f2bd9a4d539df#l1.15

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1bf93e3d2eeb
[ci] mochitest-media: Enable linux64 tsan with fission r=jmaher
https://hg.mozilla.org/integration/autoland/rev/db1db3699d77
[ci] mochitest-media: Enable windows10 debug with fission, r=jmaher

Comment 16

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/b5dcbd5f683e
https://hg.mozilla.org/mozilla-central/rev/a2eebc3bc2fa
https://hg.mozilla.org/mozilla-central/rev/2656b8cce382

Comment 17

[Chris Peterson [:cpeterson]]

(In reply to Andrew Halberstadt [:ahal] from comment #14)

Hey Chris, fyi I only had to disable one test to get Windows 10 debug green:
https://hg.mozilla.org/try/rev/254ded54737a882577b6ffaafd8f2bd9a4d539df#l1.15

Thanks! I filed bug 1702071 for that test failure on Windows 10 debug.

Comment 18

[Sandor Molnar]

https://hg.mozilla.org/mozilla-central/rev/1bf93e3d2eeb
https://hg.mozilla.org/mozilla-central/rev/db1db3699d77