Closed Bug 1694828 Opened 3 years ago Closed 3 years ago

Enable Fission mochitest-media tests on more platforms

Categories

(Testing :: Mochitest, task)

task

Tracking

(Fission Milestone:M7a, firefox-esr78 wontfix, firefox86 wontfix, firefox87 wontfix, firefox88 wontfix, firefox89 fixed)

RESOLVED FIXED
89 Branch
Fission Milestone M7a
Tracking Status
firefox-esr78 --- wontfix
firefox86 --- wontfix
firefox87 --- wontfix
firefox88 --- wontfix
firefox89 --- fixed

People

(Reporter: cpeterson, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(5 files)

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

  • debug and opt builds
    • Where “shippable” opt builds available, we prefer to test shippable opt builds on central and plain opt builds on autoland.
  • autoland and central
  • We prefer to run tests with Fission + WebRender (“qr”) where available.

Add platforms:

  • linux1804-64-qr/opt on autoland (we’re already testing opt on central; we need opt on autoland)
  • linux1804-64-asan/opt
  • linux1804-64-tsan/opt
  • windows10-64-qr/debug (we’re already testing debug on central; we need debug on central)

Remove platforms:

  • linux1804-64 opt fission without qr (because we already test linux1804-64 opt fission+qr)
  • linux1804-64-qr/debug fission without qr (because we already test linux1804-64-qr/debug fission+qr)
  • windows10-64 opt fission without qr (because we already test windows10-64 opt fission+qr)
Blocks: 1694830
Fission Milestone: M7 → M7a
Priority: P2 → P3
Severity: N/A → --
Component: Audio/Video: Playback → Mochitest
Priority: P3 → --
Product: Core → Testing

Is there any media related failures that causes not being able to run some test tasks on certain platforms? I checked the doc in the comment1, but I couldn't find that. Feel free to ping/NI me if you need media team to look any failure test. Thank you.

Flags: needinfo?(cpeterson)

(In reply to Alastor Wu [:alwu] from comment #1)

Is there any media related failures that causes not being able to run some test tasks on certain platforms? I checked the doc in the comment1, but I couldn't find that. Feel free to ping/NI me if you need media team to look any failure test. Thank you.

We are currently running the mochitest-media tests with Fission on Linux (test-linux1804-64-qr/debug-mochitest-media-fis-e10s). I don't know if there are mochitest-media failures with Fission on other platforms.

If you or @ahal write the patch to enable mochitest-media for these additional platforms and see any new failures in a Try run, I can file new bugs for those individual test failures. We can skip those individual tests and and land the patch to enable the other tests in this bug. As each individual test gets fixed in those new bugs, the engineer fixing that test can re-enable it in those bugs.

What do you recommend?

Flags: needinfo?(cpeterson) → needinfo?(alwu)

Sounds good to me. Once Fission team encounters any new error during enabling tests on new platform, feel free to NI me.

Flags: needinfo?(alwu)

Assigning remaining "Enable Fission tests on more platforms" bugs to ahal.

Assignee: nobody → ahal

This bug has been added to releng's backlog and will be picked up sometime in the next few weeks. I'll likely end up taking this, but unassigning myself for now in case someone else ends up helping out.

Assignee: ahal → nobody
Assignee: nobody → ahal
Status: NEW → ASSIGNED

These will run with webrender enabled instead. This removes the following tasks
on central:

< test-linux1804-64-shippable/opt-mochitest-media-fis-e10s
< test-linux1804-64/debug-mochitest-media-fis-e10s
< test-windows10-64-shippable/opt-mochitest-media-fis-e10s

It does not remove any tasks on autoland.

These will run with webrender enabled instead. This adds the following
configurations on autoland:

test-linux1804-64-qr/opt-mochitest-media-fis-e10s
test-windows10-64-qr/opt-mochitest-media-fis-e10s

Depends on D110157

This enables the following tasks on autoland and central:

test-linux1804-64-asan/opt-mochitest-media-fis-e10s

Depends on D110158

This enables the following tasks on autoland and central:

test-linux1804-64-tsan/opt-mochitest-media-fis-e10s

Depends on D110159

Overall looks pretty green save for an issue with Windows 10 debug and some tsan intermittents (though that platform is normally flaky, so hard to say if this is fission or just normal):
https://treeherder.mozilla.org/jobs?repo=try&revision=4810aaf69994f94d946527036635ce2c24ec7f41

I structured the series in a way that the green stuff can all go ahead and land and I'll see if we can get green on central via disabling tomorrow.

Keywords: leave-open
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b5dcbd5f683e
[ci] mochitest-media: Stop running fission tasks without webrender, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/a2eebc3bc2fa
[ci] mochitest-media: Enable webrender opt linux64 and windows10 on autoland, r=jmaher
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2656b8cce382
[ci] mochitest-media: Enable linux64 asan with fission, r=jmaher
Attachment #9212330 - Attachment description: WIP: Bug 1694828 - [ci] mochitest-media: Enable linux64 tsan with fission → Bug 1694828 - [ci] mochitest-media: Enable linux64 tsan with fission
Attachment #9212331 - Attachment description: WIP: Bug 1694828 - [ci] mochitest-media: Enable windows10 debug with fission → Bug 1694828 - [ci] mochitest-media: Enable windows10 debug with fission, r?jmaher

Hey Chris, fyi I only had to disable one test to get Windows 10 debug green:
https://hg.mozilla.org/try/rev/254ded54737a882577b6ffaafd8f2bd9a4d539df#l1.15

Flags: needinfo?(cpeterson)
Keywords: leave-open
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1bf93e3d2eeb
[ci] mochitest-media: Enable linux64 tsan with fission r=jmaher
https://hg.mozilla.org/integration/autoland/rev/db1db3699d77
[ci] mochitest-media: Enable windows10 debug with fission, r=jmaher
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 89 Branch
Depends on: 1702071

(In reply to Andrew Halberstadt [:ahal] from comment #14)

Hey Chris, fyi I only had to disable one test to get Windows 10 debug green:
https://hg.mozilla.org/try/rev/254ded54737a882577b6ffaafd8f2bd9a4d539df#l1.15

Thanks! I filed bug 1702071 for that test failure on Windows 10 debug.

Flags: needinfo?(cpeterson)
You need to log in before you can comment on or make changes to this bug.