1694830 - Enable Fission mochitest-media-gli tests on more platforms

Status: RESOLVED FIXED

(Testing :: Mochitest, task)

Description

[Chris Peterson [:cpeterson]]

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

• debug and opt builds
  • Where “shippable” opt builds available, we prefer to test shippable opt builds on central and plain opt builds on autoland.
• autoland and central
• We prefer to run tests with Fission + WebRender (“qr”) where available.

We should probably enable the Fission mochitest-media tests in bug 1694828 before enabling these mochitest-media-gli tests.

Add platforms:

• linux1804-64-qr/debug
• linux1804-64-qr/opt on autoland, shippable-qr opt on central
• linux1804-64-asan/opt
• linux1804-64-tsan/opt
• windows10-64-qr/debug
• windows10-64-qr/opt on autoland, shippable-qr opt on central

Comment 2

[Neha Kochar [:neha]]

These tests should be enabled latest by M7a milestone (the sooner the better so we can find and fix problems sooner).

Comment 3

[Chris Peterson [:cpeterson]]

Assigning remaining "Enable Fission tests on more platforms" bugs to ahal.

Comment 4

[Andrew Halberstadt [:ahal]]

This bug has been added to releng's backlog and will be picked up sometime in the next few weeks. I'll likely end up taking this, but unassigning myself for now in case someone else ends up helping out.

Comment 5

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Putting my name down here as as the media person to ping if issues come up. I can also look at enabling these myself and marking issues, but I'm not clear on which config needs to change to enable the gli test specifically.

Comment 6

[Andrew Halberstadt [:ahal]]

I could use the help and would be willing to teach you how to enable these tasks :)

This one is a bit trickier than the others since it is combining two variants. But maybe I can put up an initial patch that sorts that out and would make following patches to enable the tasks trivial.. I'll put up the prerequisite patch here either way.

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1694830 - [taskgraph] Create a 'fission-webgl-ipc' variant for mochitest-media, r?jmaher

Comment 8

[Andrew Halberstadt [:ahal]]

Bryce, with the attached patch (which can land as soon as it's reviewed) you'll be able to run these tasks via e.g ./mach try fuzzy --full -q "'mochitest-media-fis-gli"

If you're interested in picking this bug up I'd appreciate it! Basically just modify the run-on-projects line in my patch from:

fission-webgl-ipc: []

to something like:

fission-webgl-ipc:
    by-test-platform:
        (linux.*64|windows10-64)-qr/debug: ['autoland', 'mozilla-central']
        default: []

The above would run the fis-gli tasks on linux and Windows debug with webrender enabled. The by-test-platform key is kind of like a case statement where the task will receive the value from whichever regex it first matches. You'll need to add more regexes than in my example to fulfill the request in this bug.

To see a diff of how your change affects the taskgraph, you can run:

$ ./mach taskgraph target --target-kind test --fast --diff .~1 -p project=autoland

Substitute autoland with mozilla-central to see the impacts your change has there instead. Beware that using --diff will update your working directory (it will abort if working directory is dirty). Also note that if you are modifying anything other than the run-on-projects key, you should also pass in -J (without -J you just see the diff of added/removed tasks, not the diff of modifications to existing tasks).

Anyway, no worries if you can't get to this.. though it might be awhile until I or someone else has a chance to pick this up.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/64be1424df80
[taskgraph] Create a 'fission-webgl-ipc' variant for mochitest-media, r=jmaher

Comment 10

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/64be1424df80

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Thanks for the write up and the patch! That helps a lot. Grabbing this and seeing what I can do (holding NI to remind me to keep at it).

Tried a first run with just the changes suggested in comment 8 at https://treeherder.mozilla.org/jobs?repo=try&revision=8c9392801f5938dc74935dc6fc1ab35e4c0a7769 -- that looks mostly okay. The failing tests look to be known intermittents.

I'll add some more configurations and see if anything breaks.

Comment 12

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I think I misunderstood how this interacted with try initially, and thought I'd need to make modifications to get further flavours of the test to run on try, but this appears to not be true? Modifying the file above will just change what tests we're running on central and autoland?

Does it make sense following this that we'd end up with the same config here that we also arrived at for the fission tests in bug 1694828? That appears to be what I've arrived at. I'll push my patches shortly.

Comment 13

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1694830 - Enable fission-webgl-ipc media mochitests on linux64 + win10-64 debug builds. r?ahal

Comment 14

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1694830 - Enable fission-webgl-ipc media mochitests for linux64 and win10-64 opt builds on autoland. r?ahal

Depends on D110989

Comment 15

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1694830 - Enable fission-webgl-ipc media mochitests for linux64 and win10-64 shippable builds on central. r?ahal

Depends on D110990

Comment 16

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Attachment: Bug 1694830 - Enable fission-webgl-ipc media mochitests for linux64 tsan and asan builds. r?ahal

Depends on D110991

Comment 17

[Andrew Halberstadt [:ahal]]

Yeah, it makes sense. The config changes themselves are pretty easy, it's the testing and triaging failures in try that is more time consuming. Though in this case sounds like you may be in the clear.

I also asked for help because there are a lot of these bugs to work through :). So thanks for picking this up!

Comment 18

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Preparing to land. Try runs I've done look okay. I guess if these fail it'll be me as the traige owner who gets pinged anywho!

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2399b20b93fe
Enable fission-webgl-ipc media mochitests on linux64 + win10-64 debug builds. r=ahal
https://hg.mozilla.org/integration/autoland/rev/1b35023795a1
Enable fission-webgl-ipc media mochitests for linux64 and win10-64 opt builds on autoland. r=ahal
https://hg.mozilla.org/integration/autoland/rev/030c9c64e42a
Enable fission-webgl-ipc media mochitests for linux64 and win10-64 shippable builds on central. r=ahal
https://hg.mozilla.org/integration/autoland/rev/b3b783160873
Enable fission-webgl-ipc media mochitests for linux64 tsan and asan builds. r=ahal

Comment 20

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/2399b20b93fe
https://hg.mozilla.org/mozilla-central/rev/1b35023795a1
https://hg.mozilla.org/mozilla-central/rev/030c9c64e42a
https://hg.mozilla.org/mozilla-central/rev/b3b783160873