Closed Bug 1695083 Opened 3 years ago Closed 3 years ago

remove requirement in fission_filter to have a job that runs on central

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task

Tracking

(Fission Milestone:M7a, firefox88 fixed)

Status:
RESOLVED FIXED
Milestone:
88 Branch
Project Flags:
Fission Milestone M7a
Tracking Flags:
Tracking Status
firefox88 --- fixed

People

(Reporter: jmaher, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(4 files)

Bug 1695083 - [ci] Ensure browsertime tp6 ignores non-shippable fission platforms, r?#perftest-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Bug 1695083 - [ci] Don't run talos profiling tasks with fission, r?#perftest-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Bug 1695083 - [ci] Don't run raptor-unity-webgl on fission, r?#perftest-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Bug 1695083 - [taskgraph] Don't limit fission tasks to tasks that run on central, r?jmaher
48 bytes, text/x-phabricator-request
Details | Review

in bug 1693640 there is a question about how we can get run devtools tests w/fission on try with opt builds, not shippable builds. I looked into this and it comes down to the transforms fission_filter:
https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/transforms/tests.py#170

def fission_filter(task):
    return (
        runs_on_central(task)
        and task.get("e10s") in (True, "both")
        and get_mobile_project(task) != "fennec"
    )

if we remove the first line runs_on_central(task), then edits in the fission-run-on-projects such as https://searchfox.org/mozilla-central/source/taskcluster/ci/test/mochitest.yml#125 will work:

    fission-run-on-projects:
        by-test-platform:
            linux.*64(-qr)?/debug: ['trunk']
            linux.*64-shippable(-qr)?/opt: ['mozilla-central']
            windows10-64-shippable(-qr)?/opt: ['mozilla-central']
            default: []

the problem is if you added a line:
linux.*64(-qr)?/opt: ['mozilla-central']

this would run on opt, but since this isn't scheduled on central, we ignore it.

Blocks: 1693640

this opens up to a large volume of tests running in a taskgraph. doing ./mach taskgraph optimized --fast before vs after yields 553 raptor/browsertime jobs (doing ./mach taskgraph full --fast has 2509 extra jobs).

So while this is a quick win for try, I wonder what the right thing to do here is.

I think the goal is:
if editing mochitest.yml and added a line for fission-run-on-projects, then it will run on try and on the branch specified

:marco, do you have other ideas here?

Flags: needinfo?(mcastelluccio)
Fission Milestone: --- → M7a
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Component: General → Task Configuration
Product: Testing → Firefox Build System
Version: Default → unspecified

I think I have a handle on this. With some tweaks to browsertime/raptor/talos.yml we can get those tasks to disappear again. Though rather than remove the runs_on_central check I had to modify it to also look at fission-run-on-projects. That way it's possible to have fission tasks that don't exist in their non-fission versions.

That said, filtering by project is something we should be doing at the target_tasks phase, not the transform phase. Not sure if it's worth it to move it there.. if it is it should probably be its own separate bug.

Flags: needinfo?(mcastelluccio)

:ahal, thanks for picking this up. send any review my way, or I can pick up your findings and formalize into a patch.

(In reply to Andrew Halberstadt [:ahal] from comment #2)

I think I have a handle on this. With some tweaks to browsertime/raptor/talos.yml we can get those tasks to disappear again. Though rather than remove the runs_on_central check I had to modify it to also look at fission-run-on-projects. That way it's possible to have fission tasks that don't exist in their non-fission versions.

That said, filtering by project is something we should be doing at the target_tasks phase, not the transform phase. Not sure if it's worth it to move it there.. if it is it should probably be its own separate bug.

What if we removed fission-run-on-projects altogether? We already run tasks on opt by default on autoland, and shippable on m-c, removing fission-run-on-projects would simplify things.

Maybe we do simply want to remove the check. Those 2509 extra tasks will only show up in the full taskgraph and will be filtered out due to their fission-run-on-projects key.

On autoland with a backstop, the following extra tasks would run:

test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
test-linux1804-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-5
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-6
test-linux1804-64/opt-marionette-fis-e10s
test-linux1804-64/opt-marionette-headless-fis-e10s
test-windows10-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
test-windows10-64/opt-marionette-fis-e10s

On central, only the following extra tasks would run:

test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
test-linux1804-64/debug-mochitest-webgpu-fis-e10s

I could try to tweak the configs to maintain parity here if it would be a mistake to run any of these. The above is after fixing perf tasks to never run on non-shippable fission.

(In reply to Marco Castelluccio [:marco] from comment #4)

(In reply to Andrew Halberstadt [:ahal] from comment #2)

I think I have a handle on this. With some tweaks to browsertime/raptor/talos.yml we can get those tasks to disappear again. Though rather than remove the runs_on_central check I had to modify it to also look at fission-run-on-projects. That way it's possible to have fission tasks that don't exist in their non-fission versions.

That said, filtering by project is something we should be doing at the target_tasks phase, not the transform phase. Not sure if it's worth it to move it there.. if it is it should probably be its own separate bug.

What if we removed fission-run-on-projects altogether? We already run tasks on opt by default on autoland, and shippable on m-c, removing fission-run-on-projects would simplify things.

I would love to remove it.. though it initially got added because figuring out how to run fission things was too complex ><. Maybe now that fission is working pretty much everywhere, removing it would be the less complicated way to go.

There's also webrender-run-on-projects fyi :/

Either way, I don't think we should block this bug on it. I'll file a bug to investigate removing it.

Chris / James, my WIP patch would cause the tasks from comment 5 to start running on autoland / central (which are tangential to bug 1693640). Does that sound like a feature or a bug?

Flags: needinfo?(james)
Flags: needinfo?(cpeterson)

(In reply to Andrew Halberstadt [:ahal] from comment #7)

Chris / James, my WIP patch would cause the tasks from comment 5 to start running on autoland / central (which are tangential to bug 1693640). Does that sound like a feature or a bug?

If those test suites are all green with Fission, then I would be happy to have them enabled. :)

I have been filing bugs to run more test suites with Fission on more platforms. Test bugs relevant to the tasks in comment 5 that would be enabled:

Enable Fission web-platform-tests on more platforms: bug 1696042
Enable Fission Marionette tests on more platforms: bug 1694823

(In reply to Andrew Halberstadt [:ahal] from comment #5)

test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
...
test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
test-linux1804-64/debug-mochitest-webgpu-fis-e10s

@ Dzmitry: AFAICT, the only platform that currently runs webgpu tests on debug builds is test-windows10-64-mingwclang. linux1804-64, macosx1014-64, and windows10-64 only run webgpu tests on opt builds. Is that expected?

I don't mind enabling debug-mochitest-webgpu-fis-e10s for Fission, but I'd like to make sure there is no problem with debug webgpu.

Flags: needinfo?(cpeterson) → needinfo?(dmalyshau)

We can of course make the wpt tests green with suffcient metadata adjustment (assuming some baseline level of functionality). Any fission specific regressions can then be tracked by the fission team.

AIUI we originally didn't run reftests &c. in fission because it was assumed that most of the differences would be in DOM code. Now we're closer to shipping it makes sense to me to enable the additional tasks.

Flags: needinfo?(james)

Thanks, I'll do some testing on try.. if they look green I'll leave them, otherwise I'll figure out how to keep parity and we can enable them later in a separate bug.

Dzmitry: AFAICT, the only platform that currently runs webgpu tests on debug builds is test-windows10-64-mingwclang. linux1804-64, macosx1014-64, and windows10-64 only run webgpu tests on opt builds. Is that expected?

I'm trying to remember if there was a technical blocker for enabling more tests in debug configuration... in any way, this isn't necessary. If we can run on more configurations, we'd like to enable them in WebGPU.

Flags: needinfo?(dmalyshau)

Try run looks good:
https://treeherder.mozilla.org/jobs?repo=try&revision=909f989fe90f99a64fda66ff3026cf5c6e3ff8c5

I'll just leave them enabled rather than spend effort disabling them :). I'll also make a note in bug 1696042 and bug 1694823 once this lands.

All the other browsertime tasks ignore non-shippable platforms except for this
one. It was causing problems for a later patch in this stack.

These -profiling tasks are not currently running on fission. But the last patch in this stack
enables them as a side effect. This patch preserves the status quo.

Depends on D107107

This task gets enabled as a side effect of the last patch in this stack. This
patch preserves the status quo.

Depends on D107112

This check was likely added to try and limit the types of tasks that can be
created with fission. However it doesn't make sense to be filtering tasks based
on the project during the transforms stage. Tasks filtered out here don't exist
at all, so it's not possible to even schedule them on try with --full. This
type of filtering should be left to the target tasks stage of generation.

As a side effect, this patch enables the following tasks on autoland:

test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
test-linux1804-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-5
test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-6
test-linux1804-64/opt-marionette-fis-e10s
test-linux1804-64/opt-marionette-headless-fis-e10s
test-windows10-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
test-windows10-64/opt-marionette-fis-e10s

And the following tasks on central:

test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
test-linux1804-64/debug-mochitest-webgpu-fis-e10s

While this change would ideally happen in a separate commit, fission team
indicated it was desirable to enable these tasks anyway, so I decided not
to spend effort disabling them here, only to enable them again later.

Depends on D107113

Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ff6efc558fb1
[ci] Ensure browsertime tp6 ignores non-shippable fission platforms, r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/5acc76df94bf
[ci] Don't run talos profiling tasks with fission, r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/4fbac72c9da0
[ci] Don't run raptor-unity-webgl on fission, r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/376bbe652660
[taskgraph] Don't limit fission tasks to tasks that run on central, r=jmaher
You need to log in before you can comment on or make changes to this bug.