Closed Bug 1689061 Opened 3 months ago Closed 1 month ago

enable test-verify ("TV") runs with Fission

Categories

(Testing :: General, task)

task

Tracking

(Fission Milestone:M7, firefox88 fixed)

RESOLVED FIXED
88 Branch
Fission Milestone M7
Tracking Status
firefox88 --- fixed

People

(Reporter: aryx, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

With some tests running under Fission as tier 1, test-verify should also run for this configuration.

Hi Joel, who should I talk to about enabling more test suites on Treeherder for Fission? This bug in particular is about test-verify.

Tracking for Fission M7 because we'd like all tests to be running before our Fission Beta experiment.

Fission Milestone: --- → M7
Flags: needinfo?(jmaher)

:cpeterson, I would be happy to discuss this and help make things happen. We should do a quick assessment of what is running and what will be running soon as tier-1, and more specifically on what platforms/configurations.

Flags: needinfo?(jmaher)

from a quick look, I see mochitest/web-platform-tests as running as tier-1 for linux and windows10, so wpt needs some code added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/web-platform.yml#306

and mochitest needs it added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/misc.yml#44

what we would need to add is something similar to:

    variants:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: ['fission']
            windows10-64(-shippable)?-qr/opt: ['fission']
            default: []
    fission-tier:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: 1
            windows10-64(-shippable)?-qr/opt: 1
            default: 2

The above should simplify things- first I want to understand what configs we want to run on and if there are other suites coming online.

Blocks: fission-tests
No longer blocks: fission-mochitests

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

  • debug and opt builds
  • autoland only (because test-verify is only run on autoland)
  • We prefer to run tests with Fission + WebRender (“qr”) where available.

Add platforms:

linux1804-64-qr/debug
linux1804-64-qr/opt
macosx1014-64-qr/debug
macosx1014-64-qr/opt
windows10-64-qr/debug
windows10-64-qr/opt

Andrew, can we get these platforms enabled for TV runs, please?

Flags: needinfo?(ahal)

Sure!

Assignee: nobody → ahal
Status: NEW → ASSIGNED
Flags: needinfo?(ahal)

For whatever reason, test-verify only runs with -qr on macosx1014-64. So the fission test-verify jobs will be non-qr on Linux and Windows until we fix that.

This patch will enable the following tasks on autoland:

test-linux1804-64-qr/debug-test-verify-fis-e10s-1
test-linux1804-64-qr/debug-test-verify-fis-e10s-2
test-linux1804-64-qr/debug-test-verify-fis-e10s-3
test-linux1804-64-qr/opt-test-verify-fis-e10s-1
test-linux1804-64-qr/opt-test-verify-fis-e10s-2
test-linux1804-64-qr/opt-test-verify-fis-e10s-3
test-macosx1014-64-qr/debug-test-verify-fis-e10s-1
test-macosx1014-64-qr/debug-test-verify-fis-e10s-2
test-macosx1014-64-qr/debug-test-verify-fis-e10s-3
test-macosx1014-64-qr/opt-test-verify-fis-e10s-1
test-macosx1014-64-qr/opt-test-verify-fis-e10s-2
test-macosx1014-64-qr/opt-test-verify-fis-e10s-3
test-windows10-64-qr/debug-test-verify-fis-e10s-1
test-windows10-64-qr/debug-test-verify-fis-e10s-2
test-windows10-64-qr/debug-test-verify-fis-e10s-3
test-windows10-64-qr/opt-test-verify-fis-e10s-1
test-windows10-64-qr/opt-test-verify-fis-e10s-2
test-windows10-64-qr/opt-test-verify-fis-e10s-3

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there. This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

Flags: needinfo?(cpeterson)
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b74deabd2aef
[ci] Enable fission test-verify tasks on autoland, r=jmaher

(In reply to Andrew Halberstadt [:ahal] from comment #9)

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there.

Thanks! That is good. TV with qr is what we want.

This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

I don't think we need to bother with test-verify-gpu or test-verify-wpt for Fission.

Flags: needinfo?(cpeterson)
Status: ASSIGNED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → 88 Branch
Regressions: 1698849
You need to log in before you can comment on or make changes to this bug.