1689061 - enable test-verify ("TV") runs with Fission

Status: RESOLVED FIXED

(Testing :: General, task)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

With some tests running under Fission as tier 1, test-verify should also run for this configuration.

Comment 1

[Chris Peterson [:cpeterson]]

Hi Joel, who should I talk to about enabling more test suites on Treeherder for Fission? This bug in particular is about test-verify.

Tracking for Fission M7 because we'd like all tests to be running before our Fission Beta experiment.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

:cpeterson, I would be happy to discuss this and help make things happen. We should do a quick assessment of what is running and what will be running soon as tier-1, and more specifically on what platforms/configurations.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

from a quick look, I see mochitest/web-platform-tests as running as tier-1 for linux and windows10, so wpt needs some code added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/web-platform.yml#306

and mochitest needs it added here:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/misc.yml#44

what we would need to add is something similar to:

    variants:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: ['fission']
            windows10-64(-shippable)?-qr/opt: ['fission']
            default: []
    fission-tier:
        by-test-platform:
            linux.*64(-shippable)?-qr/opt: 1
            windows10-64(-shippable)?-qr/opt: 1
            default: 2

The above should simplify things- first I want to understand what configs we want to run on and if there are other suites coming online.

Comment 4

[Chris Peterson [:cpeterson]]

Our testing strategy and list of tests we'd like to enable on which platforms:

https://docs.google.com/document/d/19MergQfMob2mdb_s7hlz5Ilj7Bn-Bbi324r6d2bZfxw/edit

We want to run these tests on:

• debug and opt builds
• autoland only (because test-verify is only run on autoland)
• We prefer to run tests with Fission + WebRender (“qr”) where available.

Add platforms:

linux1804-64-qr/debug
linux1804-64-qr/opt
macosx1014-64-qr/debug
macosx1014-64-qr/opt
windows10-64-qr/debug
windows10-64-qr/opt

Comment 5

[Neha Kochar [:neha]]

Andrew, can we get these platforms enabled for TV runs, please?

Comment 6

[Andrew Halberstadt [:ahal]]

Sure!

Comment 7

[Andrew Halberstadt [:ahal]]

For whatever reason, test-verify only runs with -qr on macosx1014-64. So the fission test-verify jobs will be non-qr on Linux and Windows until we fix that.

Comment 8

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1689061 - [ci] Enable fission test-verify tasks on autoland, r?jmaher

This patch will enable the following tasks on autoland:

test-linux1804-64-qr/debug-test-verify-fis-e10s-1
test-linux1804-64-qr/debug-test-verify-fis-e10s-2
test-linux1804-64-qr/debug-test-verify-fis-e10s-3
test-linux1804-64-qr/opt-test-verify-fis-e10s-1
test-linux1804-64-qr/opt-test-verify-fis-e10s-2
test-linux1804-64-qr/opt-test-verify-fis-e10s-3
test-macosx1014-64-qr/debug-test-verify-fis-e10s-1
test-macosx1014-64-qr/debug-test-verify-fis-e10s-2
test-macosx1014-64-qr/debug-test-verify-fis-e10s-3
test-macosx1014-64-qr/opt-test-verify-fis-e10s-1
test-macosx1014-64-qr/opt-test-verify-fis-e10s-2
test-macosx1014-64-qr/opt-test-verify-fis-e10s-3
test-windows10-64-qr/debug-test-verify-fis-e10s-1
test-windows10-64-qr/debug-test-verify-fis-e10s-2
test-windows10-64-qr/debug-test-verify-fis-e10s-3
test-windows10-64-qr/opt-test-verify-fis-e10s-1
test-windows10-64-qr/opt-test-verify-fis-e10s-2
test-windows10-64-qr/opt-test-verify-fis-e10s-3

Comment 9

[Andrew Halberstadt [:ahal]]

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there. This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b74deabd2aef
[ci] Enable fission test-verify tasks on autoland, r=jmaher

Comment 11

[Chris Peterson [:cpeterson]]

(In reply to Andrew Halberstadt [:ahal] from comment #9)

As you can see from the context above, I went ahead and enabled TV-qr on Linux / Windows so we could run the -fis tasks there.

Thanks! That is good. TV with qr is what we want.

This patch only deals with test-verify though. Chris, do you also want test-verify-gpu and test-verify-wpt? If so, I'll file a new bug for them and add it to our backlog.

I don't think we need to bother with test-verify-gpu or test-verify-wpt for Fission.

Comment 12

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/b74deabd2aef