1983857 - newtab trainhop beta and release jobs should run on autoland and mozilla-central on Linux, macOS and Windows.

Status: RESOLVED FIXED

(Firefox :: New Tab Page, task)

Description

[Mike Conley (:mconley) (:⚙️)]

It looks like only beta is running on mozilla-central merges. We should try to get these to run on autoland (both release and beta) as well as central to catch newtab trainhop compatibility issues like bug 1983850.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Hey jmaher, do you know which bits I have to flip to make these jobs appear on both central and autoland, for Linux and macOS and Windows?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I see in https://searchfox.org/firefox-main/source/taskcluster/kinds/test/mochitest.yml#1293:

    run-on-projects:
        by-test-platform:
            linux(?!.*shippable).*: []
            macosx(?!.*shippable).*: []
            windows(?!.*shippable).*: []
            default: [trunk]

so this is run everywhere but linux/mac/win shippable.

we specify the trainhop tests on linu2404/win11/macosx1470 "opt":
https://searchfox.org/firefox-main/source/taskcluster/kinds/test/test-platforms.yml#80

this means it won't be considered on debug (or even shippable). But that platforms ^ file is referring to a list of "test sets", so looking at test-sets there is a trainhop definition, but a rough entry in linux-tests:
https://searchfox.org/firefox-main/source/taskcluster/kinds/test/test-sets.yml#184

that should probably be removed for consistency.

now to answer your question:

1. we don't do opt on central, so you need to scheduled on shippable, that should get you more coverage
2. not sure about autoland, the definition of [trunk] should work as that works for talos/browsertime.

if this doesn't work, ask again. I would look at the tgdiff task on a try push, this will generate artifacts like diff_mc-onpush.txt to see what is added or removed, so you can see for each branch (autoland).

Comment 3

[Mike Conley (:mconley) (:⚙️)]

Hm... so I've got this try push, and the artifacts for the tgdiff job seem to suggest that while this will add the jobs for autoland, it won't run them on central (which I'm also hoping for).

What am I missing?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

It might be as simple as "shippable" vs "opt". I see in test-platforms.yml, trainhop is on the list of tests for linux/win/mac "opt", but not "shippable":
https://searchfox.org/firefox-main/source/taskcluster/kinds/test/test-platforms.yml#367

try adding it to shippable as well :)

Comment 5

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1983857 - Make newtab train-hop jobs run on autoland and central on each platform. r?jmaher!

Comment 6

[Mike Conley (:mconley) (:⚙️)]

That seemed to work! Thanks, jmaher!

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a8a00054b11b
https://hg.mozilla.org/integration/autoland/rev/0a1bf2a34d1d
Make newtab train-hop jobs run on autoland and central on each platform. r=jmaher

Comment 8

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1983857 - Follow-up: make sure newtab train-hop jobs have access to all mochitest-browser test manifests. r?jmaher!

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/86721f1fb45e
https://hg.mozilla.org/integration/autoland/rev/7a576c7512aa
Follow-up: make sure newtab train-hop jobs have access to all mochitest-browser test manifests. r=jmaher

Comment 10

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/0a1bf2a34d1d
https://hg.mozilla.org/mozilla-central/rev/7a576c7512aa