1121003 - Add support for Talos {svgr,chromez,g1}-snow(-e10s)?

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P1)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

with the landing of new talos jobs that are osx 10.6 (snow) specific, we have a bunch of ? instead of a job name.

Comment 1

[Ed Morley [:emorley]]

Buildername/link to push/log please

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

buildername example: mozilla-central_snowleopard_test-g1-snow-e10s
log file: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-macosx64/1421155717/mozilla-central_snowleopard_test-g1-snow-e10s-bm108-tests1-macosx-build2.txt.gz

treeherder push: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=0500f1032ea1&filter-searchStr=talos

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Hidden for now. Basically, talos-other is the only OSX 10.6 talos job being properly matched. Though it's putting both the e10s and non-e10s versions in the T() group.

Comment 4

[Ed Morley [:emorley]]

Thank you for the example buildername and the link to a treeherder push showing the issue. There's several buildernames that need matching in fact, the full list is:

Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos svgr-snow
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos svgr-snow-e10s
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos chromez-snow
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos chromez-snow-e10s
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos g1-snow
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos g1-snow-e10s

The reason these show as "?" is that they don't match the existing regex:
    {"regex": re.compile('talos chromez-e10s$'), "desc": "Talos chrome e10s"},
    {"regex": re.compile('talos chrome[z]?$'), "desc": "Talos chrome"},
...
    {"regex": re.compile('talos g1-e10s$'), "desc": "Talos g1 e10s"},
    {"regex": re.compile('talos g1$'), "desc": "Talos g1"},

We need to add support for the new jobs.

Before we do that however - why is the job name different? Why can't we use the existing job name and differentiate based on OS?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

great question Ed!  We will run a different set of tests/job for osx 10.6 only.  

each of the talos jobs run a set of tests.  In cases where we don't want to run a test on a specific platform (could be it isn't useful there, doesn't work, etc.) then we need to modify the lists of tests in a specific job.

In this case, osx 10.6 provides little value except for graphics related tests.  In many of our jobs we have a set of tests which include graphics and other tests.  We don't want to run the other tests on 10.6, so the way to do that is change the configuration that provides the tests:
https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos.json

We can't modify an existing list of tests for a specific job without it affecting all platforms, so in this case we create a set of jobs that are specific to osx 10.6, and run them.  This leaves the other platforms alone and allows osx 10.6 jobs to only run what is useful.

all of the 10.6 related jobs have .*snow.* in the buildername, also support for -e10s is added as well for future proofing.

Comment 6

[Ed Morley [:emorley]]

Ah thank you for the context :-)

Comment 7

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/debe98d0da727a6ba979eebfa7e0499891653822
Bug 1121003 - Add support for 10.6-specific Talos tests