1255658 - Consider silent "No tests run" for indivisual job when --tag is specified in try syntax

Status: RESOLVED DUPLICATE of bug 1178441

(Testing :: General, defect)

Description

[Xidorn Quan [:xidorn] UTC+10]

When using tags, we could restrict the tests we run for a try push. Tests with the specified tags could be in several different chunks which is hard to judge beforehand. Since empty jobs are cheap (seems to only take ~2mins to finish), I think triggering all jobs is a reasonable way to cover all tests with given tags. However, that leads to an issue that many of the jobs may end up bustage with "No tests run or test summary not found", and people may need to check all of them to ensure nothing unexpected happens, which could be annoying.

I suggest that we suppress this specific bustage when --tag is present in the try syntax. Probably give those jobs a mild color other than green, and not count them in the title counter.

Comment 1

[Xidorn Quan [:xidorn] UTC+10]

Actually a duplicate... but in different product

Comment 2

[Ed Morley [:emorley]]

Treeherder uses the results verbatim from whatever system generated the jobs. It's really up to that system to set the appropriate job result (or ideally prevent the empty errored jobs in the first place) IMO :-)