pushes to Try without changes don't run all the requested jobs (usually for measuring talos baseline performance)

RESOLVED WONTFIX

Status

task
RESOLVED WONTFIX
4 months ago
2 months ago

People

(Reporter: aryx, Unassigned)

Tracking

Trunk
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

in the above example there is a target-tasks.json which shows all the talos jobs:
https://taskcluster-artifacts.net/LGls6pg0Qe6kF7rQyRWR3w/0/public/target-tasks.json

between this and the raw log for the decision task, I do not see any evidence that we have problem during the decision task. I am curious what code would say "there are 0 lines of code changes, do not run anything"

Extracted From a discussion in #ci last week:

Standard8: So I’ve just triggered two sets of try builds with the same command… the first only has Mac builds, the second has everything expected… I’m confused
Standard8: https://treeherder.mozilla.org/#/jobs?repo=…32ca5fb4c0af6b717265cf6cf7c5abf4659
Standard8: https://treeherder.mozilla.org/#/jobs?repo=…e6b6a7f94ee915400a76688d684524e4ec9
Standard8: unless it was the completely empty base changeset? but that would be weird...
catlee: Callek, tomprince: any ideas? looks like the target tasks are correct at first glance
tomprince: catlee: Still looking, but first guess is fallout from 'optimize_target_tasks` change.
catlee: yeah, that's what I'm wondering
tomprince: catlee: treeherder.mozilla.org/logviewer.ht…=230795087&repo=try&lineNumber=3023 suggests that is the case.
Aryx: the one with more jobs only removes 43 tasks, the first one 45
Callek: That's my guess too
catlee: boo

Comment 3

4 months ago

(In reply to Mark Banner (:standard8) from comment #2)

tomprince: catlee: Still looking, but first guess is fallout from 'optimize_target_tasks` change.

This is going to mess with productivity for folks. Which change is that, and who could look at getting this fixed so explicitly requested talos jobs continue to run?

Flags: needinfo?(mozilla)

This was deliberate change, and I think the solution is going to be, use mach try fuzzy if you need more control over what runs.

:catlee is probably better to ask about this, as the change was made to reduce CI load.

Flags: needinfo?(mozilla) → needinfo?(catlee)

Comment 5

3 months ago

FWIW, it doesn't help that when you then "add new jobs" and/or retriggers to get the talos runs anyway, we end up triggering a build job for every talos job, cf. this push https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0daea63a35c52f99728bf5436eadee7c5a7967a .

Yeah, I would recommend mach try fuzzy to get exactly what you want.

I think maybe ./mach try fuzzy --artifact -q 'win64talos | linux64talos | mactalos' --rebuild 5 will reproduce what you wanted from the try push above?

Flags: needinfo?(catlee)

TBH I find fuzzy unintuitive and difficult to use, for starters something like the command in comment 6 would assume that I know exactly how the various tests are named.

My preferred way going forward is probably going to be mach try chooser.

I can see other people potentially being hit by this - either it should work in all modes, or there should be warnings where it doesn't.

Comment 8

3 months ago

(In reply to Chris AtLee [:catlee] from comment #6)

Yeah, I would recommend mach try fuzzy to get exactly what you want.

So does mach try chooser not work? Why is it different from fuzzy?

(In reply to :Gijs (he/him) from comment #8)

So does mach try chooser not work? Why is it different from fuzzy?

mach try chooser should behave identically to fuzzy as far as this bug is concerned. If anyone is seeing differently, please let me know.

Also worth noting that this bug will be automatically fixed by bug 1400295, though not sure what the timeline for that bug is (I'm hoping sometime in 2019).

See Also: → 1400295

I think this is something that isn't going to be fixed with try syntax.

Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.