Closed Bug 1380391 Opened 2 years ago Closed 2 years ago

index decision tasks under pushlog_id

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla56

People

(Reporter: bstack, Assigned: bstack)

References

Details

Attachments

(1 file)

In [0] it is recommended that we use pushlog_id in automation to deal with pushes. I'm inclined to agree and am feeling like any other option doesn't really make sense. With that in mind, I think a lot of the issues we have with backfilling come from the crazy hacks I've made to bridge the divide of decision tasks and what treeherder thinks is a push.

* I propose we just add a new index namespace under gecko.v2 that is gecko.v2.{{project}}.pushlog-id.{{pushlog_id}}. This seems generically useful and will make backfilling as easy as querying the pushlog for the N previous pushes, and going directly to the index to find the associated decisions tasks.

* This makes backfilling a process that is entirely defined in taskcluster + hg terms which is what we already have in mozilla-taskcluster and avoids even thinking about treeherder, which really should remain mostly a ui in front of all of this stuff.

* I've heard rumblings of us having multiple decision tasks per push, but that seems like something we should think harder about before we do it. It will make backfilling an even less well-defined concept. Perhaps we allow multiple decision tasks, but say that backfilling just won't work with things other than the "normal" decision task and its graph?


[0] http://mozilla-version-control-tools.readthedocs.io/en/latest/hgmo/pushlog.html?highlight=pushlog#writing-agents-that-consume-pushlog-data
Thankfully in [0] the authors of cron.yml stuff have made it so that routes don't collide for decision tasks based on the main decision task. After talking to :dustin, we think our best path forward will be to call the new index gecko.v2.{{project}}.pushlog-id.{{pushlog_id}}.decision so that people can index other things under pushlog_id if they desire later. Rather than calling the one I'm adding decision.ci or something to distinguish it from the other types of decision tasks, the logic in [0] handles the differentiation for us.

[0] https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/cron/decision.py#69-71
Attachment #8885822 - Flags: review?(dustin)
Comment on attachment 8885822 [details]
Bug 1380391 - Index decision tasks under pushlog_id

https://reviewboard.mozilla.org/r/156596/#review161764
Attachment #8885822 - Flags: review?(dustin) → review+
Status: NEW → ASSIGNED
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/365731510976
Index decision tasks under pushlog_id r=dustin
https://hg.mozilla.org/mozilla-central/rev/365731510976
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Blocks: 1380885
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.