1274311 - Tier 2 TC tasks submitting to the same index key as tier 1 BB jobs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Rail Aliiev [:rail]]

This is the log of a buildbot linux64 opt build:

http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-linux64/1463665129/mozilla-inbound-linux64-bm72-build1-build1516.txt.gz

According to the log, it generates a TC task https://tools.taskcluster.net/task-inspector/#Cgszc2O4RNerKjJizDNUNA/ for artifacts and indexes itself as

index.gecko.v2.mozilla-inbound.revision.2fb9523055a61f02804393dc6f479bfd0427a1a8.firefox.linux64-opt


At the same time we have a tier 2 build in TC:
https://tools.taskcluster.net/task-inspector/#Na7XoA9PQiaK8dO-e1E52w/

It also indexes itself using the same index.

ATM, the index points to the BB job:
https://tools.taskcluster.net/index/#gecko.v2.mozilla-inbound.revision.2fb9523055a61f02804393dc6f479bfd0427a1a8.firefox.linux64-opt/gecko.v2.mozilla-inbound.revision.2fb9523055a61f02804393dc6f479bfd0427a1a8.firefox.linux64-opt

but it's not impossible to get it point to the tier 2 task, what would be a blocker for release promotion, which uses the index service to find the corresponding tasks and artifacts.

Maybe we should add another route to make sure we have a way to distinguish these tasks?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Given that BB jobs are going away we should maybe add another key or suffix for it like:

gecko.v2.mozilla-inbound.revision.2fb9523055a61f02804393dc6f479bfd0427a1a8.firefox.linux64-opt.buildbot

Comment 3

[Michael Shal [:mshal]]

I don't think we want to use a separate route with buildbot in the name - for some more details, see also this discussion here: https://bugzilla.mozilla.org/show_bug.cgi?id=1277666#c6

Though we do still need to address tier 2 builds. I think they should probably just have a rank of 0, so while the Taskcluster build is being worked on, it won't impact the route that points to the buildbot task. After Taskcluster is promoted to tier 1, we can shut off the equivalent buildbot build and not impact any users of the index.

Comment 4

[Rail Aliiev [:rail]]

FTR, buildbot sets the rank to the push date: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/building/buildbase.py#1392

Comment 5

[Rail Aliiev [:rail]]

The same for TC:

https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/legacy/tasks/build.yml#39
https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/kind/legacy.py#326,330,354

Comment 6

[Michael Shal [:mshal]]

TC was recently updated to use the pushdate in bug 1277666 - previously it was the pushlog id.

I think I have a patch that will just set the rank to 0 for tier > 1, so if we have both a tier 1 and tier 2 job, only the tier 1 job will show up in the index. Tier 2+ jobs will still show up in the index if there is no tier 1 equivalent, though.

Comment 7

[Rail Aliiev [:rail]]

(In reply to Michael Shal [:mshal] from comment #6)
> I think I have a patch

Whee! :)

Thanks for the explanation. If the rank solves the issue, it totally WFM!

Comment 8

[Michael Shal [:mshal]]

Attachment: Bug 1274311 - tier-2 jobs should have rank 0 in the index;

Review commit: https://reviewboard.mozilla.org/r/61052/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/61052/

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8765941 [details]
Bug 1274311 - tier-2 jobs should have rank 0 in the index;

https://reviewboard.mozilla.org/r/61052/#review57914

::: taskcluster/taskgraph/kind/legacy.py:459
(Diff revision 1)
> +            if 'treeherder' in task_extra:
> +                tier = task_extra['treeherder'].get('tier', 1)
> +                if tier != 1:
> +                    # Only tier 1 jobs use the build time as rank. Everything
> +                    # else gets rank 0 until it is promoted to tier 1.
> +                    task_extra['index']['rank'] = 0

I like this!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/47c2b7937aa7
tier-2 jobs should have rank 0 in the index; r=dustin

Comment 11

[Rail Aliiev [:rail]]

Woot!

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/47c2b7937aa7