1325398 - Make the check for duplicate Treeherder symbols, happen during `full` graph not `target` graph stage.

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Justin Wood (:Callek)]

Bug #1323633 has bitten me a few times on date, I always run `./mach taskgraph full ...` since the verification steps have been added.

And in this case, a treeherder duplicate symbol, I don't see why only validating with a target set is ideal/wanted, when the same push can have multiple target sets run on it. -- We should just validate with the whole graph.

Comment 1

[Justin Wood (:Callek)]

Attachment: Bug 1325398 - Make the check for duplicate Treeherder symbols, happen during `full` graph not `target` graph stage.

Review commit: https://reviewboard.mozilla.org/r/100546/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/100546/

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8821207 [details]
Bug 1325398 - Make the check for duplicate Treeherder symbols, happen during `full` graph not `target` graph stage.

https://reviewboard.mozilla.org/r/100546/#review101086

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Hammad, FYI -- let's do these checks earlier in the task-graph generation process so we catch errors sooner.

Comment 4

[HAMMAD AKHTAR]

When transitive closure for full task graph is computed some more dependent tasks are added so I thought maybe I should do the check after obtaining a target task graph.

Comment 5

[Justin Wood (:Callek)]

(In reply to HAMMAD AKHTAR from comment #4)
> When transitive closure for full task graph is computed some more dependent
> tasks are added so I thought maybe I should do the check after obtaining a
> target task graph.

My understanding:

* FULL
  - set of all tasks that can possibly run (scheduled by taskcluster) on this push
  - computed first
* TARGET
   - Tasks that should run on this push
   - First by selecting all tasks that would match the 'target' conditions
   - Then by adding in tasks that are dependancies of those selected ones
* OPTIMIZED
   - Tasks that *will* run on this push
   - ...

Comment 6

[HAMMAD AKHTAR]

So basically all possible tasks are loaded resulting in full task graph,
then we identify target tasks and determine there dependencies resulting in target task graph.

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Right -- so if there are duplicates in target or optimized, then those duplicates will also exist in the full graph.

Comment 8

[HAMMAD AKHTAR]

ahh right, checking after generating full task graph will catch errors early :)

Comment 9

[Justin Wood (:Callek)]

Comment on attachment 8821207 [details]
Bug 1325398 - Make the check for duplicate Treeherder symbols, happen during `full` graph not `target` graph stage.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/100546/diff/1-2/

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/614959839ad7
Make the check for duplicate Treeherder symbols, happen during `full` graph not `target` graph stage. r=dustin

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/614959839ad7