1508382 - Setting `sparse-profile: None` on a toolchain task produces an error

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Kartikaya Gupta (email:[e-mail])]

According to [1] a toolchain task should be able to set `sparse-profile: None` to not use a sparse profile. However when actually trying to do so, it produces an error:

[vcs 2018-11-19T20:14:39.356Z] abort: sparse profile build/sparse-profiles/None does not exist at revision 28421835d90c40a8af63ff8d01591e8110f07bac

[1] https://searchfox.org/mozilla-central/rev/b03a62c3c82316e733a3b09622c1cb7e59f64cc3/taskcluster/taskgraph/transforms/job/toolchain.py#54

Comment 1

[Kartikaya Gupta (email:[e-mail])]

Also note the only toolchain that tries to take advantage of this actually sets it to `null` instead of `None`:

https://searchfox.org/mozilla-central/rev/b03a62c3c82316e733a3b09622c1cb7e59f64cc3/taskcluster/ci/toolchain/linux.yml#473

Comment 2

[Gregory Szorc [:gps]]

That abort message implies that the string "None" is making its way through the config. I'm guessing YAML doesn't recognize `None` as an alias for "undefined" and that `null` is needed instead?

A quick fix for this would be for taskgraph to verify the sparse profile file as specified in the config exists in the current revision.

Comment 3

[Tom Prince [:tomprince]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #0)
> According to [1] a toolchain task should be able to set `sparse-profile:
> None` to not use a sparse profile.

This is because yaml and python spell things differently as :gps suggested. YAML spells `None` as `null`. This could be better documentated, but is working as expected.

(In reply to Gregory Szorc [:gps] from comment #2)
> A quick fix for this would be for taskgraph to verify the sparse profile
> file as specified in the config exists in the current revision.

This would definitely be useful.

Comment 4

[Tom Prince [:tomprince]]

Attachment: Bug 1508382: [taskgraph] Verify that the specifed sparse profile exists; r?gps

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a9b90a979fc7
[taskgraph] Verify that the specifed sparse profile exists; r=gps

Comment 6

[Dorel Luca [:dluca]]

Backed out changeset a9b90a979fc7 (bug 1508382) for Linting failure. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer.html#?job_id=213896994&repo=autoland&lineNumber=303

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=a9b90a979fc70966dde072d66d97bde8e69066bb

Backout:
https://hg.mozilla.org/integration/autoland/rev/ea9af5fdb20e64d55b0ff4ff748ca35f949f0353

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6417da60696e
[taskgraph] Verify that the specifed sparse profile exists; r=gps

Comment 8

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/6417da60696e

Comment 9

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/2bb6c554d574