1996563 - Generate perfherder_data artifacts for jobs running fetch-content

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[Myeongjun Go]

When fetch-content runs, we expect it to produce a perfherder-data-fetch-content.json artifact [0]. However, the file is only created when UPLOAD_DIR is defined.
Currently, UPLOAD_DIR is automatically set for tasks defined under the fetch kind [1].
But when a task in another kind uses fetches: (e.g. source-test jobs)[2], fetch-content still runs, but UPLOAD_DIR is not set, so the perfherder-data-fetch-content.json file is not created.
This happens because those tasks are not defined under fetch/kind.yml[1], where UPLOAD_DIR is configured.
So, any job[2] that uses fetch-content outside of the fetch kind currently does not produce the perfherder JSON file.
We should ensure that any job that runs fetch-content also has UPLOAD_DIR defined, so the perfherder JSON artifact is always generated.

[0]
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/third_party/python/taskcluster_taskgraph/taskgraph/run-task/fetch-content#948
[1]
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/kinds/fetch/kind.yml#8,13-24
https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/transforms/fetch.py#105-106,118
[2]
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/kinds/source-test/jsshell.yml#48
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/gecko_taskgraph/transforms/job/run_task.py#185
[3]
https://treeherder.mozilla.org/jobs?repo=try&revision=336248ef294474cd4c82279be4a87cb48f70045f&selectedTaskRun=IQTIdRkrQDyA6-C8Wm_CGw.0
https://treeherder.mozilla.org/jobs?repo=try&revision=336248ef294474cd4c82279be4a87cb48f70045f&selectedTaskRun=ORkuAbC1TTi1DCPd_ydPCA.0

Comment 1

[Myeongjun Go]

Issue:
Some jobs produce the perfherder-data-fetch-content.json artifact as expected [0], while others do not [1]
[0]
https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=HVHECVi0T0yu5DtOVClv_Q.0
https://firefoxci.taskcluster-artifacts.net/HVHECVi0T0yu5DtOVClv_Q/0/public/build/perfherder-data-fetch-content.json
[1]
https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=ejYaP9GSR7ikVwHmZiE6Ng.0

---

Hi ahal :)
I’m currently looking into an issue where fetch-content[0] runs in jobs outside of the fetch kind[1] (e.g. some source-test tasks), but UPLOAD_DIR is not set in those cases, so the perfherder-data-fetch-content.json artifact doesn’t get generated.
In the fetch kind [1], UPLOAD_DIR is explicitly set through the transform (fetch.py[2]), but when fetch-content is invoked via the fetches: field[3] in other kinds, that environment setup path isn't taken. So fetch-content[0] runs correctly, but the perfherder JSON artifact is missing.

I'm not sure where the best place to fix this is.
Do you think it would make sense to set UPLOAD_DIR for jobs using fetches: in the run-task transform, or should this be handled somewhere else in the pipeline?
Any guidance or pointers would be greatly appreciated :)
Thanks.

[0] fetch-content call site
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/gecko_taskgraph/transforms/job/run_task.py#185
[1]
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/kinds/fetch/kind.yml#8,13-24
[2]
https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/transforms/fetch.py#105-106,118
[3] Example of fetches: usage in another kind
https://searchfox.org/firefox-main/rev/643d732886fe0de4e2a3eee3c5ed9bd0d47c77cf/taskcluster/kinds/source-test/jsshell.yml#48

Comment 2

[Andrew Halberstadt [:ahal]]

Hm, yeah this is a larger issue around standardizing artifacts across all tasks or not. There are tons of tasks that use fetch-content, probably even most tasks :). I don't think it's desirable for every single one of them to set UPLOAD_DIR.

First I'd like to step back a bit and understand what it is you're trying to accomplish overall. Do we need to profile fetch-content in every task? If so, why and who's asking for this?

If we're sure this is something we want / need to do, then I think the way to go is to stop relying on UPLOAD_DIR being set in fetch-content. Instead, we should set up "file" artifact directly pointing to perfherder-data-fetch-content.json in the task's workdir. All tasks that use fetch-content, will run through this transform:
https://searchfox.org/firefox-main/rev/da0861d6f7f7280518d2a4511d07401aecf50afa/taskcluster/gecko_taskgraph/transforms/job/__init__.py#253

So that might be a good place to handle this. We would need to both create the artifact blob in the task definition, and then set an environment variable (e.g PERFHERDER_ARTIFACT_PATH or similar), then update fetch-content to write the artifact there instead of UPLOAD_DIR.

Comment 3

[Greg Mierzwinski [:sparky]]

(In reply to Andrew Halberstadt [:ahal] from comment #2)

First I'd like to step back a bit and understand what it is you're trying to accomplish overall. Do we need to profile fetch-content in every task? If so, why and who's asking for this?

There was a brief discussion about it, and it seemed like people still wanted it: https://bugzilla.mozilla.org/show_bug.cgi?id=1986824#c6

I needinfo'ed glandium about it and he still wanted to keep it. We briefly spoke about it and you mentioned you'd ask your team - have you had a chance to discuss this with them? Regarding profiling every single task or not, I think that's something that whoever owns these metrics would need to decide.

Comment 4

[Myeongjun Go]

Attachment: Bug 1996563 - Generate perfherder_data artifacts for jobs running fetch-content r=sparky

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/2a94b999bdbe
https://hg.mozilla.org/integration/autoland/rev/8d633793271a
Generate perfherder_data artifacts for jobs running fetch-content r=sparky,taskgraph-reviewers,ahal

Comment 6

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/8d633793271a