1497155 - Allow toolchain artifact downloading command to be run on Taskcluster

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Marco Castelluccio [:marco]]

Currently we don't allow the command to be run on Taskcluster (https://dxr.mozilla.org/mozilla-central/rev/90853a59691e0c77014c38049bfb38b2b6ca1e16/python/mozbuild/mozbuild/mach_commands.py#1422), because "all dependencies should be determined in the decision task".
While this is true for mozilla-central, there are other projects (e.g. the static analysis bot) which use Taskcluster which would benefit from using the toolchain artifact downloading command.

Can we make this check specific to mozilla-central, or add an option to ignore it?

Comment 1

[Nathan Froyd [:froydnj]]

(In reply to Marco Castelluccio [:marco] from comment #0)
> Currently we don't allow the command to be run on Taskcluster
> (https://dxr.mozilla.org/mozilla-central/rev/
> 90853a59691e0c77014c38049bfb38b2b6ca1e16/python/mozbuild/mozbuild/
> mach_commands.py#1422), because "all dependencies should be determined in
> the decision task".
> While this is true for mozilla-central, there are other projects (e.g. the
> static analysis bot) which use Taskcluster which would benefit from using
> the toolchain artifact downloading command.


Why would you download the dependency via mach, when you can just declare it in the task definition?

Comment 2

[Marco Castelluccio [:marco]]

(In reply to Nathan Froyd [:froydnj] from comment #1)
> (In reply to Marco Castelluccio [:marco] from comment #0)
> > Currently we don't allow the command to be run on Taskcluster
> > (https://dxr.mozilla.org/mozilla-central/rev/
> > 90853a59691e0c77014c38049bfb38b2b6ca1e16/python/mozbuild/mozbuild/
> > mach_commands.py#1422), because "all dependencies should be determined in
> > the decision task".
> > While this is true for mozilla-central, there are other projects (e.g. the
> > static analysis bot) which use Taskcluster which would benefit from using
> > the toolchain artifact downloading command.
> 
> 
> Why would you download the dependency via mach, when you can just declare it
> in the task definition?

The static analysis bot doesn't use anything from the CI code in tree right now. It clones mozilla-central and uses mach.
It's not a task defined in tree, so it doesn't benefit from automatic downloading of dependency artifacts.

Comment 3

[Nathan Froyd [:froydnj]]

OK, thanks for the explanation.  If the bot is already running on Taskcluster, then why not make it fit better into the task framework?

Comment 4

[Marco Castelluccio [:marco]]

(In reply to Nathan Froyd [:froydnj] from comment #3)
> OK, thanks for the explanation.  If the bot is already running on
> Taskcluster, then why not make it fit better into the task framework?

We have plans for that, but unfortunately it will require some time to get it done.

Comment 5

[Andi [:andi]]

Would it be something feasible to add an extra flag that would force the manager to download the artifact even if it's automation env as checked here: https://dxr.mozilla.org/mozilla-central/rev/90853a59691e0c77014c38049bfb38b2b6ca1e16/python/mozbuild/mozbuild/mach_commands.py#1422

Comment 6

[Mike Hommey [:glandium]]

replacing TASK_ID with MOZ_AUTOMATION would seem reasonable.

Comment 7

[Andi [:andi]]

Attachment: Bug 1497155 - Allow toolchain artifact downloading command to be run on Taskcluster. r=glandium.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/42bd6d4455a4
Allow toolchain artifact downloading command to be run on Taskcluster. r=glandium.

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/42bd6d4455a4

Comment 10

[Gregory Szorc [:gps]]

I think the needinfo was answered in comment 6.