Closed Bug 1296842 Opened 3 years ago Closed 3 years ago

unable to find parameters for ./mach taskgraph target, docs are not effective

Categories

(Taskcluster :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla52

People

(Reporter: jmaher, Assigned: dustin)

Details

Attachments

(4 files)

jmaher@jmaher-ThinkPad-X230:~/mozilla/mozilla-inbound$ ./mach taskgraph target
usage: mach [global arguments] taskgraph target [command arguments]
mach: error: argument --parameters/-p is required


so what is -p?  previously I would look at a try push and find a parameters.yml file, download it, then use it.

I looked at a few try pushes, inspected the tasks and cannot find parameters.yml or anything that resembles that.

according to the source:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/mach_commands.py#46

I need to look here:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/docs/parameters.rst

I try to create a parameters.yml file:
cat ~/parameters.yml 
base_repository: try
head_repository: try
head_rev: 2b150d872bbb
head_ref: 2b150d872bbb
owner: jmaher@mozilla.com
message: -b do -p linux -u crashtest -t none


I didn't know what pushlog_id was, and pushdate in unix seconds (make this easy if we are hand editing), and tree level I have never heard of before.

when I run taskgraph target with the above parameters.yml, I get:
./mach taskgraph target -p ~/parameters.yml
Traceback (most recent call last):
  File "/home/jmaher/mozilla/mozilla-inbound/taskcluster/mach_commands.py", line 216, in show_taskgraph
    target_tasks_method = taskgraph.target_tasks.get_method(target_tasks_method)
  File "/home/jmaher/mozilla/mozilla-inbound/taskcluster/taskgraph/target_tasks.py", line 29, in get_method
    return _target_task_methods[method]
KeyError: u'all_tasks'


I am asking:
1) that the docs discoverable in tree are useful
2) that mach has better help
3) that we can download an example parameters.yml file
4) fixing up the error handling when people try to use the mach commands would be nice.
From an irc chat and the above, the fixes here are:

 - include some how-to on finding a parameters file (in this case, the missing piece was that you have to look at the decision task)
 - link to the docs from the error message (not sure this is possible/practical with mach..)
 - link to https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/ for SCM Level
 - use sensible defaults for pushlog_id and pushdate (with suitable warnings)
 - set the default for target_tasks_method to `all_builds_and_tests`
Assignee: nobody → dustin
>  - include some how-to on finding a parameters file (in this case, the
> missing piece was that you have to look at the decision task)

http://gecko.readthedocs.io/en/latest/taskcluster/taskcluster/how-tos.html#hacking-task-graphs
> Find a recent decision task on the project or branch you are working on, and download its parameters.yml from the Task Inspector. This file contains all of the inputs to the task-graph generation process. Its contents are simple enough if you would like to modify it, and it is documented in Parameters.

>  - link to the docs from the error message (not sure this is
> possible/practical with mach..)
>  - link to
> https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/
> for SCM Level

upcoming patch

>  - use sensible defaults for pushlog_id and pushdate (with suitable warnings)

I actually don't want to make defaults here -- we should expect a complete parameters file.  Still, we can check that the file is complete.  There are defaults applied during decision task execution, but that's different (that's how parameters.yml is generated).

>  - set the default for target_tasks_method to `all_builds_and_tests`

Same.  But it definitely makes sense to raise a useful exception if no method is specified, or if the value is invalid.
Comment on attachment 8808307 [details]
Bug 1296842: better parameter documentation to help readers;

https://reviewboard.mozilla.org/r/91140/#review90918

::: taskcluster/docs/parameters.rst:15
(Diff revision 1)
>  a full parameters file as one of its output artifacts.  The other ``mach
>  taskgraph`` commands can take this file as input.  This can be very helpful
>  when working on a change to the task graph.
>  
> +When experimenting with local runs of the task-graph generation, it is always
> +best to find a recent decision task's ``parameters.yml`` file, and modify that

I still think we need something here to indicate we need a decision task from a push to a tree.  This isn't that clear what is a "recent decision task"
Attachment #8808307 - Flags: review?(jmaher) → review-
Comment on attachment 8808308 [details]
Bug 1296842: document `triggered_by` property;

https://reviewboard.mozilla.org/r/91142/#review90920
Attachment #8808308 - Flags: review?(jmaher) → review+
Comment on attachment 8808309 [details]
Bug 1296842: make target-task related parameters a fixed set;

https://reviewboard.mozilla.org/r/91144/#review90922
Attachment #8808309 - Flags: review?(jmaher) → review+
Comment on attachment 8808310 [details]
Bug 1296842: check parameters;

https://reviewboard.mozilla.org/r/91146/#review90924
Attachment #8808310 - Flags: review?(jmaher) → review+
Comment on attachment 8808307 [details]
Bug 1296842: better parameter documentation to help readers;

https://reviewboard.mozilla.org/r/91140/#review90932

::: taskcluster/docs/parameters.rst:15
(Diff revision 1)
>  a full parameters file as one of its output artifacts.  The other ``mach
>  taskgraph`` commands can take this file as input.  This can be very helpful
>  when working on a change to the task graph.
>  
> +When experimenting with local runs of the task-graph generation, it is always
> +best to find a recent decision task's ``parameters.yml`` file, and modify that

What other decision task could it mean?
I had a lot of confusion here knowing that I needed to get a decision task from a push to find the parameters.yml file.  It isn't obvious while hacking in a cmd shell using mach to look at a push that happened.
Right, I just don't understand how more words will help.  Also, see comment 2.
Comment on attachment 8808307 [details]
Bug 1296842: better parameter documentation to help readers;

https://reviewboard.mozilla.org/r/91140/#review90944

ok, after revisiting this- I am not seeing any way to add more text to make this understandable- I think we have cleaned this up a good bit!
Attachment #8808307 - Flags: review- → review+
Cool, thank you!
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4a1847c6164f
better parameter documentation to help readers; r=jmaher
https://hg.mozilla.org/integration/autoland/rev/8d60c718ff15
document `triggered_by` property; r=jmaher
https://hg.mozilla.org/integration/autoland/rev/f29234ab0d3c
make target-task related parameters a fixed set; r=jmaher
https://hg.mozilla.org/integration/autoland/rev/126467fa0641
check parameters; r=jmaher
This caused bug 1316961.
You need to log in before you can comment on or make changes to this bug.