1101548 - [trychooser] Expose a simple API for retrieving the current try syntax

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

Trychooser and the hg try extensions are constantly getting out of date. It would be awesome if there were an api so tools like those could automatically update themselves.

I think such an api could return a json object:

{
    'buildtypes': {
        'opt': 'o',
        'debug': 'd'
    },
    'platforms': {
        'linux': 'linux',
        'b2g desktop': 'linux32_gecko',
        ...
    },
    ...
}

I guess it would get more complicated when you consider special cases like non-default builds, restricted tests and android/b2g only tests.

If an API is too hard, then a library that can extract this information out of configuration somehow would be the next best thing.

Comment 1

[Steve Fink [:sfink] [:s:]]

Attachment: builds available via trychooser

Note that my trychooser extension uses a jobs.json file that describes the available options.

It is... a little weird. I was trying to optimize for human editing and reading, and somewhat also because I had some notion that part of it would be autogenerated (as per this bug) and the rest would be decorations (eg help text). So it's structured as a couple of "parallel" structures that I traverse together. It would be much simpler to put everything in one structure, though that means that it would need to have default-ness, help text, trychooser job grouping info (eg "mochitests" is equivalent to this set of tests), and possibly an initial/default selection of jobs for try pushes.

Attaching the current jobs.json file for inspiration (or maybe as a warning?).

Comment 2

[Armen [:armenzg]]

I have filed this:
https://github.com/armenzg/mozilla_ci_tools/issues/69

Hopefully someone picks it up.

Comment 3

[Andrew Halberstadt [:ahal]]

Is this basically finished in mozci now? If so should this be bug be closed?

Comment 4

[Armen [:armenzg]]

I belive so:
http://people.mozilla.org/~armenzg/permanent/graph.json

We just need someone to try to use this data and see what is lacking.

This is the script that generates that data:
https://github.com/armenzg/mozilla_ci_tools/blob/master/scripts/misc/write_tests_per_platform_graph.py

I also filed this:
https://bitbucket.org/sfink/trychooser/issue/8/use-mozci-to-help-build-an-up-to-date

Comment 5

[Armen [:armenzg]]

To summarize, we can generate an up-to-date structure (graph.json), however, we need the trychooser [1] use it and see what is lacking.

In a sense we should work on bug 983802 and use this bug for managing to generate the fully functional piece of data. It can either be graphs.json or something else.

Options like which platforms get triggered by default don't show up on allthethings.json [3] which graphs.json uses to be generated. Adding that information is filed as bug 1110847.
allthethings.json can be generate with this script: generate_allthethings_json.sh [4]
graphs.json can be generated with this script: write_tests_per_platform_graph.py [5]

Another note is that graphs.json does not include any information about b2g build jobs that currently run on TaskCluster (since mozci does not yet support it).
By the end of June we should be close to fix this issue.

However, I believe we should get a prototype working without the information of bug 1110847 and hardcoding b2g so we can at least somewhere.

[1] http://trychooser.pub.build.mozilla.org
[2] http://hg.mozilla.org/build/tools/file/default/trychooser
[3] https://secure.pub.build.mozilla.org/builddata/reports/allthethings.json
[4] https://hg.mozilla.org/build/braindump/file/09c16abb129d/community/generate_allthethings_json.sh
[5] https://github.com/armenzg/mozilla_ci_tools/blob/master/mozci/scripts/misc/write_tests_per_platform_graph.py

Comment 6

[dihaynes77]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #5)
Hi - I'm looking at this bug for the DX/Ateam Summer of Contribution project, and I want to make sure I understand the next deliverable. Are you looking for someone to make trychooser a front end (html, jquery, etc.) for [5] or am I oversimplifying this?

Comment 7

[Armen [:armenzg]]

Thanks for helping us!

I will set a flag to help me review this tomorrow. I have some new ideas I will explain tomorrow (I'm out for the day).

Comment 8

[dihaynes77]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #7)
Thanks! I'm looking forward to learning more.

Comment 9

[Armen [:armenzg]]

Hi dihayness77,
The front end that keeps on falling out of date is this:
http://trychooser.pub.build.mozilla.org
The code lives in here:
http://hg.mozilla.org/build/tools/file/default/trychooser

There are now two different systems that listen to the commit message that developers add when pushing to the try repository:
https://hg.mozilla.org/try

Developers need to pick a try message to trigger different set of platforms and test jobs.
Unfortunately, the UI does not show them the most accurate options.
You can see in here that each push to the try repository gets different set of build/test jobs depending on their try message:
https://treeherder.mozilla.org/ui/#/jobs?repo=try

To build an up-to-date UI, you will the data for you to use and display.

This bug was originally filed to create an API that would return the most up-to-date information.

Were you looking to build the UI?
Or were you looking to generate the data?

In order to generate the data, I suggest you to work on the script write_tests_per_platform_graph.py.
That will only generate 80% of the data since FirefoxOS/B2G builds and some tests would be generated differently.

For the B2G jobs, you will have to checkout mozilla-central:
http://hg.mozilla.org/mozilla-central
You can then grab the patch in here:
https://reviewboard.mozilla.org/r/11423/diff/1#index_header
You would have to modify that to generate the right data you want.
Read bug 1174313 to understand a bit more.

Generating the data structure for the B2G jobs is going to be easier.

If you were to work on the UI, I would say to work on generating a sample data structure and build a better UI.
Once we were to have the auto-generated data in this bug we could plug it into your UI.
I would suggest using bug 983802 to work on the UI and keep this bug in order to generate the data.

Comment 10

[dihaynes77]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #9

> Were you looking to build the UI?
> Or were you looking to generate the data?

Your explanation helped narrow the scope (re: where to start) so thanks for that. I would rather help generate the data.

Thank you for looking into this. 


> In order to generate the data, I suggest you to work on the script
> write_tests_per_platform_graph.py.
> That will only generate 80% of the data since FirefoxOS/B2G builds and some
> tests would be generated differently.
> 
> For the B2G jobs, you will have to checkout mozilla-central:
> http://hg.mozilla.org/mozilla-central
> You can then grab the patch in here:
> https://reviewboard.mozilla.org/r/11423/diff/1#index_header
> You would have to modify that to generate the right data you want.
> Read bug 1174313 to understand a bit more.
> 
> Generating the data structure for the B2G jobs is going to be easier.