1048650 - Submit mozilla-central Sphinx documentation generation job results to treeherder

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

It's my understanding that treeherder was designed to accept job results from <not buildbot>. If so, we have https://ci.mozilla.org/job/mozilla-central-docs/ doing Sphinx documentation generation for mozilla-central. I'd like to have this Jenkins job reflected on treeherder so a) docs bustages are discovered easier b) increase visibility into the existence of the Sphinx docs.

Please let me know what you need to make this happen.

Comment 1

[Gregory Szorc [:gps]]

I'm not sure how treeherder requests like this get triaged. Needinfo on edmorley to provide that.

Comment 2

[Ed Morley [:emorley]]

The main priority at the moment is fixing the items that are preventing the sheriffs (and other devs) from moving from TBPL to treeherder. Until that point, populating treeherder with additional data sources is likely on the back burner. 

That said, since in the treeherder-world results are posted to the treeherder-service, rather than being polled via explicit treeherder support, I imagine most of the work will be on the jenkins side, and as such something you could do outside of the treeherder-devs workload. Needinfo'ing jeads since he'll be able to explain more.

Comment 3

[Jonathan Eads ( :jeads )]

Not entirely sure I understand the request here. But if you want to associate documentation with a particular Jenkins job in treeherder you can do that with a job artifact https://github.com/mozilla/treeherder-client#job-artifacts-format. There's a special artifact called the "Job Info" that allows you to associate any number of url's with a particular job from any build/test system. These urls are displayed in the job info panel in the UI when a job is selected, so you can include links to whatever build/test job documentation you require.

{

    "blob": {
        "job_details": [
            {
                "url": "https://www.mozilla.org",
                "value": "website",
                "content_type": "link",
                "title": "Mozilla home page"
            },
            {
                "value": "bar",
                "content_type": "text",
                "title": "Foo"
            },
            {
                "value": "This is <strong>cool</strong>",
                "content_type": "raw_html",
                "title": "Cool title"
            }
        ],
    },
    "type": "json",
    "name": "Job Info"
}

Comment 4

[Ed Morley [:emorley]]

(In reply to Jonathan Eads ( :jeads ) from comment #3)
> Not entirely sure I understand the request here. But if you want to
> associate documentation with a particular Jenkins job in treeherder

gps wishes to submit results for externally run jobs to treeherder, so that the jobs can appear next to the appropriate gecko revision on mozilla-central. These jobs happen to generate gecko build documentation, but we don't need to add URLs to the docs as artefacts - we just want the results of these externally scheduled jobs to appear on treeherder.

Could you outline the steps gps will need to perform to achieve this? :-)

Comment 5

[Mauro Doglio [:mdoglio]]

This sounds very similar to what  Dave Hunt is doing for bug 1043718.
I think this is a way to do it:
1- fetch from the treeherder api a revision hash that identifies your result set (aka push) given a revision. The url to fetch would look like https://treeherder.mozilla.org/api/project/mozilla-inbound/revision-lookup/?revision=b17cad2d1e5e
2- compute a unique identifier for your job (job guid)
3- use the treeherder client to submit a job object to treeherder.
Depending on where you want to show your job in the UI, you need to provide some properties for the job, like platform & option (to show it in the right row) and job type/group (to set the position within the row ). I guess in your case you will have a completely new platform for your docs jobs. I suggest you to start hacking on it using the vagrant setup provided with treeherder-service.

Comment 6

[Ed Morley [:emorley]]

That's really helpful, thank you Mauro :-)

Comment 7

[Gregory Szorc [:gps]]

This is all helpful!

I'm totally OK with abusing the Linux64 Opt platform for reporting docs if others are :)

We already use Vagrant from Jenkins to generate the docs and we have a virtualenv for running Sphinx. So hooking up the treeherder client Python package is totally doable.

Only remaining question is authentication. How is the Treeherder API authenticated? I wonder how we'll protect the secret key on ci.mozilla.org...

Comment 8

[Mauro Doglio [:mdoglio]]

Treeherder service implements 2-legged oauth, so we need to provide you with a key/secret pair. The client library will take care of everything, you just have to pass the credentials when creating the TreeherderRequest object. To keep the oauth credentials, can you store it safely as environment variables or somewhere on filesystem?

Comment 9

[Bob Clary [:bc] (inactive)]

(In reply to Jonathan Eads ( :jeads ) from comment #3)
est job documentation you require.
> 
> {
> 
>     "blob": {
>         "job_details": [
>             {
>                 "url": "https://www.mozilla.org",
>                 "value": "website",
>                 "content_type": "link",
>                 "title": "Mozilla home page"
>             },
>             {
>                 "value": "bar",
>                 "content_type": "text",
>                 "title": "Foo"

Please note that though the documentation in https://github.com/mozilla/treeherder-client/blob/master/README.md says "text" is an available content type, only link, raw_html and TalosResult are actually supported in ://github.com/mozilla/treeherder-ui/blob/performance-data/webapp/app/plugins/pluginpanel.html#L162

I can't find a bug on that. Has that been fixed on a branch and not merged to master?

>             },
>             {
>                 "value": "This is <strong>cool</strong>",
>                 "content_type": "raw_html",
>                 "title": "Cool title"
>             }
>         ],
>     },
>     "type": "json",
>     "name": "Job Info"
> }

Comment 10

[Mauro Doglio [:mdoglio]]

The "text" content type falls in the default case in the switch at this line
https://github.com/mozilla/treeherder-ui/blob/performance-data/webapp/app/plugins/pluginpanel.html#L182
For now there is no special treatment in the ui for text other than escaping all the html content. But I like the idea of having it explicitly declared in the artifact, maybe one day another downstream application will make good use of it.

Comment 11

[Ed Morley [:emorley]]

Moving this bug to build config, since unless Treeherder bugs are unearthed, most of the work here needs to be done as part of the Jenkins CI config and not in treeherder.

Comment 12

[Gregory Szorc [:gps]]

This is made much easier with the Task Cluster work in bug 1245953.

Comment 13

[Gregory Szorc [:gps]]

Attachment: Bug 1048650 - TaskCluster task for Sphinx documentation generation;

I've wanted to have automated test coverage of Sphinx documentation
generation for years. Now that we can add new automation in-tree
using TaskCluster, this is easy to implement. So do it.

Probably the hackiest thing about this is the task reporting to the
"lint" platform. We don't really have a more appropriate place to put
this.

Review commit: https://reviewboard.mozilla.org/r/71812/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/71812/

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8781368 [details]
Bug 1048650 - TaskCluster task for Sphinx documentation generation;

https://reviewboard.mozilla.org/r/71812/#review69608

::: taskcluster/ci/legacy/tasks/tests/sphinx.yml:40
(Diff revision 1)
> +    artifacts:
> +      'public/docs.tar.gz':
> +        type: file
> +        path: '/home/worker/checkouts/gecko/docs.tar.gz'
> +        expires:
> +          relative-datestamp: '1 year'

Is the artifact necessary?  Isn't this just to make sure the docs build?

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8781368 [details]
Bug 1048650 - TaskCluster task for Sphinx documentation generation;

https://reviewboard.mozilla.org/r/71812/#review69610

Comment 16

[Gregory Szorc [:gps]]

Comment on attachment 8781368 [details]
Bug 1048650 - TaskCluster task for Sphinx documentation generation;

https://reviewboard.mozilla.org/r/71812/#review69608

> Is the artifact necessary?  Isn't this just to make sure the docs build?

There have been a few people over the years that have wanted to re-host the docs. MDN comes to mind. See bug 920314. Providing the artifact unblocks this from happening.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f5e13a9a2e36
TaskCluster task for Sphinx documentation generation; r=dustin

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/f5e13a9a2e36