1428139 - Run `rustdoc` on libgkrust, and host the results somewhere

Status: ASSIGNED

(Developer Infrastructure :: Source Documentation, enhancement)

Description

[Nika Layzell [:nika] (ni? for response)]

It would be really handy to have the `rustdoc` for libgkrust hosted somewhere. This would contain the documentation for `gkrust`, every rust crate in the source tree, and every vendored rust crate which we depend on, which could be very handy for rust developers.

For example, when writing the `nsstring` crate, I tried to make sure that the crate had good documentation when rendered with rustdoc, but that isn't hosted anywhere right now as far as I know.

It might also be good to add a `mach doc rust` or similar mach command (I believe mach doc runs jsdoc right now?) which runs rustdoc and generates documentation in the objdir so that it is easy to run rustdoc on libgkrust locally.

To make it easier to learn how to use `xpcom` in rust when I land my bindings in bug 1293362, I'm hoping to generate good rustdoc documentation for the new xpcom crate, and that will be much more useful if there's an easy way to view it.

Comment 1

[Gregory Szorc [:gps]]

It is trivial to add a task to CI that runs `rustdoc`. That task would likely be defined at https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/ci/source-test/doc.yml.

The tricky parts are:

* Getting a Rust toolchain in the execution environment. We may need to modify the Docker image. However, the Rust toolchain is pinned in the source repo and there is TaskCluster magic to set up task dependencies and build system code to download an appropriate Rust toolchain.
* Uploading the docs somewhere. We currently upload the Sphinx docs to an S3 bucket. There is a TaskCluster "secret" containing AWS credentials for that bucket. We'll need to figure out an appropriate destination path in the S3 bucket for the docs.

Dustin or I can help with the latter bit. But the first step is to get a task generating the docs: we can worry about upload after.

glandium: are the toolchain task dependencies in such a state to facilitate easily using them here? Or would it be simpler to just one-off `rustup install` in a Dockerfile until they are?

Comment 2

[Mike Hommey [:glandium]]

You can't use toolchain task artifacts from Dockerfiles, because of the kind dependencies (toolchains depend on docker images). What you can do is install the artifacts from the doc task instead of "hardcoding" it in the docker image. That's what we do for everything.

Comment 4

[Nicholas Nethercote [inactive]]

FWIW, you can locally generate and view rustdocs for a crate in m-c like so:

cd xpcom/rust/xpcom
MOZ_TOPOBJDIR=/path/to/objdir cargo doc
cd -
firefox target/doc/xpcom/index.html

Which is better than nothing.

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

Recognizing that hosting is not the main effort here, searchfox could host the rendered docs which could enable a high confidence[1] "go to rustdoc for FOO" context menu option in searchfox and appropriate annotations in search results.

1: High confidence in this case meaning that searchfox could gate the availability of the context menu on having downloading the rendered rustdoc directly corresponding to the indexed revision and extracted it. It would of course always be possible for searchfox to blindly generate links to the docs hosted elsewhere, it's just harder to guarantee invariants about the links.

Comment 6

[Manuel Bucher [:manuel]]

When trying to compile the docs on my linux setup I get build errors regarding std::os::windows not being available. But building the docs on windows might work (I don't have a build environment available for that, but we could use the CI).

BUILDCONFIG_RS=/home/user/dev/gecko/obj-x86_64-pc-linux-gnu/build/rust/mozbuild/buildconfig.rs MOZ_TOPOBJDIR=obj-x86_64-pc-linux-gnu/ cargo doc --workspace

Andrew, is that enough for you to host the docs on searchfox? I think that'd be great. I have no experience in CI, so couldn't try running that command (modified to the windows environment) there yet. I would like to help with this bug if further work is required and someone could mentor me (as this is also marked as good-first-bug).

Comment 7

[Andrew Sutherland [:asuth] (he/him)]

(I'm no longer working on searchfox, clearing need-info.)

Comment 8

[Manuel Bucher [:manuel]]

Attachment: Bug 1428139 - Add `./mach cargo doc` sub command r=sylvestre

This is running "cargo doc" on gkrust. The results are in the object
directly. On linux the entry point of the documentation could look like:

obj-x86_64-pc-linux-gnu/x86_64-unknown-linux-gnu/doc/gkrust/index.html

The output of obj-x86_64-pc-linux-gnu/x86_64-unknown-linux-gnu/doc/
contains all the documentation of crates that can be hosted somewhere.

The documentation build takes 4:25 minutes on my computer.

Comment 9

[Manuel Bucher [:manuel]]

A few other steps necessary:

• create an archive with the docs https://searchfox.org/firefox-main/rev/804ffad3df416ac8418f3c0df7679a8fe3d718fa/toolkit/mozapps/installer/packager.mk#88-95 (thanks emilio for giving this example)
• retrieve this archive to host the documentation somewhere. Hosting it somewhere. Directly on searchfox feels a bit misfitting. On firefox-source-docs would probably be better. Perhaps "https://firefox-source-docs.mozilla.org/rustdoc/"?

@sylvestre: Can you mentor on after building this in CI how to put this on source-docs?

Comment 10

[Emilio Cobos Álvarez [:emilio]]

(see above)

Comment 11

[Emilio Cobos Álvarez [:emilio]]

FWIW I think our docs are currently built like this

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Ahal knows the upload part more than I
(but I am curious, i would like to enable https on the staging doc page :)

Comment 13

[Manuel Bucher [:manuel]]

I'm now considering to just build the rust doc as part of ./mach doc so that it automatically gets picked up by the current CI job instead of adding a separate archive that needs to be retrieved separately. (Leaving the needinfo for https on the staging doc page.)

Comment 14

[Andrew Halberstadt [:ahal]]

I'm now considering to just build the rust doc as part of ./mach doc so that it automatically gets picked up by the current CI job instead of adding a separate archive that needs to be retrieved separately. (Leaving the needinfo for https on the staging doc page.)

Yes, this is what I was going to suggest. If you can find a Sphinx extension that can integrate rust docs into the existing Sphinx build, that's even better. After a quick search I found:
https://sphinxcontrib-rust.readthedocs.io/en/stable/

But I'm not familiar with Rust docs and am unsure if this satisfies the need you all are talking about here. The "Motivation" blurb certainly sounds like our use case though:

This is primarily meant for teams and projects that are already using Sphinx as a documentation build tool, and would like to include documentation for Rust projects in it along with Python, C, and other languages.

Comment 15

[Andrew Halberstadt [:ahal]]

A benefit of bundling it with the current docs is the ability to use the roles and directives the above extension provides.

E.g, in any random doc file you'll be able to write:

:rust:struct:`Foo`
:rust:struct:`See Foo struct for more details <Foo>`

(or the myst_parser equivalent for markdown files)

And those will create a link directly to the API docs for the targeted item.

Comment 16

[Sylvestre Ledru [:Sylvestre]]

I'm now considering to just build the rust doc as part of ./mach doc

if it takes 4 minutes, i would rather not for now - at least in the local builds
in the ci, why not