917988 - Better support for in-tree build system documentation

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

I'd like better in-tree documentation for the build system.

I started a thread on dev.builds titled "Build system documentation." However Google Groups doesn't have the original message, so linking to a reply:

https://groups.google.com/forum/#!topic/mozilla.dev.builds/IZEInVj_fgo

My current proposal is to use Sphinx (http://sphinx-doc.org/). It's pretty powerful and extensible. It scales well up to large doc trees.

I have a preview at https://people.mozilla.org/~gszorc/build_docs/.

I'm thinking we establish build/docs, check in the Sphinx config files, move existing documentation there, and set up a mach command (or a standalone make file) to generate them. Possible followups include:

* Support for aggregating documentation from across the tree. I suspect once this lands, others will find it useful and may want to piggyback to add their own docs. We'll deal with that if/when it becomes a problem.

* Publishing the results somewhere. Bug 877197 is relevant.

Since we have some semi-urgent needs for docs in bug 901990, I plan to put some minor polish on my existing patch and throw it up for review.

Comment 1

[Gregory Szorc [:gps]]

Oh, I forgot to state explicitly that I don't want to check Sphinx into the tree (at least not yet). It's NPOTB so we don't technically need to import it. We can have a mach command |pip install sphinx| as part of the documentation generation process easily enough.

Comment 2

[Gregory Szorc [:gps]]

Attachment: Use Sphinx for generating build system documentation

I created a new directory: build/docs.

I created a Sphinx config file. It uses the MDN theme.

I moved the README.rst files from python/mozbuild under build/docs.

While I was bored I created some new pages to kinda/sorta document
mozconfigs and PGO. They are half-baked. I also created a basic
glossary.

Finally, I created a "build-docs" mach command. It activates the
virtualenv and installs the MDN sphinx theme via pip. This magically
pulls in the sphinx dependencies.

It just works.

This is just the beginning, but I believe it is good enough to check in.

Comment 3

[Benjamin Smedberg]

Comment on attachment 806956 [details] [diff] [review]
Use Sphinx for generating build system documentation

+from __future__ import unicode_literals

Why? The important part of this file appears to use u'' syntax anyway...

+import mdn_theme

Where does this come from? I didn't see it in the tree or patch.

Comment 4

[Gregory Szorc [:gps]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #3)
> +from __future__ import unicode_literals
> 
> Why? The important part of this file appears to use u'' syntax anyway...

This is just my SOP for Python these days. Helps ensure forward compatibility with Python 3.

The u'' in this file came from the auto generator. I'll strip them out on landing.

> +import mdn_theme
> 
> Where does this come from? I didn't see it in the tree or patch.

From the mach command:

self.virtualenv_manager.install_pip_package('mdn-sphinx-theme==0.3')

Magic!

Comment 5

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/85811665eed7

Comment 6

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Comment on attachment 806956 [details] [diff] [review]
Use Sphinx for generating build system documentation

Review of attachment 806956 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/docs/conf.py
@@ +23,5 @@
> +source_suffix = '.rst'
> +master_doc = 'index'
> +project = u'Mozilla Build System'
> +year = datetime.now().year
> +copyright = u'%s, Mozilla Foundation, CC BY-SA 3.0' % year

I'd like gerv to sign this off, just to be sure.

Comment 7

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/85811665eed7

Comment 8

[Gervase Markham [:gerv]]

Comment on attachment 806956 [details] [diff] [review]
Use Sphinx for generating build system documentation

Thanks for asking; CC-BY-SA is our standard licence for documentation, so that's great (see e.g. MDN). Please add the text "or any later version".

Gerv

Comment 9

[Mike Hommey [:glandium]]

(In reply to Gervase Markham [:gerv] from comment #8)
> Comment on attachment 806956 [details] [diff] [review]
> Use Sphinx for generating build system documentation
> 
> Thanks for asking; CC-BY-SA is our standard licence for documentation, so
> that's great (see e.g. MDN). Please add the text "or any later version".

But... does that really work if the documentation is derived from code?

Comment 10

[Gregory Szorc [:gps]]

Added "or any later version."

https://hg.mozilla.org/integration/mozilla-inbound/rev/e192b21e0934

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/e192b21e0934

Comment 12

[Gervase Markham [:gerv]]

(In reply to Mike Hommey [:glandium] from comment #9)
> But... does that really work if the documentation is derived from code?

Ah, sorry, good point. No, in that case the docs need to be MPL2. Can someone please fix this, or open a new bug?

Thanks, and sorry for the delay in getting back to this,

Gerv

Comment 13

[Gregory Szorc [:gps]]

Let's just iron it out here since this is where all the context is.

That being said, if you go to https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/index.html, you'll see we are getting the copyright line from the MDN template. So I don't think the copyright line is relevant. If things work after deleting it, I'll just do that.

Comment 14

[Gervase Markham [:gerv]]

That page says: "Content is available under these licenses", with a link to https://developer.mozilla.org/en-US/docs/Project:MDN/About , which says "Unless otherwise indicated, the content is available under the terms of the Creative Commons Attribution-ShareAlike license (CC-BY-SA), v2.5 or any later version." 

So we could either:

1) Update each auto-generated page to say "Actually, this page is MPL 2"
2) Update the "these licenses" page to say "Autogenerated stuff is MPL 2"
3) Update the autogenerated pages to have a different link.

I'd probably go for 1), as that seems to be the current accepted exception mechanism.

Gerv