Open Bug 1535452 (first-class-docs) Opened 4 years ago Updated 2 days ago

[meta] Make writing and reading our in-tree documentation a first class experience


(Developer Infrastructure :: Source Documentation, enhancement)



(Not tracked)


(Reporter: ahal, Unassigned)


(Depends on 15 open bugs)


(Keywords: meta)

Despite being critically important, our in-tree documentation system is neglected, without owner and without direction. Writing documentation can be painful, and reading the output of the documentation is even more painful:

This bug is going to track all the issues we need to fix to improve the status quo. It will also act as the launching point for a GSoC proposal I've submitted (which is my main motivation for filing this meta bug).

Summary: Make writing and reading our in-tree documentation a first class experience → [meta] Make writing and reading our in-tree documentation a first class experience

This comment is a description of how the doc system works with some links to code. This is a very general overview that is intended to provide context for anyone working on any of the dependencies filed against this meta bug.


The goal of our doc system is to allow individual teams and developers the ability to create and maintain their own domain specific documentation. We don't want to act as gatekeepers. In order for this to work, we need some mechanism for developers to register a doc dir.

Documentation gets registered by developers via files. For example:

There are two variables, SPHINX_TREES and SPHINX_PYTHON_PACKAGE_DIRS and some information on them can be found here:


Documentation is generated using sphinx with some custom tooling to hook in all of the directories that are registered via Aside from sphinx itself, there are 3 pieces to this stage:

  1. mach command
    This is the command line entry point for generating and uploading docs. It is mostly just a thin wrapper around sphinx-build, but also acts like the glue that holds the system together.

  2. custom sphinx extension
    A custom sphinx extension that acts as a hook from which we can run all our customized operations before the regular 'sphinx-build' kicks in. This module is registered in the main This extension adds a couple custom sphinx directives as well, but otherwise isn't very interesting.

  3. moztreedocs
    A little python module containing the meat of the custom generation code. It is invoked from above and is responsible for reading the variables and copying all of the relevant source files into a staging directory in the objdir. Once all the source files have been copied, sphinx-build will run against the staging dir.

At a high level, the doc generation process will go something like this:

  1. User runs mach doc or mach doc <subdir>
  2. Mach command runs sphinx-build with tools/docs/ as the config file.
  3. The mozbuild.sphinx extension is registered in this config file.
  4. This extension has a setup hook which calls into moztreedocs.
  5. moztreedocs reads all the sphinx variables and copies all the source files to a staging dir.
  6. The mozbuild.sphinx extension gets sphinx-build to build the docs from this staging directory.
  7. sphinx-build resumes as normal, typically a directory next to the staging dir will contain the html output.


Publishing docs is a separate step from generation, but uses the same mach command: mach doc --upload. This step is much simpler than the generation one. The docs are hosted on a static Amazon S3 bucket behind this domain:

The mach doc --upload command uses Amazon's boto3 library to push the output dir generated in step 7 above up to the static server:

And that's it.

Continuous Integration

While it is possible to generate docs locally, publishing requires special credentials. Generation and publication are handled by two separate tasks (Doc and DocUp respectively). They are defined here:

These tasks will run whenever someone modifies a doc source file, so changes to the documentation should be automatically picked up and published once the patch lands to mozilla-central. While it's nice to have this automated system, we do still care about running mach doc locally as developers will typically want to see how their modifications look before pushing.

Depends on: 1536103
Depends on: 1556460
Depends on: 1571671
Depends on: 1394074
Depends on: 1574948
Depends on: 1576912
Depends on: 1594925
Depends on: 1599372
Depends on: 1601578
Depends on: 1601611
Depends on: 1601612
Depends on: 1605742
Depends on: 1621950
Depends on: 1628275
Depends on: 1629012
Depends on: 1629893
Depends on: 1428139
Depends on: 1643034
Depends on: 1385460
Depends on: 1643295
Depends on: 1643293
Depends on: 1646627
No longer depends on: 1646627
Depends on: 1650197
Depends on: 1657925
Depends on: 1657926
Depends on: 1658812
Depends on: 1660615
Depends on: 1660747
Depends on: 1660749
Depends on: 1666257
Depends on: 1668466
Depends on: 1679396
Depends on: 1679631
Depends on: 1679638
Depends on: 1683628
Depends on: 1684863
Depends on: 1685147
Depends on: 1690989

(In reply to Andrew Halberstadt [:ahal] from comment #0)
HI @Andrew Halberstadt,
I'm a newcomer. I read description of this bug but I don't know how to start.
May you help me with some simple guidelines?
Thank you

I'm a newcomer. I read description of this bug but I don't know how to start.

Description of the bug explain the architecture of our docs infra and how it runs.

May you help me with some simple guidelines?

After setting up Firefox Docs locally, you can look into the bugs on which this meta bug is depended. If you have some questions feel free to ask in #firefox-source-docs in

Depends on: 1703037
Depends on: 1720922
Depends on: 1727824
Depends on: 1729217
Depends on: 1730407
Depends on: 1731022
Depends on: 1737307
Depends on: 1740681
Depends on: 1750450
Depends on: 1763208
Product: Firefox Build System → Developer Infrastructure
Severity: normal → S3
Depends on: 1802924
Depends on: 1557020
Depends on: 1763971
Depends on: 1749472
Depends on: 1749469
Depends on: 1765365
Depends on: 1807820
Depends on: 1807831
No longer blocks: 1803426
Depends on: 1803426
Depends on: 1810702
You need to log in before you can comment on or make changes to this bug.