920314 - Publish mozilla-central sphinx docs with MDN theme

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: General, defect)

Description

[Gregory Szorc [:gps]]

In bug 917988 we added Sphinx documentation to mozilla-central. Files at https://hg.mozilla.org/mozilla-central/file/default/build/docs. Generated site at http://build-docs.paas.allizom.org/. It's already using the Sphinx MDN theme.

I believe the MDN people want to aggregate all external docs on MDN. So, filing a bug to track getting this new doc source integrated.

Comment 1

[Les Orchard [:lorchard]]

A few things, with an eye toward getting this hooked up with MDN:

We've been focusing initially on docs that use readthedocs.org as the build-and-host service (bug 877197) and temporarily using mod_proxy to "host" those docs at MDN URLs (bug 882927) until we can offer our own generic Sphinx build service (bug 882919).

That said, you've already got docs building on the PAAS. So, you may or may not care about Read the Docs, if your current solution works for you.

If we could get all the IT stars to align, we could set up some sort of scheduled transfer (eg. rsync in a cronjob?) that just plonks your built docs into a directory on our servers.

Long story short, what can we do that works well for you & your process to get your docs served up from an MDN URL?

Comment 2

[Gregory Szorc [:gps]]

I'd prefer to not have the MDN publishing relying on my personal PAAS service.

I could probably set up a job on https://jenkins.mozilla.org/ that invokes Sphinx and makes a tarball (or similar) available. Then someone could install a CRON on MDN to sync things over. Would that work?

Comment 3

[Luke Crouch [:groovecoder]]

IMO ReadTheDocs is ideal - the better we can refine that process the more projects we can bring into the fold.

Comment 4

[Gregory Szorc [:gps]]

I set up a Jenkins job:

https://ci.mozilla.org/job/mozilla-central-docs/

It produces HTML docs:

https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/?

And a .tar.gz:

https://ci.mozilla.org/job/mozilla-central-docs/lastSuccessfulBuild/artifact/build-docs.tar.gz

Do you still want a ReadTheDocs setup?

Comment 5

[Gregory Szorc [:gps]]

ReadTheDocs wants a setup.py in the root directory of mozilla-central. I'm unwilling to create said file because it will only serve to cause confusion. We have enough problems with Python in m-c as it is. I don't want to pile on.

Is the tarball on ci.mozilla.org sufficient?

Comment 6

[Luke Crouch [:groovecoder]]

+jezdez - any idea if we can get around the setup.py file requirement for mozilla-central to hit ReadTheDocs?

Comment 7

[Gregory Szorc [:gps]]

New home for the generated docs is:

https://ci.mozilla.org/job/mozilla-central-docs/Tree_Documentation/
https://ci.mozilla.org/job/mozilla-central-docs/lastSuccessfulBuild/artifact/tree-docs.tar.gz

This reflects changes in bug 939367 that allow any part of the tree to have its docs aggregated by Sphinx. It's quite possible in-tree docs writing might explode because of this new feature.

Is there an ETA on the mdn-sphinx-theme Python package upgrading to the hot new MDN theme?

Comment 8

[Luke Crouch [:groovecoder]]

No ETA, sorry. Current priority is improving performance on the new theme first. :)

Comment 9

[Gregory Szorc [:gps]]

https://gecko.readthedocs.org/ is now a thing. Per comment #1, I guess this means we're only a single mod_proxy rule away from exposing docs on MDN. Although, we'd need a working MDN Sphinx theme, which I don't believe has been working for about a year (since the MDN theme refresh).

Comment 10

[Luke Crouch [:groovecoder]]

Note: I sent https://github.com/mozilla/kuma/pull/2981 to update the sphinx-template-generator for the new MDN design. Once that's merged, I can work with :lorchard to make a new MDN sphinx template release.

Comment 11

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/3a88bf3e5fba9046eed70a0e4cfd6971ff77255a
fix bug 920314 - tweak template & css for sphinx

https://github.com/mozilla/kuma/commit/2a89af779afac31c5e639010b58dc3ab7d50a499
Merge pull request #2981 from groovecoder/update-sphinx-template-920314

fix bug 920314 - tweak template & css for sphinx

Comment 12

[Luke Crouch [:groovecoder]]

needinfo? for myself and :jezdez to pair on the PyPI release of the theme

Comment 13

[Luke Crouch [:groovecoder]]

https://github.com/mozilla/mdn-sphinx-theme/pull/2

Comment 14

[Luke Crouch [:groovecoder]]

https://pypi.python.org/pypi/mdn-sphinx-theme is updated and ready to use.

:gps - you should be able to set your theme to this new mdn theme now.

Comment 15

[Gregory Szorc [:gps]]

Attachment: Build Sphinx docs with MDN theme

The MDN theme works again, let's use it.

Comment 16

[Gregory Szorc [:gps]]

When using the new theme, some things aren't rendering too well. Please apply the patch against mozilla-central, run |./mach build-docs| and take a look in objdir/docs/html/ for the output.

Notably:

* Definition lists have no formatting, making them very difficult to read
* Pages like build/buildsystem/mozbuild-symbols.html are much harder to read on MDN compared to RTD.
* Spacing generally feels off.
* The TOC doesn't have indentation

It feels like we're missing CSS rules or something. But I didn't see any 404s from my network panel. I didn't dig too deeply, however.

I think an MDN guru should have a look and assess whether this is a Sphinx theme bug or MDN lacking styling bug.

Comment 17

[Luke Crouch [:groovecoder]]

Sorry, I've never worked with mozilla-central. :/

Can you point me at an article that explains how to "apply the patch against mozilla-central, run ./mach build-docs" ?

Comment 18

[Gregory Szorc [:gps]]

$ hg clone https://hg.mozilla.org/mozilla-central
$ cd mozilla-central
$ hg import --exact https://bug920314.bugzilla.mozilla.org/attachment.cgi?id=8544707
$ hg up tip
$ ./mach build-docs

Comment 19

[Luke Crouch [:groovecoder]]

Thanks Gregory. I was able to build and see it locally. I'm going to needinfo? either :openjck or :shobson to take a look when they can.

:openjck, :shobson - you can use Gregory's steps in Comment 18 to build the mozilla-central docs with the MDN sphinx theme. Though if you change the last step to:

./mach build-docs docs-with-mdn-theme

Then it's a little easier to open at:

open docs-with-mdn-theme/docs/html/index.html

If you get a chance, can one of you take a quick look and estimate whether the theme needs markup changes, or if we just need to add some more CSS styles to sphinx.styl?

Comment 20

[Luke Crouch [:groovecoder]]

Grr ... that open line should be:

open docs-with-mdn-theme/html/index.html

(i.e., take out the extra "docs" directory I put between theme/ and /html/)

Comment 21

[John Karahalis [:openjck]]

(In reply to Gregory Szorc [:gps] from comment #16)
> When using the new theme, some things aren't rendering too well. Please
> apply the patch against mozilla-central, run |./mach build-docs| and take a
> look in objdir/docs/html/ for the output.
> 
> Notably:
> 
> * Definition lists have no formatting, making them very difficult to read
> * Pages like build/buildsystem/mozbuild-symbols.html are much harder to read
> on MDN compared to RTD.

These issues can be corrected by adding the class "text content" to the element #wiki-content.

> * Spacing generally feels off.

Hm. Is there anything in particular that feels off? We do use more/different page elements in our production documentation, so that might have something to do with it. The spacing looks similar to me when compared to a normal page [1] with the sidebar hidden.

> * The TOC doesn't have indentation

RTD uses different classes to indicate TOC hierarchy. The TOC looks as it should with this rule added:

    #toc li:not(.toctree-l1) {
        display: none;
    }

More styling would be needed to handle the case where sub-menus /should/ be shown.

> It feels like we're missing CSS rules or something. But I didn't see any
> 404s from my network panel. I didn't dig too deeply, however.
> 
> I think an MDN guru should have a look and assess whether this is a Sphinx
> theme bug or MDN lacking styling bug.

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/border-image

Comment 22

[John Karahalis [:openjck]]

Correction: The class should be "text-content" not "text content".

Comment 23

[Gregory Szorc [:gps]]

I'm not actively working on this.

Comment 24

[Kan-Ru Chen [:kanru] (UTC+9)]

Hi Stephanie, you closed this bug as FIXED, does it mean the in-tree docs are now automatically pushed to MDN?

Comment 25

[Stephanie Hobson [:shobson]]

Sorry, didn't mean to mark as fixed, just wanted to remove the needs info for myself.

Comment 26

[John Whitlock [:jwhitlock]]

We're deprecating the MDN Sphinx theme (bug 1361729), so you may want to look at other theme options for these documents.

I'm -1 on this approach for integrating docs into MDN. They look like MDN docs, because they have the same header, footer, and style. Unlike MDN-native docs, they won't be integrated into site search, and won't be available for macro processing, translation, or live editing.  I think it adds confusion rather than meets user needs.  A larger cross-team discussion is needed if we're going to build an all-in-one documentation site, starting with "why" rather than "how".

Comment 27

[Gregory Szorc [:gps]]

Then I guess there isn't a future for this bug then.