2008577 - Upgrade sphinx to 9.0.4

Status: RESOLVED FIXED

(Developer Infrastructure :: Source Documentation, task)

Description

[Sylvestre Ledru [:Sylvestre]]

we can't upgrade to 9 because of https://github.com/executablebooks/MyST-Parser/pull/1076

Sphinx 8 changelog: https://www.sphinx-doc.org/en/master/changes/8.2.html

dependency upgrades needed:
- myst-parser: 2.0 → 4.0.1
- sphinx-design: 0.5.0 → 0.6.1
- sphinx-rtd-theme: 2.0.0 → 3.0.2
- sphinx-js: 4.0.0 → 5.0.2

Comment 1

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - Upgrade sphinx to 9.0.4 and related dependencies r?#firefox-source-docs-reviewers

Upgrade Sphinx from 7.4.7 to 8.2.3 along with compatible versions of:

• myst-parser: 2.0 → 4.0.1
• sphinx-design: 0.5.0 → 0.6.1
• sphinx-rtd-theme: 2.0.0 → 3.0.2
• sphinx-js: 4.0.0 → 5.0.2

Comment 2

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: Add mach.commands import warnings to allowed list r?#firefox-source-docs-reviewers

Add mach.commands.commandinfo and mach.commands.settings import
warnings to the known failures list. These modules have deep
dependency chains that would require adding many build infrastructure
dependencies to the docs environment.

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: fix and ignore some errors r=#firefox-source-docs-reviewers

Add missing vendored dependencies to fix import errors:

• mozilla_taskgraph (fixes ~150 gecko_taskgraph import failures)
• ply (fixes mozwebidlcodegen import failure)

Fix Sphinx compatibility warning in moztreedocs by using pathlib.Path
objects instead of string paths for comparisons.

Comment 4

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: Fix JSDoc type parsing errors for Sphinx compatibility r?#firefox-source-docs-reviewers

Convert TypeScript-style JSDoc type syntax to standard JSDoc format
that sphinx-js can parse:

• Arrow function types (param: Type) => ReturnType → Function
• Index signatures {[key: string]: Type} → Object
• Tuple types [Type1, Type2] → Array
• Generic types Values<typeof X>, Parameters<X> → * or base type
• Intersection types Type1 & Type2 → Type1 or Object
• Import types import("file").Type → *

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: Fix JSDoc @type tags with descriptions for Sphinx compatibility r?#firefox-source-docs-reviewers

The JSDoc @type tag does not permit descriptions.

Fixed three patterns:

1. Multi-line comments where @type was first, followed by description
   lines - moved descriptions before @type

2. Single-line comments like /** @type {X} Description / - reordered
   to /* Description @type {X} */

3. Comments with descriptions both before and after @type - removed
   redundant descriptions after @type

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc/nspr: remove some invalid link r?#firefox-source-docs-reviewers,kaie,glandium

It won't regress the doc. The links are already invalid.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/23d1f2b82e89
https://hg.mozilla.org/integration/autoland/rev/01c20c3ea9c7
doc/nspr: remove some invalid link r=firefox-source-docs-reviewers,championshuttler DONTBUILD

Comment 8

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/01c20c3ea9c7

Comment 9

[Takanori MATSUURA]

(In reply to Sylvestre Ledru [:Sylvestre] from comment #0)

we can't upgrade to 9 because of https://github.com/executablebooks/MyST-Parser/pull/1076

The issue has been fixed and MyST-Parser 5.0.0 has been released.
Will upgrading to Sphinx 9 be considered a separate bug?

Comment 10

[Sylvestre Ledru [:Sylvestre]]

yeah, i have been working on the doc platform

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: make mermaid diagrams legible after the sphinxcontrib-mermaid upgrade r?#firefox-source-docs-reviewers

sphinxcontrib-mermaid 2.x forces every diagram into a 100% x 500px box,
which scales tall diagrams down until their labels are unreadable and
stretches short ones across the whole page. Size the diagrams to their
content instead and bump the label font size.

The nodes in how_to_submit_a_patch hardcode a light blue fill but left
the label color to the theme, giving grey on light blue. Set an explicit
dark label color so they are readable in both themes.

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: fix JSDoc markup rejected by myst-parser 5 r?#firefox-source-docs-reviewers

The updated sphinx-js/jsdoc pipeline no longer accepts the {*} any-type
shorthand, so spell it {any} everywhere it is used.

Also update the allowed warnings for sphinx 9 and vendor third_party/python/ply
for the docs virtualenv.

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - Upgrade vendored markdown-it-py to 4.2.0 r?ahal

myst-parser 5.x requires markdown-it-py ~=4.2, but the tree vendors 3.0.0
and taskcluster/requirements.in caps it below 4. That cap was added in
524c505b0b94 when the previous myst-parser 5.1.0 update was backed out,
and is marked temporary: it only exists to keep myst-parser 2.0.0
resolvable. Lift it so the sphinx 9 upgrade can proceed.

rich is the only real consumer and requires markdown-it-py>=2.2.0.
Neither taskcluster-taskgraph nor mozilla-taskgraph depends on it.

Comment 14

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 2008577 - doc: keep relative document links inside the staging tree r?#firefox-source-docs-reviewers

Caused by the Sphinx 7.4.7 -> 9.0.4 upgrade earlier in this stack:
BuildEnvironment.relfn2path used to normalise with path.normpath(), and
now uses Path.resolve(). Since the docs are built from a staging tree of
symlinks into the source tree, resolve() follows the symlink of the
document being linked to and lands outside the Sphinx source directory,
so myst-parser reports "Unknown source document" - 1339 warnings on a
clean build.

Override relfn2path to normalise textually again, so ".." still collapses
but the path stays in the staging tree. Known failures drop from 1624 to
298. The 13 newly reported ones are the already-allowlisted "local id not
found in doc": links reaching the right document but a stale anchor,
previously masked by the link failing outright.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/03636c5df8ea
https://hg.mozilla.org/integration/autoland/rev/285917b42707
Upgrade vendored markdown-it-py to 4.2.0 r=ahal,mach-reviewers,ahochheiden
https://github.com/mozilla-firefox/firefox/commit/bec2739c10c6
https://hg.mozilla.org/integration/autoland/rev/0c30ba5d18fc
Upgrade sphinx to 9.0.4 and related dependencies r=firefox-source-docs-reviewers,ahal
https://github.com/mozilla-firefox/firefox/commit/62268677ac71
https://hg.mozilla.org/integration/autoland/rev/3a9ae208f398
doc: make mermaid diagrams legible after the sphinxcontrib-mermaid upgrade r=firefox-source-docs-reviewers,ahal
https://github.com/mozilla-firefox/firefox/commit/c4d632c0ad8d
https://hg.mozilla.org/integration/autoland/rev/2be8eac87ae0
doc: fix JSDoc markup rejected by myst-parser 5 r=firefox-source-docs-reviewers,ahal

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/873894ed0edc
https://hg.mozilla.org/integration/autoland/rev/cfca1a328e77
Revert "Bug 2008577 - doc: fix JSDoc markup rejected by myst-parser 5 r=firefox-source-docs-reviewers,ahal" for causing ci failures

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/54a1310e36e4
https://hg.mozilla.org/integration/autoland/rev/b605a56ef86d
Upgrade vendored markdown-it-py to 4.2.0 r=ahal,mach-reviewers,ahochheiden
https://github.com/mozilla-firefox/firefox/commit/b852c7e3fe37
https://hg.mozilla.org/integration/autoland/rev/857ef7b3a6f8
Upgrade sphinx to 9.0.4 and related dependencies r=firefox-source-docs-reviewers,ahal
https://github.com/mozilla-firefox/firefox/commit/a808f679f11a
https://hg.mozilla.org/integration/autoland/rev/f6778f90c4bb
doc: make mermaid diagrams legible after the sphinxcontrib-mermaid upgrade r=firefox-source-docs-reviewers,ahal
https://github.com/mozilla-firefox/firefox/commit/74287815dbb2
https://hg.mozilla.org/integration/autoland/rev/fd13a137dd67
doc: fix JSDoc markup rejected by myst-parser 5 r=firefox-source-docs-reviewers,ahal

Comment 18

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/b605a56ef86d
https://hg.mozilla.org/mozilla-central/rev/857ef7b3a6f8
https://hg.mozilla.org/mozilla-central/rev/f6778f90c4bb
https://hg.mozilla.org/mozilla-central/rev/fd13a137dd67

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/21ec718ad5fe
https://hg.mozilla.org/integration/autoland/rev/defe56966f67
doc: keep relative document links inside the staging tree r=firefox-source-docs-reviewers,ahal DONTBUILD

Comment 20

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/defe56966f67