1763532 - Index and cross-ref markdown (.md) documentation files with markdown semantic source listing and optional full display rendering accessible via hotkey

Status: NEW

(Webtools :: Searchfox, enhancement)

Description

[Andrew Sutherland [:asuth] (he/him)]

Until bug 1417222 searchfox would attempt to render .rst and .md files via pandoc, but this came at the expense of searchfox's blame and diff functionality. While I think it still makes sense to try and treat firefox-source-docs.mozilla.org as the canonical rendering of docs and link to them (bug 1458882), especially now that firefox-source-docs supports markdown, I think searchfox should try and cross-reference markdown-based docs.

This feature-set potentially has a bunch of overlap with bug 1697671 and in particular the comment cross-referencing functionality I proposed. Unless we already have introduced a "linker" post-initial-cross-referencing re-processing of the (source/)analysis files, this would probably want to start out very minimal.

The minimal first step for this might be:

• Create new qkinds of "doc-def" and "doc-use" or equivalent. The goal would be to distinguish something that's clearly documentation about the thing (ex: in a header) versus a mention. "doc-def" would be just below "def" in our ranking and "doc-use" would be just above "use".
  • These kinds would be distinct from comments on definitions and declarations. I think the presence of a comment on def/decl would want to be metadata and (continue to) impact the peekRange. In particular, trying to treat the comment as separable from the def/decl as a separate qkind seems like an implementation nightmare.
• Add an analyzer that uses pulldown-cmark to process markdown files and looks for explicit back-ticked text that's not already part of a link and that matches a heuristic that suggests it's an identifier. Emit the new qkinds from above for these cases.
  • CamelCase suggests identifier.
  • cpp::looking::Stuff:things suggests identifier.
• Have the default searchfox display of markdown files continue to be the markdown source rep, but with the values in backticks that look like symbols have the full data-symbols treatment.
• Have an optional rendered mode that's also pre-baked into the HTML and that can be toggled to via hotkey/navigation bar button.
  • I wonder if it's possible/sane to do something clever such that the rendered markdown is emitted in divs that act like an extra column in the table display we use for source display so that the rendering can still be tied to the line numbers and have blame, but also be styled. For things like ul/li and ol/li where it's unlikely for the semantics to be maintained in a sane fashion across the role=cell for each li for a11y purposes, it does seem that aria-rowspawn is a thing and could work.
  • This would require that when the rendering is displayed that we drop all max-height constraints on lines.
  • This could allow for display of both the source + rendered mode, or just one or the other, which could be fairly neat!

Specific non-goals:

• This would not support .rst files and .rst support would be WONTFIX. firefox-source-docs supports markdown, tools like pandoc can convert .rst to markdown, and there just isn't a rust ecosystem for .rst. It's also my impression that people primarily authored .rst docs to get rendered in firefox-source-docs because they either didn't know markdown was an option or until bug 1746782 was fixed ~4 months ago the markdown support was less powerful than the rst support.