1115452 - Generate XPIDL docs in Sphinx

Status: NEW

(Firefox Build System :: General, defect)

Description

[Gregory Szorc [:gps]]

I know there was an effort once upon a time to generate XPIDL interface documentation as part of the Sphinx docs. I'm pretty sure that effort stalled.

I have a proof-of-concept working. It would need lots of enhancement before it can land. I'll upload the patch here in hopes someone wants to take things over.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Generate XPIDL interface documentation in Sphinx

This patch starts to implement XPIDL interface documentation in Sphinx.
Essentially, we scan the source directory for .idl files, parse them,
then convert the interfaces into ReST, which Sphinx will turn into HTML
for us.

The current implementation is far from robust. We're not rendering
attributes, parameters to methods, etc. Also, we probably need to parse
the comment strings for @param references, etc. I imagine it is a few
hours of work.

One thing to note is that this patch makes Sphinx documentation
painfully slow. I suspect Sphinx is doing some clownshoesy things. Will
need to profile to learn more.

Comment 2

[Gregory Szorc [:gps]]

Erik Rose may like the code in case he wants DXR to grow XPIDL parsing abilities.

Comment 3

[Joshua Cranmer [:jcranmer]]

(Note to look at this when I have more time)

Comment 4

[Joshua Cranmer [:jcranmer]]

Comment on attachment 8541369 [details] [diff] [review]
Generate XPIDL interface documentation in Sphinx

Review of attachment 8541369 [details] [diff] [review]:
-----------------------------------------------------------------

We have used Doxygen to parse IDL in the past. I know Doxygen has choked on xpidl before (%{C++ %} in particular), which makes me wary of actually relying on Doxygen itself. The downside is that Doxygen possesses very powerful documentation-generating faculties, but unfortunately, there's no way to get at those faculties without using it for parsing as well. That we've used Doxygen in the past means we have fallen into the trap of using some of the more advanced features from time to time (Doxygen suffers from the malaise of supporting every documentation style under the sun, so there is often a bajillion ways to do the same thing). However, we completely control the syntax and use of XPIDL, so we could enforce our own rules for documentation (e.g., the build fails if the documentation for an interface doesn't make semantic sense).

A separate issue that needs to be raised is what the goal of building XPIDL docs with Sphinx actually does. Is the purpose to provide people documentation on their local machines--or are we looking to eventually submit this documentation to MDN automagically? If we're talking about the latter, we need to come to grips with somehow reflecting differences between different versions ({m,c}-{c,a,b,r,e}), since our IDL interfaces are by no means static and stable.

A third issue to think about is the long-term goal of documentation. We have C++ header files, XPIDL, and JS files (both frontend and .jsms) that all strongly desire some sort of documentation, and possibly other things (XBL components? Maybe even some HTML/XUL files? What about whatever Gaia uses?). It would be unfortunate to end up using multiple documentation tools for all of these things, although I'll admit that I don't particularly see a route to solving this.

By no means are these to be taken as requirements for an acceptable patch--these are just issues and potentially desirable features that would require leveraging whatever would be added in this bug. If we can start with a design that makes these features easier rather than harder to implement, it is a better start.

::: tools/docs/moztreedocs/__init__.py
@@ +139,5 @@
> +        finder = FileFinder(self._topsrcdir, find_executables=False,
> +            ignore={'obj*'})
> +
> +        xpidl_paths = set(p[0] for p in finder.find('**/*.idl'))
> +        xpidl_paths = set(os.path.join(self._topsrcdir, p) for p in xpidl_paths)

Wouldn't it make more sense to grab everything in XPIDL_SOURCES? And, arguably, we should also ignore IDL files that don't get installed (I forgot the magic build verbs for that).

There's also a risk of mixing up MSIDL with XPIDL--we have MSIDL files in-tree (accessibility and MAPI).

@@ +182,5 @@
> +
> +    def _get_xpidl_interface_docs(self, i):
> +        # This function is horrible. We probably need full docblock parsing
> +        # support to extract @param.
> +        def format_comments(c):

I'd argue that comment formatting of doccomments belongs in xpidl.py, on the basis that the actual spelling of the comment is merely an artifact of the transmission medium of source code.

Needless to say, it's worth pointing out:
A. /* */ isn't usually considered a documentation comment.
B. Since we've used doxygen off and on, /// has also been used at times. Although I don't think we've tried any of the //! or //< forms.

Comment 5

[Gregory Szorc [:gps]]

There are some comments on my blog that are worth reading. http://gregoryszorc.com/blog/2014/12/30/firefox-source-documentation-versus-mdn/

Maybe sheppy and I can find sometime to talk about technical docs future sometime. My Q1 dance card is likely full though :(

Comment 6

[Eric Shepherd [:sheppy]]

Yeah, I'd like to talk about this sometime in Q2 or so if you're available. Ideas have been floating around for a long time to help build a real solution for this, and I think we could make something happen.