1109804 - Parse IDL

Status: RESOLVED FIXED

(Webtools Graveyard :: DXR, defect)

Description

[Erik Rose [:erik][:erikrose]]

There's even a Python, PLY-based parser in mozilla-central: http://dxr.mozilla.org/mozilla-central/source/xpcom/idl-parser/xpidl.py

Comment 1

[Erik Rose [:erik][:erikrose]]

Note that there are a bunch of interface definition langs in the codebase: xpidl (for XPCOM), webidl (for newer DOM stuff), ipdl (for IPC), and MS IDL.

Comment 2

[Erik Rose [:erik][:erikrose]]

MXR never supported WebIDL nor ipdl.

Comment 3

[Joshua Cranmer [:jcranmer]]

xpidl, webidl, and MS IDL are all variants on IDL; IPDL is completely different.

The core IDL language could be roughly compared to a variant of C++ consisting of only declarative elements, but there are semantic differences. For XPIDL in particular:
1. In method argument lists, each argument is preceded by an in, out, or inout keyword.
2. Interfaces, typedefs, arguments, functions, attributes, etc. can be preceded with attribute declarations: these are a space-separated list of terms contained in brackets that precede their modifying declaration. (Compare with C++11, where attributes are in double brackets). Some terms have arguments, e.g., size_is(arg).
3. XPIDL allows %{C++ %} syntax, which includes the text in the generated .h header file. Usually, these contain C++ defines, but non-virtual functions within interfaces are sometimes generated.
4. Attributes are declared as readonly attribute long bar; or attribute long bar;

MSIDL notably differs from XPIDL in that the in keywords are specified only via attributes (e.g., HRESULT foo([in] bar) in MS IDL).

Comment 4

[Erik Rose [:erik][:erikrose]]

dmajor tells me that bz is most familiar with IDL name mangling.

Comment 5

[Peter Elmers [:new_one]]

I'd like to take this bug, beginning with xpidl specifically. I've been doing some poking around and will leave some notes here.

As Erik mentions, xpidl.py does the parsing of IDL files. Calling the IDLParser() constructor invokes PLY to create lex and yacc files if necessary.

header.py (https://dxr.mozilla.org/mozilla-central/source/xpcom/idl-parser/header.py) logics the AST into a C++ header; if we hook in at this point we would be able to build refs from the IDL to the generated source.

typelib.py generates binary XPT files which are used by XPConnect. Since they're binary, DXR wouldn't be too interested.

More to come as I continue digging.

Comment 6

[Peter Elmers [:new_one]]

Updates:
In mozilla-central, xpcom/idl-parser (https://dxr.allizom.org/mozilla-central/source/xpcom/idl-parser) is now a Python package, which I have mirrored at https://github.com/pelmers/xpidl.

My pull request, https://github.com/pelmers/xpidl, imports the xpidl package and walks the AST of idl files to yield refs and regions for IDL constructs. I also borrow header.py, which generates the C++ headers, to include jumps from IDL code to its generated C++. I copy over the header writing code, replacing the actual file writes with line increments. http://git.io/vm2RQ
An index of mozilla-central with 4 workers under CPython and only the xpidl plugin activated (meaning no syntax highlighting, no C++ plugin) takes about 35 minutes to complete.

Comment 7

[Peter Elmers [:new_one]]

Corrected link to my pull request: https://github.com/mozilla/dxr/pull/456

Comment 8

[[github robot]]

Commit pushed to master at https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/commit/a7b6a86065f91a98e9b2f86118c5fe24dd193e0c
Write plugin to support XPIDL files.

The xpidl plugin borrows the xpidl.py parser from mozilla-central to
attach structural data and refs to Cross-Platform IDL files, used in the
Mozilla codebase.
The plugin implements jumps to generated header code, search by
interface, method, const, and attribute declarations, and search by
interface subclassing.
The plugin accepts two options: header_bucket to control where C++
headers are dumped include_folders to define the search path for IDL
include statements.

Fixes bug 1109804.