1904202 - Linkify WebIDL extended attributes to documents and/or related code in WebIDL.py/Codegen.py

Status: NEW

(Webtools :: Searchfox, enhancement)

Description

[Tooru Fujisawa [:arai]]

bug 1416899 is going to add indexing for WebIDL files, which covers most of the identifiers in WebIDL,
but it doesn't cover extended attributes.

Some extended attributes are part of the spec
https://webidl.spec.whatwg.org/#js-extended-attributes

and some other extended attributes are Gecko-specific extension:
https://firefox-source-docs.mozilla.org/dom/webIdlBindings/index.html#custom-extended-attributes

it would be nice to linkify them to:

• definition of the extended attribute, either in the spec or the document
• corresponding handling in WebIDL.py and Codegen.py

So that it's easier to figure out the effect of the extended attributes, and also find which code implemented them.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

This would be amazing (and definitely something I've dreamed of having)! Do you have any rough plans for how to approach this? I ask because I think this is one of the more exciting areas where searchfox can really transcend traditional code browsers, and also because I am planning to take PTO for the first week of July and dig into first steps for dataflow/taint analysis and that could potentially find synergy for linking into WebIDL.py and Codegen.py[1].

I know for things like this it can be faster to have a real-time discussion on matrix or a video call, so if you want to reply, but not here, feel free to ping via other channels. Also, no need to let any of my possible plans block any experimentation or efforts on your part. A really nice thing about the pragmatic approach of searchfox with its sort of heterogeneous/ad-hoc pipeline is that we can have a bunch of different pipelines that are potentially better suited for different things.

1: In particular, there's some potential to perform slicing based on the various string literals and their comparisons.

Comment 2

[Tooru Fujisawa [:arai]]

Things I was thinking are the following:

For implementation part, either:

• (A) refactor WebIDL.py and Codegen.py to have dedicate struct for the list of supported extended attributes and its effect, and index them as definition
  • e.g. WebIDL.py: add dedicate function/class/mixin for each extended attribute, move the validation and extra operation there (e.g. body of addExtendedAttributes and finish), and index them with the extended attribute name
  • this is cleaner
  • this can regress the build performance, due to extra indirection and complexity (ref. bug 1885914 and bug 1646937)
  • for "WebIDL in SpiderMonkey", making these parts extensible by default is good preparation, given it will use yet another set of custom extended attributes
• (B) add special comment to related parts in WebIDL.py and Codegen.py, similar to [DOMDOC] and [SMDOC] and index them
  • this is less invasive way and won't affect performance
  • the definition may span across multiple place

also, for the documentation part, one of:

• (C) Add special comment to WebIDL.py to point the corresponding spec or document URLs and index them as declaration
  • this will work nicely with (A)
• (D) Parse the spec and the document and index extended attributes appeared there
  • not sure if this is plausible, but at least possible
• (E) Have a list of all supported extended attributes, separated into defined-by-spec and gecko-specific, and simply construct the URLs based on them
  • this also work nicely with (A), as long as the URL is simple enough

and then, for WebIDL files, perform the following:

• Make sure the extended attribute names have correct token position
• Index them as use for each extended attribute

Comment 3

[Kagami Rosylight [:saschanaz] (they/them)]

Just curious, is using WebIDL.py the strict requirement here? What do we want to get from there, and what if we use a separate parser without touching it?

Comment 4

[Tooru Fujisawa [:arai]]

WebIDL.py is used for 2 purposes here:

• (1) to parse the WebIDL files and get AST with token positions, in order to generate analysis records (which is already done by bug 1416899 for other identifiers)
  • for this purpose, WebIDL.py is copied out of the target repository (if it exists), or from mozilla-central (if it doesn't exist)
  • this is repository agnostic
• (2) as a target of indexing, for the extended attribute handling (I had hard time figuring out how the extended attributes are implemented. having a link from the WebIDL file should make it easier to understand the underlying implementation)
  • for this purpose, we need to index the actual WebIDL.py file in the repository

for (1), WebIDL.py is not strict requirement, but adding yet another parser won't worth I think, unless there's already a compatible parser.
for (2), WebIDL.py is strict requirement, given that's the file where each extended attribute is implemented.