1015411 - Move filters to language plugins

Status: RESOLVED FIXED

(Webtools Graveyard :: DXR, defect)

Description

[Erik Rose [:erik][:erikrose]]

For modularity's (and thus comprehensibility's) sake, move language-specific filters to plugins. Here's a partially thought-out strawman:

1. The query parser breaks the query into terms, each with a filter type (like "var-decl:" or "function:").
2. Through plugin registration, each filter type points to a list of implementors, typically 0 or 1 for each language.
3. Each plugin registered for the filter type of a term gets a chance to contribute to its part of the query.
4. The contributions get ORed together. Thus the part of the query that handles one term is completed.

Some filter types will result in the same mapping fields and ES query patterns across several languages. For example, we might imagine the "function:" filter being like this. These fields should be promoted into core and given well-defined formats and semantics. The filter type registrations for those fields should live in core, and the language plugins that would otherwise need to deal with them should simply do nothing. In this way, we factor up common code *and* avoid adding repeated clauses to the query.

Still to be determined is how to handle the "lang:" filter, which should limit the language plugins considered. We could pass it to the plugins, register plugins not against filter types but against filter-type/language pairs, or give plugins access to the whole parsed DXR query and let them figure it out themselves.

Comment 1

[Erik Rose [:erik][:erikrose]]

Incidentally, because there's not a lot of overlap between people who know ES and people who know compilers, we should probably have a very thing veneer layer between the plugins and the generation of raw ES queries: maybe something as simple as a prefix_match('some phrase') function (and and few others) the plugins can call. And then, of course, they can always fall back to emitting raw ES if they have special needs. I don't think it's worth being more abstract than that, since (1) we'll be on ES for the forseeable future, (2) the preceding is sufficiently abstract for anything of similar data shape, and (3) any drastic change in data shape will almost inevitably change everything else as well.

Comment 2

[Erik Rose [:erik][:erikrose]]

Correction: filter types that result in the same mapping fields and ES query patterns across several languages should *not* be plonked into the same field. That would make it impossible to do queries like `lang:C function:frob`. Instead, we should have separate function-c and function-rust fields (etc.), and the query `function:frob` should perform an OR. (It wouldn't suffice to label an entire file as being of a certain language, especially with the HTML and JS and CSS that go on around here. It's less bad to label an entire line as a language, but there's still room for error with things like PHP vs. JS.)

Comment 3

[Erik Rose [:erik][:erikrose]]

This has been done on the es branch for months. It's what the new plugin architecture ended up looking like.