1781178 - Improve Searchfox's ability to resolve symbol definitions in templated C++ code

Assignee

Description

•

2 years ago

Providing accurate code navigation in templated C++ code is a hard problem. Many symbol names in templated code are "dependent names" (meaning, they depend in some way on a template parameter), which the compiler only resolves to a symbol definition during instantiation -- but we read and write and want to navigate uninstantiated code.

Tools typically employ heuristics to provide navigation features in templated code, and Searchfox has some good ones already.

As someone who also uses clangd to navigate and edit mozilla-central locally, I've noticed that there are some cases that clangd's heuristics handle but Searchfox's currently do not.

I'd like to port over the missing pieces of clangd's template heuristics (many of which I implemented in clangd) to Searchfox. Since clangd and Searchfox are both based on clang's C++ API, Searchfox should in principle be able to implement anything that clangd does.

Here are some examples of symbol names in templated code which Searchfox currently does not resolve to a symbol definition but clangd does:

Function/method names
- ProjectPoint() here
- HasPositiveWCoord() here
- FromUnknownMatrix() and ToUnknownMatrix() here
- (Other similar examples in our Matrix library, these likely have a small set of common root causes.)
- TransformPoint() here
  - Here, clangd's current behaviour isn't ideal either (it offers all the overloads of the target function, not just the relevant one), but better than nothing
- Cases using the template disambiguator, such as downcast here
- format here (maybe related to the use of a union?)
Type names
- Identity here (I filed bug 1621789 about this earlier)
- const_iterator here
- ArgType here

(Note 1: I provided permalinks to protect against code rot, but testing Searchfox's navigation behaviour requires clicking "Go to latest revision" first, since only the latest revision is indexed semantically.)

(Note 2: There are also cases where Searchfox is able to resolve a definition that clangd doesn't! In some cases, this is due to known limitations of clangd, such as only indexing code on one platform at a time. In other cases, I imagine Searchfox knows some tricks that clangd doesn't, which I'm hoping to learn about and port back over to clangd in turn :))

In this bug, I plan to focus on Searchfox's "go to definition" feature. "Find references" in templated code could also use improvement (e.g. bug 1535149 and related), but I suspect that's less a matter of porting over heuristics from clangd (which unfortunately also doesn't handle this very well), and more a matter of getting Searchfox to index template instantiations.

Over the course of this work, I would also like to keep an eye out for possible opportunities to "upstream" code implementing relevant heuristics to a place that both clangd and Searchfox can reuse (e.g. libclangAST/libclangSema).