1768354 - Implement "compile-results" searchfox-tool command to approximate router.py's SearchResults' compile_result and sort_compiled logic

Status: RESOLVED FIXED

(Webtools :: Searchfox, enhancement)

Description

[Andrew Sutherland [:asuth] (he/him)]

Part of searchfox's secret sauce is the SearchResults mechanism that flattens semantic crossref-lookup results, prioritizes them, and integrates text results but suppressing text results that have already been provided by crossref-lookup semantic results. This needs to be ported for the new "query" mechanism and will be treated as a junction command that will pull from multiple pipelines; one from search-files, one from search-identifiers that has been piped through crossref-lookup, and one from search-text.

For the purposes of performing traversal of the "overrides", "overriddenBy", "supers", and "subclasses" edges from symbol "meta" information, I am planning to do that explicitly as a new pipeline command expand-results which would go in the "semantic-search" pipeline group after crossref-lookup, as it would primarily be concerned about expanding the set of semantic results and this arguably should be decoupled from the presentation logic that is compile-results. That said, I did shoe-horn the expansion logic into router.py inside the "expand_keys" method, but that was primarily due to wanting to minimally disrupt the existing logic and because there was a time-pressure due to the override regressions that weren't fully intentional.

expand-results will likely end up with its own bug to be implemented in the future unless it turns out the compilation logic structurally wants to understand how the "this result is here because of BLAH" and potential faceting of a unioned results group for overrides/etc. would work. (Right now router.py makes us aware of overrides and has an "upsearch" mechanism to let you search on that, but it's almost aggressively unhelpful in the case of things like nsIInputStream::Close where there's just a boatload of overrides.)

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Attachment: Landed PR: compile-results and crossref-expand and augment-results