1657786 - Create searchfox taskcluster mode/variant that can run the C++ indexer only against changed files for try builds / phabricator requests

Status: NEW

(Webtools :: Searchfox, enhancement)

Description

[Andrew Sutherland [:asuth] (he/him)]

It seems like it should be possible to get hg/git to provide the set of changed files relative to the base mozilla-central revision and then to run a build targeting only the given C++ files. This could produce a minimal set of complete analysis data for the given C++ files and any headers.

This could be used to support:

• A basic semantic diff between a given try build/phabricator request and the current mozilla-central tip as exposed by searchfox. (If your patch is based on a 2-week old mozilla-central, you still get diffed against trunk, even though the appropriate searchfox index would be one 2 weeks ago.)
• Searchfox could dynamically render fully symbolicated versions of the page.
  • The rust magic behind output-file.rs can totally happen at runtime in response to an HTTP POST of analysis data and the contents of the file corresponding to the built revision. This is basically what the dynamic revision rendering does, it just knows it doesn't have semantic data available.
    • That said, jumps and other global lookups might potentially want to be ignored under any initial implementation, although the extra memory overhead could certainly be provided for as needed if such a thing caught on.

Note that this is different and much lighter weight than my proposal in bug 1598502. The core idea here would be:

• The taskcluster job gets run on like 1 platform and takes less than 10 minutes and produces a limited amount of data: really just the new analysis files.
• The searchfox server isn't storing any additional state locally, it's just signing up to support a dynamic mode of operation. Any extra legwork would happen on other servers or on the client.
  • For example, a ServiceWorker could provide a unionfs-type view. It downloads the artifact from taskcluster into client-side storage, and then asks searchfox to render the HTML pages for the changed analysis files. All page loads check the cache for these overlays, falling back to the network-state mozilla-central too.

Language-wise, this wouldn't work for rust as easily because the taskcluster indexing builds just produce save-analysis files which require subsequent processing by the rust indexer which is only designed for offline processing. JS could work.