1640723 - Add more indexing checks and make sure we like the check mechanism

Status: RESOLVED FIXED

(Webtools :: Searchfox, enhancement)

Description

[Andrew Sutherland [:asuth] (he/him)]

As part of bug 1626648 in https://github.com/mozsearch/mozsearch/pull/299 and https://github.com/mozsearch/mozsearch-mozilla/pull/87 I added a check mechanism and scripts for the tests repo and mozilla-central:

• https://github.com/mozsearch/mozsearch/blob/master/tests/tests/check
• https://github.com/mozsearch/mozsearch-mozilla/blob/master/mozilla-central/check

The relevant portion of the m-c check script is:

# We are passed in the name of the helper script to invoke to run each check.
# The syntax is:
#   "$@" <searchfox user exposed path> <canonical symbol name as found in data-symbols>
#
# Note that "$@" is used because the command is expected to have curried
# arguments.

### Rust
## First-Party Rust
# libwebrender
"$@" "gfx/webrender_bindings/src/moz2d_renderer.rs" "webrender_bindings::moz2d_renderer::Box2d"

## Third-Party Rust
"$@" "third_party/rust/euclid/src/rect.rs" "euclid::rect::Rect"

## XPIDL Bindings (libxpcom)
"$@" "__GENERATED__/dist/xpcrs/rt/nsIChannel.rs" "xpcom::interfaces::idl::nsIChannel"

I only added rust checks because I both wanted to avoid further scope creep as well as not commit us to this specific check mechanism. But it would make sense for us to add indexing checks for C++ and JS and anything that reliably works right now. I'm assuming we'll add targeted checks for other things as we fix them, like IPDL.

This key things about the checks right now:

1. They are fatal. Any check we add will break indexing if it fails.
2. They are pretty naive. They're just looking for evidence of a symbol existing at all (possibly just as a substring of another symbol).
3. They are looking for specific symbols, so if we pick symbols that experience churn, we'll have to keep fixing the checks.
4. Basic web runtime checks are supported but not hooked up in any way.
   • These can be run on a web server instance via ~/mozsearch/infrastructure/web-server-check.sh ~/config ~/index ~ and will also run file-system checks. So if adding new checks, it probably makes sense to just test them on a server instance. (Like, add a git remote for the mozsearch-mozilla repo and pull your branch and check it out and then run.)

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

This checks mechanism has been superseded because although we liked the mechanism, we did not love it. We did add some more checks!