1787667 - build-blame failure on rust repo in config4 indexer, probably on git submodule removal

Status: RESOLVED FIXED

(Webtools :: Searchfox, defect)

Description

[Andrew Sutherland [:asuth] (he/him)]

The config4 indexer had the following warning today (which previously was fatal but because of the enhancement to allow trees to be marked as "continue" is just a warning):

[2022-08-28T12:13:27Z INFO  build_blame] Transforming 4a7e2fbb7bfdb7ddcb65b6fcf23bb5b272e66d42 (hg None) progress 38/40
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { code: -3, klass: 9, message: "object not found - no match for id (4d8b0a19986a4daab37287a5b5fe2da0775d1873)" }', src/bin/build-blame.rs:487:60
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The unwrap is throwing at https://github.com/mozsearch/mozsearch/blob/e5de767b4d47bd40cf12596943c15220bd09100f/tools/src/bin/build-blame.rs#L487 and the mentioned hash of 4d8b0a19986a4daab37287a5b5fe2da0775d1873 seems to correspond to the removal of the rls submodule in this commit https://github.com/rust-lang-ci/rust/commit/d5ef528bebbe0d8e350e547316fdc7d7476614d2#diff-2314f0a87cd168fcb68d8e73307bdb44c7278bcf0e8aefbe7f2ff7b04e61e41a where the linked-to "src/tools/rls" line says "Submodule rls deleted from 4d8b0a"

Note that this differs from what we saw in blinkyum's failure to blame index at https://bugzilla.mozilla.org/show_bug.cgi?id=1782285#c2 where the unwrap was on a None at https://github.com/mozsearch/mozsearch/blob/e5de767b4d47bd40cf12596943c15220bd09100f/tools/src/bin/build-blame.rs#L418 and we had the contextual error message apparently from git-fast-import:

fatal: Not a commit (actually a tree): M 160000 4b825dc642cb6eb9a060e54bf8d69288fbee4904 native_client_sdk/src/site_scons
fast-import: dumping crash report to fast_import_crash_27936

This failure is likely to be permanent in nature.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

From reading the code and trying to remember what it does, I think this error occurs if a submodule is replaced by a regular directory, which appears to be the case here. In such a scenario, the e variable on the offending line would be a TreeEntry with kind Commit, and the oid on it would be a commit hash from the submodule. So attempting to call to_object(git_repo) fails, because it can't find the hash in the superproject (which is what git_repo is).

In terms of fixing the panic, I think just replacing the unwrap() with an ok() should be sufficient. This will let the parent_subtree become None and the code should treat it as though a new directory was created. I'd try this out myself but it's been so long since I've built searchfox now that I need to upgrade/install a bunch of stuff before I can and it'll take some time.

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Aha! Thank you!! I'd missed that rls had become a directory in a commit (although now on reading your comment, someone I follow on twitter had been mentioning the bad UX experience of the submodule being replaced by a regular file...) and generally have a very limited understanding of the blame code at this time (especially as it relates to submodules and the specialized logic in the block above the block in issue).

I'll do a mozsearch try "dev" channel run on the rust repo with such a fix and land it if it works. Thanks!

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

Attachment: Landed PR: Avoid getting confused by removed submodules that are now dirs

The fix proposed by :kats in comment 1 successfully ran on the dev channel for rust.