1636068 - ./mach vendor rust broken by problems with spirv_cross

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P3)

Description

[Lars T Hansen [:lth]]

Current mozilla-central, clean:

[lhansen@yojimbo m-c]$ ./mach vendor rust
error: failed to resolve patches for `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to load source for a dependency on `spirv_cross`

Caused by:
  Unable to update https://github.com/kvark/spirv_cross?branch=wgpu2#f252537b

Caused by:
  failed to update submodule `spirv_cross/src/vendor/SPIRV-Cross`

Caused by:
  object not found - no match for id (76b9ab1d96db3b4da2875862f263ffa389ad5ecf); class=Odb (9); code=NotFound (-3)
Error running mach:

    ['vendor', 'rust']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

subprocess.CalledProcessError: Command '['/home/lhansen/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo', 'update', '-p', 'gkrust']' returned non-zero exit status 101.

  File "/home/lhansen/m-c/python/mozbuild/mozbuild/mach_commands.py", line 1337, in vendor_rust
    vendor_command.vendor(**kwargs)
  File "/home/lhansen/m-c/python/mozbuild/mozbuild/vendor_rust.py", line 365, in vendor
    subprocess.check_call([cargo, 'update', '-p', 'gkrust'], cwd=self.topsrcdir)
  File "/usr/lib64/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)

Comment 1

[Benjamin Bouvier [:bbouvier] (inactive)]

A new variant of bug 1629891. I wonder if there's a way to make sure that it doesn't happen again in the future, by automated checks in the mach vendor rust process.

Comment 2

[Mike Hommey [:glandium]]

There is no automated check the mach vendor rust process can do to avoid this, because it involves a commit disappearing from github. Okay, it's still there if you browse through the repo to the submodule, but it's not reachable from the submodule refs anymore.

Well, actually, there is an automated check we could do: forbid any repo and submodule that is not under github.com/mozilla or something. So that commits don't disappear from under us.

Comment 3

[Mike Hommey [:glandium]]

$ git clone https://github.com/grovesNL/SPIRV-Cross
Cloning into 'SPIRV-Cross'...
remote: Enumerating objects: 2, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 24252 (delta 0), reused 1 (delta 0), pack-reused 24250
Receiving objects: 100% (24252/24252), 10.58 MiB | 4.06 MiB/s, done.
Resolving deltas: 100% (16801/16801), done.
Checking out files: 100% (2635/2635), done.
$ cd SPIRV-Cross/
$ git show 76b9ab1d --
fatal: bad revision '76b9ab1d'

Interestingly this are different through the submodule:

$ git clone -n --recurse-submodules https://github.com/kvark/spirv_cross
Cloning into 'spirv_cross'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 1876 (delta 8), reused 22 (delta 5), pack-reused 1848
Receiving objects: 100% (1876/1876), 1.18 MiB | 1.50 MiB/s, done.
Resolving deltas: 100% (1037/1037), done.
$ cd spirv_cross
$ git checkout f252537b
Note: checking out 'f252537b'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at f252537 Wrap the rlib inside a gecko layer
$ git submodule update
Cloning into '/home/glandium/spirv_cross/src/vendor/SPIRV-Cross'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 2 (delta 1), reused 1 (delta 1), pack-reused 1
Unpacking objects: 100% (2/2), done.
From https://github.com/grovesNL/SPIRV-Cross
 * branch              76b9ab1d96db3b4da2875862f263ffa389ad5ecf -> FETCH_HEAD
Submodule path 'spirv_cross/vendor/SPIRV-Cross': checked out '76b9ab1d96db3b4da2875862f263ffa389ad5ecf'

This suggests git might be doing something special here. But cargo is not using git, it's using libgit2, which may not be doing the same thing.

Comment 4

[Benjamin Bouvier [:bbouvier] (inactive)]

Yes, the submodule issue very much sounds like the same as the previous issue I ran into.

Well, actually, there is an automated check we could do: forbid any repo and submodule that is not under github.com/mozilla or something. So that commits don't disappear from under us.

Not sure that would prevent it; people could still force-push on a mozilla-owned Github repository (which was partially the cause of the issue I ran into the first time; mach vendor rust had been run with a commit that disappeared after a force push on the Github repo it was linked from), and/or use external Git submodules that get out of sync (as seems to be the case here, and was the case before too).

Comment 5

[Dzmitry Malyshau [:kvark]]

Nothing has changed with regards to the Cargo.toml override or the target branch in the last few weeks. I'm also not able to reproduce this locally, even after removing everything from "~/.cargo/registry/cache/github.com-1ecc6299db9ec823/spirv*".

Comment 6

[Mike Hommey [:glandium]]

Yeah, there's also something happening server-side on github that seems to make things not deterministic. We've had intermittent failures on automation after a revision disappeared on something else, rather than a consistent permanent failure.

Comment 7

[Glenn Watson [:gw]]

I'm seeing this locally when trying to re-vendor locally to enable the profiling library in WR - happy to help debug it if there's anything I can do from here.

Is there a known workaround to be able to vendor rust crates at the moment?

Comment 8

[Benjamin Bouvier [:bbouvier] (inactive)]

Does it reproduce if you remove the local Github cache entry in the directory whose name looks like ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/spirv*?

Comment 9

[Dzmitry Malyshau [:kvark]]

I just hit that myself upon doing cargo update -p gleam, which confirms to me that this is a cargo/git issue.
Mike:

fatal: bad revision '76b9ab1d'

This revision is not visible from master. However, git fetch origin 76b9ab1d96db3b4da2875862f263ffa389ad5ecf works.
What happened is that the "minimal" branch of https://github.com/grovesNL/SPIRV-Cross/compare/master...grovesNL:minimal has moved, so the old commit got stale.
I'll try to update the vendored pointer and/or make sure we keep the branch around.

Comment 10

[Dzmitry Malyshau [:kvark]]

Attachment: Update spirv-cross to 0.20.0

This change updates spirv-cross to a new version.
Unlike the previous one, it now points to a SPIRV-Cross submodule in
https://github.com/kvark/SPIRV-Cross behind "gecko3" branch, which will guarantee the rev
to be accessible.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/faf6d1c5c9e7
Update spirv-cross to 0.20.0 r=groves

Comment 12

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/faf6d1c5c9e7