1878575 - Perma rust vendor error: the lock file /builds/worker/checkouts/gecko/Cargo.lock needs to be updated but --frozen was passed to prevent this

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Cristian Tuns]

Push with failures:https://treeherder.mozilla.org/jobs?repo=mozilla-central&group_state=expanded&selectedTaskRun=JfLRQse-TyWJn-GBI1JOyg.0&searchStr=linting%2Copt%2Crust%2Cchecks%2Csource-test-vendor-rust%2Cvendor&revision=6cb794603d85d30882e94931d5c7cba2c609e8dd
Failure log: https://treeherder.mozilla.org/logviewer?job_id=445891027&repo=mozilla-central

[fetches 2024-02-04T21:48:35.586Z] fetching artifacts
[fetches 2024-02-04T21:48:35.588Z] executing ['/usr/bin/python3', '-u', '/builds/worker/checkouts/gecko/taskcluster/scripts/misc/fetch-content', 'task-artifacts']
[fetches 2024-02-04T21:48:35.651Z] attempt 1/5
[fetches 2024-02-04T21:48:35.651Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/d85fdQnAQoKwQTHHEE1RYg/artifacts/public/build/cargo-vet.tar.zst to /builds/worker/fetches/cargo-vet.tar.zst
[fetches 2024-02-04T21:48:35.652Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/d85fdQnAQoKwQTHHEE1RYg/artifacts/public/build/cargo-vet.tar.zstattempt 1/5
[fetches 2024-02-04T21:48:35.652Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YR5_sIDhQ3C80vGYQAGxnw/artifacts/public/build/python.tar.zst to /builds/worker/fetches/python.tar.zst
[fetches 2024-02-04T21:48:35.652Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YR5_sIDhQ3C80vGYQAGxnw/artifacts/public/build/python.tar.zst
[fetches 2024-02-04T21:48:35.652Z] 
[fetches 2024-02-04T21:48:35.653Z] attempt 1/5
[fetches 2024-02-04T21:48:35.653Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Imy0h0MWTcuwuETQWwdyGw/artifacts/public/build/rustc.tar.zst to /builds/worker/fetches/rustc.tar.zst
[fetches 2024-02-04T21:48:35.654Z] Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Imy0h0MWTcuwuETQWwdyGw/artifacts/public/build/rustc.tar.zst
[fetches 2024-02-04T21:48:36.035Z] https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YR5_sIDhQ3C80vGYQAGxnw/artifacts/public/build/python.tar.zst resolved to 67684749 bytes with sha256 1da777f18c3e6df2eb205efa0bfc639c0277ab6a793d2264224ba4271a1360c2 in 0.383s
[fetches 2024-02-04T21:48:36.035Z] Extracting /builds/worker/fetches/python.tar.zst to /builds/worker/fetches
[fetches 2024-02-04T21:48:36.162Z] https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/d85fdQnAQoKwQTHHEE1RYg/artifacts/public/build/cargo-vet.tar.zst resolved to 7465447 bytes with sha256 5894c82d09d069be290fc84fb783a2af8f9c90f58319073aa39b05252664944b in 0.510s
[fetches 2024-02-04T21:48:36.163Z] Extracting /builds/worker/fetches/cargo-vet.tar.zst to /builds/worker/fetches
[fetches 2024-02-04T21:48:36.218Z] /builds/worker/fetches/cargo-vet.tar.zst extracted in 0.055s
[fetches 2024-02-04T21:48:36.218Z] Removing /builds/worker/fetches/cargo-vet.tar.zst
[fetches 2024-02-04T21:48:36.834Z] /builds/worker/fetches/python.tar.zst extracted in 0.799s
[fetches 2024-02-04T21:48:36.834Z] Removing /builds/worker/fetches/python.tar.zst
[fetches 2024-02-04T21:49:18.843Z] https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Imy0h0MWTcuwuETQWwdyGw/artifacts/public/build/rustc.tar.zst resolved to 303722810 bytes with sha256 bc1e6c1d3cea6b6f6388694950c8132b69cc3815d19af689d7ec0b530c3d63da in 43.190s
[fetches 2024-02-04T21:49:18.843Z] Extracting /builds/worker/fetches/rustc.tar.zst to /builds/worker/fetches
[fetches 2024-02-04T21:49:20.919Z] /builds/worker/fetches/rustc.tar.zst extracted in 2.075s
[fetches 2024-02-04T21:49:20.919Z] Removing /builds/worker/fetches/rustc.tar.zst
[fetches 2024-02-04T21:49:20.949Z] PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 45.299457114000006, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-04T21:49:20.960Z] finished fetching artifacts
[setup 2024-02-04T21:49:20.960Z] Setting up local python environment
[setup 2024-02-04T21:49:20.960Z] updated PATH with python artifact: /builds/worker/fetches/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[task 2024-02-04T21:49:20.960Z] executing ['bash', '-cx', './mach vendor rust --issues-json /builds/worker/issues.json']in /builds/worker/checkouts/gecko
[task 2024-02-04T21:49:20.964Z] + ./mach vendor rust --issues-json /builds/worker/issues.json
[task 2024-02-04T21:49:25.957Z]     Updating git repository `https://github.com/hsivonen/chardetng`
[task 2024-02-04T21:49:26.452Z]     Updating git repository `https://github.com/hsivonen/chardetng_c`
[task 2024-02-04T21:49:26.813Z]     Updating git repository `https://github.com/chris-zen/coremidi.git`
[task 2024-02-04T21:49:27.201Z]     Updating git repository `https://github.com/franziskuskiefer/cose-rust`
[task 2024-02-04T21:49:27.582Z]     Updating git repository `https://github.com/servo/rust-cssparser`
[task 2024-02-04T21:49:28.233Z]     Updating git repository `https://github.com/mozilla/application-services`
[task 2024-02-04T21:49:32.983Z]     Updating git submodule `https://github.com/mozilla/glean`
[task 2024-02-04T21:49:35.299Z]     Updating git repository `https://github.com/mozilla/midir.git`
[task 2024-02-04T21:49:35.758Z]     Updating git repository `https://github.com/glandium/mio`
[task 2024-02-04T21:49:36.487Z]     Updating git repository `https://github.com/seanmonstar/warp`
[task 2024-02-04T21:49:36.965Z]     Updating crates.io index
[task 2024-02-04T21:49:37.174Z]     Updating crates.io index
[task 2024-02-04T21:49:37.174Z]     Updating git repository `https://github.com/mozilla-spidermonkey/jsparagus`
[task 2024-02-04T21:49:45.225Z]     Updating git repository `https://github.com/mozilla/neqo`
[task 2024-02-04T21:49:47.211Z]     Updating git repository `https://github.com/FirefoxGraphics/aa-stroke`
[task 2024-02-04T21:49:47.855Z]     Updating git repository `https://github.com/mozilla/audioipc`
[task 2024-02-04T21:49:50.598Z]     Updating git repository `https://github.com/mozilla/cubeb-coreaudio-rs`
[task 2024-02-04T21:49:51.647Z]     Updating git repository `https://github.com/mozilla/cubeb-pulse-rs`
[task 2024-02-04T21:49:52.331Z]     Updating git repository `https://github.com/jfkthame/mapped_hyph.git`
[task 2024-02-04T21:49:52.759Z]     Updating git repository `https://github.com/mozilla/mp4parse-rust`
[task 2024-02-04T21:49:53.711Z]     Updating git submodule `https://github.com/AOMediaCodec/av1-avif.git`
[task 2024-02-04T21:50:15.092Z]     Updating git submodule `https://github.com/link-u/avif-sample-images`
[task 2024-02-04T21:50:18.490Z]     Updating git repository `https://github.com/mozilla/uniffi-rs.git`
[task 2024-02-04T21:50:19.690Z]     Updating git repository `https://github.com/FirefoxGraphics/wpf-gpu-raster`
[task 2024-02-04T21:50:20.303Z]     Updating git repository `https://github.com/cloudflare/quiche`
[task 2024-02-04T21:50:22.655Z]     Updating git submodule `https://github.com/google/boringssl.git`
[task 2024-02-04T21:50:48.106Z]     Updating git repository `https://github.com/gfx-rs/wgpu`
[task 2024-02-04T21:50:56.804Z]       Adding bindgen v0.63.999 (/builds/worker/checkouts/gecko/build/rust/bindgen-0.63)
[task 2024-02-04T21:50:56.804Z]     Removing bindgen v0.63.999 (/builds/worker/checkouts/gecko/build/rust/bindgen-0.64)
[task 2024-02-04T21:50:56.804Z]     Updating bindgen v0.69.2 -> v0.69.4
[task 2024-02-04T21:50:56.804Z]       Adding memmap2 v0.8.999 (/builds/worker/checkouts/gecko/build/rust/memmap2)
[task 2024-02-04T21:50:56.804Z]     Removing memmap2 v0.8.999 (/builds/worker/checkouts/gecko/build/rust/memmap2-0.5)
[task 2024-02-04T21:50:56.804Z]     Removing peeking_take_while v0.1.2
[task 2024-02-04T21:50:56.804Z]       Adding windows-targets v0.52.999 (/builds/worker/checkouts/gecko/build/rust/windows-targets)
[task 2024-02-04T21:50:56.804Z]     Removing windows-targets v0.52.999 (/builds/worker/checkouts/gecko/build/rust/windows-targets-0.48)
[task 2024-02-04T21:50:57.073Z] Loading graph configuration.
[task 2024-02-04T21:50:57.712Z] Using Parameters(id=9d27b9b38c0b) (from defaults)
[task 2024-02-04T21:50:57.712Z] Loading kinds
[task 2024-02-04T21:50:57.712Z] Limiting kinds to following kinds and dependencies: toolchain
[task 2024-02-04T21:50:57.737Z] Generating full task set
[task 2024-02-04T21:50:57.976Z] Generated 102 tasks for kind fetch
[task 2024-02-04T21:50:58.006Z] Generated 17 tasks for kind packages
[task 2024-02-04T21:50:59.266Z] Generated 223 tasks for kind toolchain
[task 2024-02-04T21:50:59.413Z] Generated 54 tasks for kind docker-image
[task 2024-02-04T21:51:01.211Z] Vet error: `cargo metadata` exited with an error:
[task 2024-02-04T21:51:01.211Z] error: the lock file /builds/worker/checkouts/gecko/Cargo.lock needs to be updated but --frozen was passed to prevent this
[task 2024-02-04T21:51:01.211Z] If you want to try to generate the lock file without accessing the network, remove the --frozen flag and use --offline instead.
[task 2024-02-04T21:51:01.211Z] 
[task 2024-02-04T21:51:01.211Z]  help: You may need to run `cargo generate-lockfile` to create a Cargo.lock
[task 2024-02-04T21:51:01.211Z] Run `./mach cargo vet` for more information.
[task 2024-02-04T21:51:01.212Z] Creating default state directory: /builds/worker/.mozbuild
[task 2024-02-04T21:51:01.212Z] Running "pip check" to verify compatibility between the system Python and the "mach" site.
[task 2024-02-04T21:51:01.212Z] Creating local state directory: /builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69
[task 2024-02-04T21:51:01.212Z] Running "pip check" to verify compatibility between the system Python and the "common" site.
[task 2024-02-04T21:51:01.212Z] Errors occured; new rust crates were not vendored.
[taskcluster 2024-02-04 21:51:02.996Z] === Task Finished ===
[taskcluster 2024-02-04 21:51:03.330Z] Unsuccessful task run with exit code: 1 completed in 398.578 seconds

Comment 1

[Emilio Cobos Álvarez (:emilio)]

So the issue is this:

[task 2024-02-04T21:50:56.804Z] Updating bindgen v0.69.2 -> v0.69.4

So this was "caused" by me publishing a newer bindgen version. So, the good news is bug 1878536 would fix it and already has a patch written.

But then the question then is, why is cargo vendor updating a dependency? Mike, do you happen to have stumbled across this in the past by any chance? I don't see anything special with bindgen that would explain this afaict.

Comment 3

[Intermittent Failures Robot]

10 failures in 3952 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5
• autoland: 5

Platform and build breakdown:

• lint: 10
  • opt: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878575&startday=2024-01-29&endday=2024-02-04&tree=all

Comment 4

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1878575 - Remove build/rust/bindgen-0.64. r=nical!

The only use of it is nss-gk-api which can work with 0.69 as per its
dependency definition.

It seems cargo is removing this package, then updating bindgen (even
though it shouldn't), but this works around the issue and is harmless.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d67e058935ba
Remove build/rust/bindgen-0.64. r=nical

Comment 6

[Emilio Cobos Álvarez (:emilio)]

I think this is likely a bug in cargo, afaict

Comment 7

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/d67e058935ba

Comment 8

[Mike Hommey [:glandium]]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #6)

I think this is likely a bug in cargo, afaict

Definitely a cargo bug. Do you want to file it?

Comment 9

[Sandor Molnar[:smolnar]]

Still happening

Comment 10

[Mike Hommey [:glandium]]

The nightly snap is fine, so this is actually bug 1878584, which I unduped.

Comment 11

[Intermittent Failures Robot]

6 failures in 4009 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 6

Platform and build breakdown:

• lint: 3
  • opt: 3
• linux64-snap-amd64-beta: 3
  • debug: 2
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878575&startday=2024-02-05&endday=2024-02-11&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 3125 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• lint: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1878575&startday=2024-02-12&endday=2024-02-18&tree=all