1849632 - some (all?) makecab dependencies are unpinned

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[[e-mail] (:bhearsum)]

We rebuilt cached tasks on esr102 today. This result in a new makecab toolchain task, which failed with:

[task 2023-08-21T15:22:32.111Z]    Compiling flate2 v1.0.27
[task 2023-08-21T15:22:32.112Z]      Running `rustc --crate-name flate2 --edition=2018 /builds/worker/.cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.27/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="any_impl"' --cfg 'feature="miniz_oxide"' --cfg 'feature="rust_backend"' -C metadata=a390ccc70c570c10 -C extra-filename=-a390ccc70c570c10 --out-dir /builds/worker/fetches/rust-makecab/target/release/deps -L dependency=/builds/worker/fetches/rust-makecab/target/release/deps --extern crc32fast=/builds/worker/fetches/rust-makecab/target/release/deps/libcrc32fast-a7f363acb68d8a1f.rmeta --extern miniz_oxide=/builds/worker/fetches/rust-makecab/target/release/deps/libminiz_oxide-9100149f74c33613.rmeta --cap-lints allow`
[task 2023-08-21T15:22:32.170Z] error[E0658]: deriving `Default` on enums is experimental
[task 2023-08-21T15:22:32.170Z]   --> /builds/worker/.cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.27/src/gz/mod.rs:90:17
[task 2023-08-21T15:22:32.170Z]    |
[task 2023-08-21T15:22:32.170Z] 90 | #[derive(Debug, Default)]
[task 2023-08-21T15:22:32.170Z]    |                 ^^^^^^^
[task 2023-08-21T15:22:32.170Z]    |
[task 2023-08-21T15:22:32.170Z]    = note: see issue #86985 <https://github.com/rust-lang/rust/issues/86985> for more information
[task 2023-08-21T15:22:32.170Z]    = note: this error originates in the derive macro `Default` (in Nightly builds, run with -Z macro-backtrace for more info)
[task 2023-08-21T15:22:32.170Z] 
[task 2023-08-21T15:22:32.312Z]      Running `/builds/worker/fetches/rust-makecab/target/release/build/num-traits-fdb639790af2d1fc/build-script-build`
[task 2023-08-21T15:22:32.404Z] For more information about this error, try `rustc --explain E0658`.
[task 2023-08-21T15:22:32.408Z] error: could not compile `flate2` due to previous error
[task 2023-08-21T15:22:32.408Z] 

Upon inspection, I found that despite the most recent makecab fetch and the previously cached makecab fetch tasks using the same revision, they picked up different dependencies. Of note in particular is flate2, which went from 1.0.25 to 1.0.27, resulting in us hitting https://github.com/rust-lang/flate2-rs/issues/370.

For posterity, this is the previously cached toolchain task, and this is the new toolchain task.

Comment 1

[[e-mail] (:bhearsum)]

I guess it's possible this is an issue in other/all of the toolchains that are Rust based...

Comment 2

[[e-mail] (:bhearsum)]

Echo'ing glandium's suggestion from elsewhere:

My suggestion would be to "vendor" a Cargo.lock in the tree for each of them.
and make the build-rust-based-toolchain.sh script complain when there isn't one

Presumably we would create/update this whenever we add or update a fetch task for a Rust project.

Comment 3

[Mike Hommey [:glandium]]

So... I'm testing a patch that will barf when Cargo.lock is not there when using build-rust-based-toolchain.sh, as per that suggestion of mine. But you know what's funny? I actually had submitted a Cargo.lock upstream https://github.com/luser/rust-makecab/commit/0996ca57fdcc838bb9ac7adbf720403b5ee2cb24 but forgot about it.

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1849632 - Update makecab to latest upstream.

Turns out I had submitted a Cargo.lock (and other changes)... 3 years
ago, and they were merged.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1849632 - Reject building rust-based toolchains without a Cargo.lock.

We provide an escape hatch, allowing to provide a Cargo.lock in
taskcluster/scripts/misc.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/586fd157e5a0
Update makecab to latest upstream. r=firefox-build-system-reviewers,ahochheiden
https://hg.mozilla.org/integration/autoland/rev/d8768bf47f8e
Reject building rust-based toolchains without a Cargo.lock. r=firefox-build-system-reviewers,ahochheiden

Comment 7

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/586fd157e5a0
https://hg.mozilla.org/mozilla-central/rev/d8768bf47f8e