1353557 - Intermittent make[5]: *** [force-cargo-library-build] Error 101 after failed to read `/builds/slave/try-m64-d-00000000000000000000/build/src/third_party/rust/error-chain/Cargo.toml`

Status: RESOLVED DUPLICATE of bug 1342292

(Firefox Build System :: General, defect)

Description

[Mike Hommey [:glandium]]

AFAICT, this only happens on OSX debug builds on buildbot (not TC). The error looks like the following:

INFO -  error: failed to load source for a dependency on `$crate`
INFO -  Caused by:
INFO -    Unable to update registry https://github.com/rust-lang/crates.io-index
INFO -  Caused by:
INFO -    failed to update replaced source `registry https://github.com/rust-lang/crates.io-index`
INFO -  Caused by:
INFO -    failed to read `/builds/slave/try-m64-d-00000000000000000000/build/src/third_party/rust/error-chain/Cargo.toml`
INFO -  Caused by:
INFO -    No such file or directory (os error 2)
INFO -  make[5]: *** [force-cargo-library-build] Error 101

Where $crate has, so far, been url, gleam and num-traits.

None of those crates actually depend on error-chain. And yes, error-chain is somehow not vendored (that is a separate problem ; will file a separate bug for that), but I don't see why cargo is trying to use it there, it should only be used when building servo, which those builds are *not* building.

Full logs:
https://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-edb46e71942c2d180d93063988d58c2f3e2a18aa/try-macosx64-debug/try-macosx64-debug-bm87-try1-build30396.txt.gz
https://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-630d05d21f125dfd12515c88e6a106c58b69fd87/try-macosx64-debug/try-macosx64-debug-bm87-try1-build30389.txt.gz
https://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-09ff6492f42064c69a62d4bb009acdcab527a052/try-macosx64-debug/try-macosx64-debug-bm87-try1-build30457.txt.gz

Comment 1

[Alex Crichton [:acrichto]]

This may be the same as https://bugzilla.mozilla.org/show_bug.cgi?id=1342292? Is `error-chain` an empty directory perhaps left around from a different build?

Comment 2

[Mike Hommey [:glandium]]

(In reply to Mike Hommey [:glandium] from comment #0)
> error-chain is somehow not vendored (that is a separate problem ; will file a separate bug
> for that)

Actually, it looks like error-chain is not vendored because it's not actually used, just like the things that depend on it.

So the question now is why the hell does cargo thinks it needs it?

Comment 3

[Alex Crichton [:acrichto]]

Cargo currently probes the whole vendor directory when loading crates (building its internal index), so if one's invalid it'll error out

Comment 4

[Mike Hommey [:glandium]]

(In reply to Alex Crichton [:acrichto] from comment #1)
> This may be the same as
> https://bugzilla.mozilla.org/show_bug.cgi?id=1342292? Is `error-chain` an
> empty directory perhaps left around from a different build?

Ah, that seems plausible... turns out bug 1349692 removed a bunch of vendored crates, including error-chain.

which brings us back to what ted was saying on irc yesterday:
21:16 <ted> apparently we don't have servo in the list of things that get vendored in `mach vendor rust` :-/
21:16 <ted> froydnj: ^^
21:16 <ted> the servo vcssync process vendors servo deps, but presumably someone could break it by running `mach vendor rust` and committing

Comment 5

[Mike Hommey [:glandium]]

Waw, that's a lot of vendored crates missing:
third_party/rust/adler32/
third_party/rust/alloc-no-stdlib/
third_party/rust/android_glue/
third_party/rust/antidote/
third_party/rust/arrayvec/
third_party/rust/audio-video-metadata/
third_party/rust/backtrace-sys/
third_party/rust/backtrace/
third_party/rust/base64/
third_party/rust/block/
third_party/rust/blurdroid/
third_party/rust/blurmock/
third_party/rust/blurz/
third_party/rust/brotli/
third_party/rust/bytes/
third_party/rust/caseless/
third_party/rust/cgl/
third_party/rust/cmake/
third_party/rust/cocoa/
third_party/rust/color_quant/
third_party/rust/compiletest_rs/
third_party/rust/cookie/
third_party/rust/dbghelp-sys/
third_party/rust/dbus/
third_party/rust/debug_unreachable/
third_party/rust/deflate/
third_party/rust/dwmapi-sys/
third_party/rust/encoding-index-japanese/
third_party/rust/encoding-index-korean/
third_party/rust/encoding-index-simpchinese/
third_party/rust/encoding-index-singlebyte/
third_party/rust/encoding-index-tradchinese/
third_party/rust/encoding/
third_party/rust/encoding_index_tests/
third_party/rust/energy-monitor/
third_party/rust/enum_primitive/
third_party/rust/error-chain/
third_party/rust/expat-sys/
third_party/rust/flate2/
third_party/rust/foreign-types/
third_party/rust/futf/
third_party/rust/gif/
third_party/rust/glx/
third_party/rust/harfbuzz-sys/
third_party/rust/heapsize_derive/
third_party/rust/heartbeats-simple-sys/
third_party/rust/heartbeats-simple/
third_party/rust/html5ever-atoms/
third_party/rust/html5ever/
third_party/rust/httparse/
third_party/rust/hyper-openssl/
third_party/rust/hyper/
third_party/rust/hyper_serde/
third_party/rust/image/
third_party/rust/immeta/
third_party/rust/inflate/
third_party/rust/io-surface/
third_party/rust/iovec/
third_party/rust/ipc-channel/
third_party/rust/jpeg-decoder/
third_party/rust/language-tags/
third_party/rust/lazycell/
third_party/rust/leak/
third_party/rust/leaky-cow/
third_party/rust/lzw/
third_party/rust/mac/
third_party/rust/malloc_buf/
third_party/rust/metadeps/
third_party/rust/mime/
third_party/rust/mime_guess/
third_party/rust/miniz-sys/
third_party/rust/mio/
third_party/rust/miow/
third_party/rust/mp3-metadata/
third_party/rust/mp4parse/
third_party/rust/multistr/
third_party/rust/net2/
third_party/rust/nodrop/
third_party/rust/num-iter/
third_party/rust/num-rational/
third_party/rust/objc/
third_party/rust/odds/
third_party/rust/offscreen_gl_context/
third_party/rust/ogg/
third_party/rust/ogg_metadata/
third_party/rust/open/
third_party/rust/openssl-sys/
third_party/rust/openssl/
third_party/rust/osmesa-sys/
third_party/rust/parse-hosts/
third_party/rust/png/
third_party/rust/ref_filter_map/
third_party/rust/ref_slice/
third_party/rust/rust-webvr/
third_party/rust/rustc-demangle/
third_party/rust/rustc_version/
third_party/rust/scoped_threadpool/
third_party/rust/semver/
third_party/rust/servo-egl/
third_party/rust/servo-fontconfig-sys/
third_party/rust/servo-fontconfig/
third_party/rust/servo-freetype-sys/
third_party/rust/servo-glutin/
third_party/rust/servo-skia/
third_party/rust/servo-websocket/
third_party/rust/sha1/
third_party/rust/shared_library/
third_party/rust/shell32-sys/
third_party/rust/sig/
third_party/rust/simd/
third_party/rust/slab/
third_party/rust/string_cache/
third_party/rust/string_cache_codegen/
third_party/rust/string_cache_shared/
third_party/rust/synstructure/
third_party/rust/tendril/
third_party/rust/tinyfiledialogs/
third_party/rust/toml/
third_party/rust/traitobject/
third_party/rust/truetype/
third_party/rust/typeable/
third_party/rust/unicase/
third_party/rust/unicode-script/
third_party/rust/url_serde/
third_party/rust/user32-sys/
third_party/rust/utf-8/
third_party/rust/webdriver/
third_party/rust/ws/
third_party/rust/ws2_32-sys/
third_party/rust/x11-dl/
third_party/rust/x11/
third_party/rust/xi-unicode/
third_party/rust/xml5ever/

(that's what mach vendor rust added after I added servo to its list of things)

Comment 6

[Mike Hommey [:glandium]]

From irc:
08:03 <glandium> ... and mach vendor rust doesn't run vendor for servo
08:04 <gps> by design. otherwise it pulls in a crap ton of dependencies we don't yet want

Comment 7

[Mike Hommey [:glandium]]

So, the problem is that before bug 1349692, webrender *did* have a dependency on error-chain, and it lost it, which is why it was removed. So in the end, this is just a dupe of bug 1342292, but it happened on automation this time.