Closed Bug 1342292 Opened 5 years ago Closed 3 years ago

error: failed to load source for a dependency on '<any>' when there is empty directory into third_party/rust

Categories

(Firefox Build System :: General, defect)

Unspecified
Windows
defect
Not set
normal

Tracking

(firefox54 affected)

RESOLVED FIXED
Tracking Status
firefox54 --- affected

People

(Reporter: m_kato, Unassigned)

References

Details

When there is empty directory into third_party/rust/, I cannot build Gecko due to the following build error.


 0:13.98 Elapsed: 0.40s; From ../../dist/idl: Kept 0 existing; Added/updated 1104; Removed 0 files and 0 directories.
 0:15.87 force-cargo-library-build
 0:16.92 error: failed to load source for a dependency on `parking_lot`
 0:16.92
 0:16.92 Caused by:
 0:16.92   Unable to update registry https://github.com/rust-lang/crates.io-index
 0:16.92
 0:16.92 Caused by:
 0:16.92   failed to update replaced source `registry https://github.com/rust-lang/crates.io-index`
 0:16.92
 0:16.92 Caused by:
 0:16.92   failed to read `C:\Development\hg.mozilla.org\mozilla-inbound\third_party\rust\app_units-0.3.0\Cargo.toml`
 0:16.92
 0:16.92 Caused by:
 0:16.92   隰厄ソス・ョ螢ケ・・ケァ蠕娯螺郢晁シ斐<郢ァ・、郢晢スォ邵コ迹夲スヲ荵昶命邵コ荵晢ス 顔クコ・セ邵コ蟶呻ス鍋クイ・ス(os error 2)
 0:16.93 c:/Development/hg.mozilla.org/mozilla-inbound/config/rules.mk:959: recipe for target 'force-cargo-library-build' failed
 0:16.93 mozmake.EXE[5]: *** [force-cargo-library-build] Error 101


app_units-0.3.0 and app_units-0.3.0/src are empty, but build system tries to use it.

$ ls -al /c/Development/hg.mozilla.org/mozilla-inbound/third_party/rust/app_units-0.3.0/src
total 4
drwxr-xr-x 2 mkato Administrators    0 Feb 24 10:46 .
drwxr-xr-x 3 mkato Administrators 4096 Feb 24 10:46 ..

Of course, I remove this directory, I can build it well.
(In reply to Makoto Kato [:m_kato] from comment #0)
>  0:16.92 Caused by:
>  0:16.92   隰厄ソス・ョ螢ケ・・ケァ蠕娯螺郢晁シ斐<郢ァ・、郢晢スォ邵コ迹夲スヲ荵昶命邵コ荵晢ス 顔クコ・セ邵コ蟶呻ス鍋クイ・ス(os
> error 2)

Is the mojibake from your copy/paste, or is it like that on your console? There might be a separate bug to file for that.
(In reply to Mike Hommey [:glandium] from comment #1)
> (In reply to Makoto Kato [:m_kato] from comment #0)
> >  0:16.92 Caused by:
> >  0:16.92   隰厄ソス・ョ螢ケ・・ケァ蠕娯螺郢晁シ斐<郢ァ・、郢晢スォ邵コ迹夲スヲ荵昶命邵コ荵晢ス 顔クコ・セ邵コ蟶呻ス鍋クイ・ス(os
> > error 2)
> 
> Is the mojibake from your copy/paste, or is it like that on your console?
> There might be a separate bug to file for that.

In command prompt, this corrupted message is shown. (Maybe, error is UTF-8, but console is Shift JIS. This is another bug).   error code is 2, so it shows "file not found".
(In reply to Makoto Kato [:m_kato] from comment #2)
> In command prompt, this corrupted message is shown. (Maybe, error is UTF-8,
> but console is Shift JIS. This is another bug).   error code is 2, so it
> shows "file not found".

It would be worth determining whether that's the mach wrapping that causes this, or if cargo displays that on its own. If the latter, you should file a bug against cargo (which might be a rust libstd bug)
As for the original bug, here's how I finally was able to reproduce:

Checkout a tree before bug 1340838 (at least 2 pushes before, because the tree is busted on the previous push, so up to and including 16f9a685598b).

Create an empty third_party/rust/app_units-0.3.0/src

And then build (shortcut: mach configure; mach build toolkit/library/rust/force-cargo-library-build)

It looks very much like this is purely a cargo vendoring bug, whereby there is:
- an empty $vendor_path/$crate-$version
- a $vendor_path/$crate directory with the right $version

and it decides to pick the first and not try the second. The first probably is there because of some weird issue during the tree update from VCS.

A smaller testcase than doing this with a clone of mozilla-central on the right revision can be done with simple crates with dependencies, like so:

$ cargo new testcase
$ cd testcase
$ echo 'log = "0.3"' >> Cargo.toml
$ cargo build  # build once to create Cargo.lock, that cargo vendor wants
$ cargo vendor

Do what cargo vendor says, then

$ mkdir vendor/log-0.3.6
$ cargo build

This last command fails.
Flags: needinfo?(acrichton)
Thanks for the cc, this is actually a bug in Cargo I believe rather than vendoring itself. Cargo currently assumes that all directories in the vendor dir are valid crate dirs (e.g. those with Cargo.toml), but it looks like that's not the case here!

Were these empty dirs created and/or left around by git? If it's normal operation that creates these then we should probably just ignore empty dirs in Cargo itself.

FWIW I think this is related to these lines in Cargo [1] where we force all directories to have a valid package, but a missing Cargo.toml is an error (e.g. we could just ignore those errors and skip that directory entry)

[1]: https://github.com/rust-lang/cargo/blob/163de44297b052c8c8668f75a51029b6ed8a0416/src/cargo/sources/directory.rs#L84-L86
Flags: needinfo?(acrichton)
If using Windows, when user's current directory is third_party/rust/app_units-0.3.0/src, it isn't removed even if HG tries removing this.  (OSX and Linux remove it correctly...)
Does that end up being a common ocurrence? It'd be a bit of a shame to ignore malformed directories because that may ignore legitimate bugs by accident (causing hard to track down issues), but we could also perhaps just ignore explicitly empty directories (unsure if theres child directories to worry about here)
Duplicate of this bug: 1353557
I've sent a PR to Cargo (https://github.com/rust-lang/cargo/pull/3899) to update its behavior
Product: Core → Firefox Build System

It looks like this fix shipped in cargo a while ago: cargo 0.20 which shipped with Rust 1.19.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.