Closed Bug 1579761 Opened 1 year ago Closed 11 months ago

ERROR: Couldn't execute `cargo metadata` with manifest "/builds/worker/workspace/build/src/toolkit/library/rust/Cargo.toml"

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox69 unaffected, firefox70 fixed, firefox71 fixed, firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox69 --- unaffected
firefox70 --- fixed
firefox71 --- fixed
firefox72 --- fixed

People

(Reporter: jbeich, Assigned: glandium)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

For reproducibility network access is often disabled during downstream build.

$ ./mach bootstrap
$ ifconfig foo0 down # may differ on Linux
$ echo "mk_add_options MOZ_OBJDIR=/tmp/objdir" >>.mozconfig
$ ./mach build
[...]
0:25.12 layout/style/nsCSSPropertyID.h.stub
0:25.22 layout/style/ServoCSSPropList.h.stub
0:25.32 layout/style/CompositorAnimatableProperties.h.stub
0:25.42 layout/style/nsComputedDOMStyleGenerated.inc.stub
0:25.54 layout/style/nsCSSPropsGenerated.inc.stub
0:25.63 layout/style/ServoStyleConsts.h.stub
0:25.88 ERROR: Couldn't execute cargo metadata with manifest "toolkit/library/rust/Cargo.toml": Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", stderr: " Updating crates.io index\nwarning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)\nwarning: spurious network error (1 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)\nerror: failed to fetch https://github.com/rust-lang/crates.io-index\n\nCaused by:\n failed to resolve address for github.com: Name does not resolve; class=Net (12)\n" })
0:25.88 ERROR: Couldn't generate bindings for toolkit/library/rust.
0:25.88 gmake[4]: *** [backend.mk:72: .deps/ServoStyleConsts.h.stub] Error 1
0:25.88 gmake[3]: *** [config/recurse.mk:101: layout/style/export] Error 2
0:37.16 gmake[2]: *** [config/recurse.mk:34: export] Error 2
0:37.16 gmake[1]: *** [config/rules.mk:391: default] Error 2
0:37.16 gmake: *** [client.mk:125: build] Error 2
0:37.17 0 compiler warnings present.

Moving MOZ_OJBDIR under TOPSRCDIR appears to help.

Summary: cargo tries to fetch dependencies during build if MOZ_OBJDIR is set by user → Cargo phones home if MOZ_OBJDIR is outside of TOPSRCDIR

I think I'm facing the same issue on Solaris:

 1:32.53 xpcom/base/gk_rust_utils_ffi_generated.h.stub
 3:35.67 ERROR: Couldn't execute `cargo metadata` with manifest "/builds/psumbera/FIREFOX-2/toolkit/library/rust/Cargo.toml": Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", stderr: "    Updating crates.io index\nwarning: spurious network error (2 tries remaining): cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_nBM0Tb\': Disc quota exceeded; class=Os (2)\nwarning: spurious network error (1 tries remaining): cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_ikiCga\': Disc quota exceeded; class=Os (2)\nerror: failed to fetch `https://github.com/rust-lang/crates.io-index`\n\nCaused by:\n  cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_WGQ_Ua\': Disc quota exceeded; class=Os (2)\n" })
 3:35.67 ERROR: Couldn't generate bindings for /builds/psumbera/FIREFOX-2/toolkit/library/rust.
 3:35.69 gmake[4]: *** [backend.mk:11: .deps/webrender_ffi_generated.h.stub] Error 1

What is MOZ_OBJDIR?

Does this still happen on current mozilla-central?

Yes, it does (496448:3fa65bda1e50). And I see the typo now: MOZ_OJBDIR->MOZ_OBJDIR... I will check whether it helps.

My MOZ_OJBDIR is actually under TOPSRCDIR by default. So the workaround doesn't help me..

What version of cargo are you using?

I can't reproduce comment 0 after mozilla-central changeset 7728710226af. Beta 13 still fails.

I do use cargo 1.37.0 (though in /usr/bin/ there is older 1.35.0, but PATH have 1.37 as first one).

(In reply to Jan Beich from comment #7)

I can't reproduce comment 0 after mozilla-central changeset 7728710226af. Beta 13 still fails.

7728710226af, not 64cab9bb87f4?

Following cargo command causes problem on Solaris:

ulx-0 09:08 /builds/psumbera/FIREFOX-2: /builds/psumbera/rustc-1.37.0/bin/cargo metadata --all-features --format-version 1 --manifest-path /builds/psumbera/FIREFOX-2/toolkit/library/rust/Cargo.toml
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_ipvJIc': Disc quota exceeded; class=Os (2)
warning: spurious network error (1 tries remaining): failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_kbsHSa': Disc quota exceeded; class=Os (2)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_PRW9Bd': Disc quota exceeded; class=Os (2)

The cargo command from above comment is executed by cbindgen:

/builds/psumbera/cbindgen-0.9.1/bin/cbindgen /builds/psumbera/FIREFOX-2/toolkit/library/rust --lockfile /builds/psumbera/FIREFOX-2/Cargo.lock --crate geckoservo

And since I'm using not very big NFS home it gets full by '~/.cargo'. The immediate workaround is to set for build HOME to different path with enough space (for created ~/.cargo - 209MB).

But why Firefox build needs to download another sources from internet? So far we were able to build Firefox offline. Please, please don't do it :-)

Duplicate of this bug: 1587568

I found the problem. There's a race condition between $OBJDIR/.cargo/config being created and the script invoking cbindgen running.

Assignee: nobody → mh+mozilla

.cargo/config is necessary for all invocations of cargo, which, apart
from normal rust targets, actually also happens indirectly when we
invoke cbindgen.

Summary: Cargo phones home if MOZ_OBJDIR is outside of TOPSRCDIR → ERROR: Couldn't execute `cargo metadata` with manifest "/builds/worker/workspace/build/src/toolkit/library/rust/Cargo.toml"
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/b5ae1bbf82da
Add missing dependencies on .cargo/config. r=mshal

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Beta/Release Uplift Approval Request

  • User impact if declined: Random build failures in some environments. In other cases, unwanted download of rust code from crates.io while generating bindings.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Simple change to build system dependencies.
  • String changes made/needed: N/A
Attachment #9104104 - Flags: approval-mozilla-release?
Attachment #9104104 - Flags: approval-mozilla-beta?
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Low-risk for beta and no end user impact, uplift approved for 71 beta 5.

Attachment #9104104 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Fix for build issues, OK for uplift to m-r.

Attachment #9104104 - Flags: approval-mozilla-release? → approval-mozilla-release+
You need to log in before you can comment on or make changes to this bug.