1579761 - ERROR: Couldn't execute `cargo metadata` with manifest "/builds/worker/workspace/build/src/toolkit/library/rust/Cargo.toml"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

For reproducibility network access is often disabled during downstream build.

$ ./mach bootstrap
$ ifconfig foo0 down # may differ on Linux
$ echo "mk_add_options MOZ_OBJDIR=/tmp/objdir" >>.mozconfig
$ ./mach build
[...]
 0:25.12 layout/style/nsCSSPropertyID.h.stub
 0:25.22 layout/style/ServoCSSPropList.h.stub
 0:25.32 layout/style/CompositorAnimatableProperties.h.stub
 0:25.42 layout/style/nsComputedDOMStyleGenerated.inc.stub
 0:25.54 layout/style/nsCSSPropsGenerated.inc.stub
 0:25.63 layout/style/ServoStyleConsts.h.stub
 0:25.88 ERROR: Couldn't execute `cargo metadata` with manifest "toolkit/library/rust/Cargo.toml": Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", stderr: "    Updating crates.io index\nwarning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)\nwarning: spurious network error (1 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)\nerror: failed to fetch `https://github.com/rust-lang/crates.io-index`\n\nCaused by:\n  failed to resolve address for github.com: Name does not resolve; class=Net (12)\n" })
 0:25.88 ERROR: Couldn't generate bindings for toolkit/library/rust.
 0:25.88 gmake[4]: *** [backend.mk:72: .deps/ServoStyleConsts.h.stub] Error 1
 0:25.88 gmake[3]: *** [config/recurse.mk:101: layout/style/export] Error 2
 0:37.16 gmake[2]: *** [config/recurse.mk:34: export] Error 2
 0:37.16 gmake[1]: *** [config/rules.mk:391: default] Error 2
 0:37.16 gmake: *** [client.mk:125: build] Error 2
 0:37.17 0 compiler warnings present.

Comment 1

[Jan Beich]

Moving MOZ_OJBDIR under TOPSRCDIR appears to help.

Comment 2

[Petr Sumbera]

I think I'm facing the same issue on Solaris:

 1:32.53 xpcom/base/gk_rust_utils_ffi_generated.h.stub
 3:35.67 ERROR: Couldn't execute `cargo metadata` with manifest "/builds/psumbera/FIREFOX-2/toolkit/library/rust/Cargo.toml": Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", stderr: "    Updating crates.io index\nwarning: spurious network error (2 tries remaining): cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_nBM0Tb\': Disc quota exceeded; class=Os (2)\nwarning: spurious network error (1 tries remaining): cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_ikiCga\': Disc quota exceeded; class=Os (2)\nerror: failed to fetch `https://github.com/rust-lang/crates.io-index`\n\nCaused by:\n  cannot extend packfile \'/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_WGQ_Ua\': Disc quota exceeded; class=Os (2)\n" })
 3:35.67 ERROR: Couldn't generate bindings for /builds/psumbera/FIREFOX-2/toolkit/library/rust.
 3:35.69 gmake[4]: *** [backend.mk:11: .deps/webrender_ffi_generated.h.stub] Error 1

What is MOZ_OBJDIR?

Comment 3

[Mike Hommey [:glandium]]

Does this still happen on current mozilla-central?

Comment 4

[Petr Sumbera]

Yes, it does (496448:3fa65bda1e50). And I see the typo now: MOZ_OJBDIR->MOZ_OBJDIR... I will check whether it helps.

Comment 5

[Petr Sumbera]

My MOZ_OJBDIR is actually under TOPSRCDIR by default. So the workaround doesn't help me..

Comment 6

[Mike Hommey [:glandium]]

What version of cargo are you using?

Comment 7

[Jan Beich]

I can't reproduce comment 0 after mozilla-central changeset 7728710226af. Beta 13 still fails.

Comment 8

[Petr Sumbera]

I do use cargo 1.37.0 (though in /usr/bin/ there is older 1.35.0, but PATH have 1.37 as first one).

Comment 9

[Mike Hommey [:glandium]]

(In reply to Jan Beich from comment #7)

I can't reproduce comment 0 after mozilla-central changeset 7728710226af. Beta 13 still fails.

7728710226af, not 64cab9bb87f4?

Comment 10

[Petr Sumbera]

Following cargo command causes problem on Solaris:

ulx-0 09:08 /builds/psumbera/FIREFOX-2: /builds/psumbera/rustc-1.37.0/bin/cargo metadata --all-features --format-version 1 --manifest-path /builds/psumbera/FIREFOX-2/toolkit/library/rust/Cargo.toml
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_ipvJIc': Disc quota exceeded; class=Os (2)
warning: spurious network error (1 tries remaining): failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_kbsHSa': Disc quota exceeded; class=Os (2)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to create temporary file '/home/psumbera/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_PRW9Bd': Disc quota exceeded; class=Os (2)

Comment 11

[Petr Sumbera]

The cargo command from above comment is executed by cbindgen:

/builds/psumbera/cbindgen-0.9.1/bin/cbindgen /builds/psumbera/FIREFOX-2/toolkit/library/rust --lockfile /builds/psumbera/FIREFOX-2/Cargo.lock --crate geckoservo

And since I'm using not very big NFS home it gets full by '~/.cargo'. The immediate workaround is to set for build HOME to different path with enough space (for created ~/.cargo - 209MB).

But why Firefox build needs to download another sources from internet? So far we were able to build Firefox offline. Please, please don't do it :-)

Comment 13

[Mike Hommey [:glandium]]

I found the problem. There's a race condition between $OBJDIR/.cargo/config being created and the script invoking cbindgen running.

Comment 14

[Mike Hommey [:glandium]]

Attachment: Bug 1579761 - Add missing dependencies on .cargo/config.

.cargo/config is necessary for all invocations of cargo, which, apart
from normal rust targets, actually also happens indirectly when we
invoke cbindgen.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b5ae1bbf82da
Add missing dependencies on .cargo/config. r=mshal

Comment 16

[Mike Hommey [:glandium]]

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Beta/Release Uplift Approval Request

• User impact if declined: Random build failures in some environments. In other cases, unwanted download of rust code from crates.io while generating bindings.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Simple change to build system dependencies.
• String changes made/needed: N/A

Comment 17

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/b5ae1bbf82da

Comment 18

[Pascal Chevrel:pascalc]

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Low-risk for beta and no end user impact, uplift approved for 71 beta 5.

Comment 19

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/releases/mozilla-beta/rev/d049a5699739

Comment 20

[Liz Henry (:lizzard) (privacy/fingerprinting team)]

Comment on attachment 9104104 [details]
Bug 1579761 - Add missing dependencies on .cargo/config.

Fix for build issues, OK for uplift to m-r.

Comment 21

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-release/rev/86f4df89ffaa