Rust detection on Windows is broken with latest rustup.rs

RESOLVED FIXED in Firefox 53

Status

defect
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: xidorn, Assigned: glandium)

Tracking

Trunk
mozilla53
Dependency tree / graph

Firefox Tracking Flags

(firefox53 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

mach fails to configure with my local m-c (checked out one or two days ago) with the latest version of rustup (which I just update this morning).

THe issue is that configure located "cargo.EXE" and "rustc.EXE". Those files exist on Windows because the filesystem on Windows is case-insensitive. However, rustup's proxy cannot handle that correctly (probably because it only recognize the lowercased ".exe" somehow), so things are broken.

I think we probably need to fix this on both rustup and m-c, in case rustup doesn't get a quick enough update.

A temporary workaround would be lowercasing everything in the PATHEXT environment variable.
Blocks: oxidation
Oh, this is the bug that RyanVM was hitting yesterday.
Is there anything we can do about this on the m-c side? If so, can we get that into the tree so people's builds don't break en masse on affected versions of win10 or whatever it is that's causing this?
strawman proposal: make us use "rustc.exe" and "cargo.exe" on Windows in whatever the build vars are for those tools, just like we already harcode "hg.exe" in some places.
Just for others coming here, I added this to my .profile, based on comment 0:
export PATHEXT=$( echo "$PATHEXT" | tr -s  '[:upper:]'  '[:lower:]' )
Comment on attachment 8821086 [details]
Bug 1323901 - Hardcode rustc.exe and cargo.exe for detection on Windows.

https://reviewboard.mozilla.org/r/100462/#review101134

Let's kill 2 birds with one stone instead. (Patch incoming)
Attachment #8821086 - Flags: review?(mh+mozilla) → review-
Attachment #8821086 - Attachment is obsolete: true
Assignee: nobody → mh+mozilla
Comment on attachment 8821316 [details]
Bug 1323901 - Force `which` to use lowercase extensions on Windows.

https://reviewboard.mozilla.org/r/100616/#review101404

::: build/moz.configure/util.configure:157
(Diff revision 2)
> +        for ext in exts:
> +            if ext == ".exe":
> +                break
> +        else:
> +            exts = ['.com', '.exe', '.bat']

This might be clearer as `if '.exe' not in exts:` etc.
Attachment #8821316 - Flags: review?(cmanchester) → review+
(In reply to Marco Bonardo [::mak] from comment #5)
> Just for others coming here, I added this to my .profile, based on comment 0:
> export PATHEXT=$( echo "$PATHEXT" | tr -s  '[:upper:]'  '[:lower:]' )
Thanks Marco, that got me going.

My rust experience wasn't great so far. I've installed it from the stand-alone installer, which suggested C:\Users\jorgk\.cargo\bin as install location. C:\Users\jorgk\.cargo\bin; got added to the Windows environment variables.

However, the build still didn't work. So ran |mach bootstrap|. The result was that c:/mozilla-source/.cargo/bin got added to my bash path (echo $PATH). That obviously didn't work, since that's not were it was installed. Since I couldn't work out where that was done, I moved the .cargo/bin to where it was expected. And then I ran into this bug here.

So can someone tell me where the rust installation path is set to c:/mozilla-source/.cargo/bin. (**)

Also, what is C:\Users\jorgk\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\rustc.exe ?

Oh, just for added fun: C:\Users\jorgk\.cargo\bin\ reappeared during the build despite being moved previously.

Given that C:\Users\jorgk\.cargo\bin\ reappeared, I should be able to remove the copy in c:/mozilla-source/.cargo/bin, but then I need to know where c:/mozilla-source/.cargo/bin is set up as path, see (**) above.
Sorry about the off-topic question ;-)

(In reply to Jorg K (GMT+1) from comment #11)
> So can someone tell me where the rust installation path is set to
> c:/mozilla-source/.cargo/bin. (**)
Found it: C:\mozilla-build\msys\etc\profile.d\profile-rustup.sh

#!/bash/sh
if test -n "$MOZILLABUILD"; then
    WIN_HOME=$(cd "$HOME" && pwd)
    PATH="$WIN_HOME/.cargo/bin:$PATH"
    export PATH
fi
And since I have HOME=C:\mozilla-source for my Mozilla development, I get c:/mozilla-source/.cargo/bin.
Blocks: 1325917
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/21e27f6e2770
Force `which` to use lowercase extensions on Windows. r=cmanchester+432261
https://hg.mozilla.org/mozilla-central/rev/21e27f6e2770
https://hg.mozilla.org/mozilla-central/rev/536b05a23fa0
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
Hm, so I'm building with 232424f9511d and I'm still getting the cargo.EXE issue.  I don't have an updated rustup proxy -- do I need to?

https://hg.mozilla.org/mozilla-central/file/tip/python/mozbuild/mozbuild/vendor_rust.py#l27 is calling 'which.which' directly, which returns "cargo.EXE".  Note that I did not have cargo-vendor installed, so it tried to build it first (and failed since I didn't have cmake in my path, but that's a different thing).
I was running from an unbuilt tree, so I didn't have substs['CARGO'].  However, can we somehow capture stdout/stderr from these commands and print them? All I get now is:

 0:11.39 c:\Users\vladimir\.cargo\bin\cargo.exe update --manifest-path c:/proj/moz/central/toolkit/library/gtest/rust/Cargo.toml -p gkrust-gtest
Error running mach:

    ['vendor', 'rust', '--ignore-modified']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 101: [u'c:\\Users\\vladimir\\.cargo\\bin\\cargo.exe', u'update', u'--manifest-path', u'c:/proj/moz/central/toolkit/library/gtest/rust/Cargo.toml', u'-p', u'gkrust-gtest']


When what I really needed to see was:

$ cargo update --manifest-path c:/proj/moz/central/toolkit/library/gtest/rust/Cargo.toml -p gkrust-gtest
    Updating registry `https://github.com/rust-lang/crates.io-index`
error: failed to select a version for `dwrote` (required by `webrender`):
all possible versions conflict with previously selected versions of `dwrote`
  version 0.1.3 in use by dwrote v0.1.3
  possible versions to select: 0.1.4
File a new bug on that? Seems reasonable, I think I just didn't want `mach vendor rust` to produce a ton of output in the normal case.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.