1320721 - Mach Bootstrap fails to recognize Rust path in Windows

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Robin Steuber (she/her) [:bytesized]]

I attempted to follow the instructions laid out by the email "Rust required to build Gecko". I am running Windows 10. These are the steps that I followed:

1. Added line to ~/.bashrc: |export PATH="${PATH}:~/.cargo/bin"|
2. Pulled changes: |hg pull -u|
3. Ran Mach Bootstrap |./mach bootstrap|

Rust seems to have installed successfully, but this error was printed:

> You have some rust files in c:/Users/ksteuber\.cargo\bin, but they're not part of the
> standard PATH.
> 
> To make these available, please add this directory to the PATH variable in your shell initialization script, which may be called ~/.bashrc or ~/.bash_profile or ~/.profile. Edit this and add the following line:
> 
>     source c:/Users/ksteuber\.cargo/env
> 
> Then restart your shell and run the bootstrap script again.

I checked that my path does contain "~/.cargo/bin".

Additional Info:

> $ which rustc
> /c/Users/ksteuber/.cargo/bin/rustc.exe
> $ cat ~/.cargo/env
> cat: /c/Users/ksteuber/.cargo/env: No such file or directory

Running |./mach bootstrap| again results in the same error message.

Comment 1

[Robin Steuber (she/her) [:bytesized]]

Hmm. Closed command prompt and re-opened it and the error message seems to have gone away. Not sure how that works.

Comment 2

[Ralph Giles (:rillian)]

In my testing, the installer successfully adds ~/.cargo/bin to the path of whatever environment you run it on, but on Windows it doesn't provide a ~/.cargo/env. I think exporting path changes to the parent shell is impossible for security reasons, so one needs to restart the command prompt to see the changes.

I think a good fix here is to update the message to (a) suggest restarting the shell after install and (b) suggest a PATH update instead of `source ~/.cargo/env` if ~/.cargo/env isn't present.

Comment 3

[Robin Steuber (she/her) [:bytesized]]

I want to reiterate that in my situation, this line was incorrect:

> You have some rust files in c:/Users/ksteuber\.cargo\bin, but they're not part of the
> standard PATH.

The files WERE in the PATH. It seems to me that the script should perform this check correctly or else not print that error.

Comment 4

[Robin Steuber (she/her) [:bytesized]]

Oh, I should have mentioned that I did restart my console after editing my .bashrc. The correct list of steps that I followed should have been:

1. Added line to ~/.bashrc: |export PATH="${PATH}:~/.cargo/bin"|
2. Closed and reopened console
3. Pulled changes: |hg pull -u|
4. Ran Mach Bootstrap |./mach bootstrap|

Comment 5

[Ralph Giles (:rillian)]

Ok, When you said after restarting the command prompt (the second time) the error went away. I took that to mean that your manual step to add `~/.cargo/bin` to `~/.bashrc` made the files appear in your config, but the change wasn't active in the command prompt you were using. After opening a new window, the .bashrc change or the attempts the installer makes to add it to the msys or windows path were picked up and `which rustc` and `./mach bootstrap` both found `~/.cargo/bin/rustc`.

Are you saying that `which rustc` and `rustc --version` were succeeding in the same window where `./mach boostrap` was failing to find rust? If so, I have no idea how to explain that. I don't have much Windows experience though; maybe someone more familiar has an idea.

Maybe your prompt isn't actually reading `.bashrc` and it's the restart after the installer runs for the first time that does the trick? That doesn't explain `which` working but `bootstrap` not. Are all of these invocations from the msys shall started by `mozilla-build\start_shell.bat` or are some of them from the Windows/Visual Studio command prompt?

Comment 6

[Robin Steuber (she/her) [:bytesized]]

Yes, I am saying that I got this error:

> You have some rust files in c:/Users/ksteuber\.cargo\bin, but they're not part of the
> standard PATH.

and these command outputs:

> $ which rustc
> /c/Users/ksteuber/.cargo/bin/rustc.exe
> $ cat ~/.cargo/env
> cat: /c/Users/ksteuber/.cargo/env: No such file or directory

From the same console, without restarting it.

I am sure that my .bashrc was being read, because I checked my $PATH and it contained |~/.cargo/bin|, which I added via my .bashrc. All commands are from a shell started via "C:\mozilla-build\start-shell-msvc2015.bat".

Comment 7

[Ralph Giles (:rillian)]

I worry that checking $PATH isn't enough. Maybe try setting another variable there and see if it propagates.

I say this because:

On unix rustup writes ~/.cargo/env with a bash-style `export PATH` line, and sources it in ~/.profile.

On Windows, it adds ~/.cargo/bin to the PATH through the windows registry. So I think it would be in path after the installer runs regarless.

Further, the mozilla-build installer in `./mach boostrap` pokes the rust path into `msys/etc/profile.d`, which is a third way the path could be updated. I don't know which of those three win, or how they interact between the Windows shell and msys.

Thanks for helping debug this. I'm learning things!

Comment 8

[Ralph Giles (:rillian)]

https://github.com/rust-lang-nursery/rustup.rs/blob/a07ee8c0e3b7634bd0b5725348780c19c82f1446/src/rustup-cli/self_update.rs#L964 is the PATH update implementation, if you're curious.

The mozilla-build bootstrap code is https://hg.mozilla.org/mozilla-central/file/bfa85d23df57/python/mozboot/mozboot/mozillabuild.py#l35

Comment 9

[Robin Steuber (she/her) [:bytesized]]

So, first of all, I am 100% sure that my .bashrc is being sourced. I have a LOT of stuff in there and I would notice if it were missing immediately since one of the things it sets is a custom prompt.

I am confused though. If $PATH contains "~/.cargo/bin" and |which rustc| returns the correct path, why should it even matter if my .bashrc is being sourced properly? You said:

> I worry that checking $PATH isn't enough.

but |./mach bootstrap|'s error was:

> You have some rust files in c:/Users/ksteuber\.cargo\bin, but they're not part of the
> standard PATH.

So is the PATH the issue, or not?

Comment 10

[Ralph Giles (:rillian)]

The bootstrap error is triggered with the boostrap code's 'which' method doesn't find rustc in path, but os.path.exists('~/.cargo/bin/rustc') returns true.

It seems like the python environment which implementation wasn't seeing the same PATH as the shell. So something wasn't propagating, or perhaps the python version was getting the wrong `os.pathsep` so the traversal failed?

Oh! Trying this manually the WindowsBootstrapper module won't load, deferring to the MozillaBuildBootstrapper. So maybe I was wrong to expect the windows implementation of the `which` method to append '.exe', and it's failing because it's checking for `~/.cargo/bin/rustc' but you have '~/.cargo/bin/rustc.exe'. The second check manually appends '.exe' for Windows. But again: how does it ever work if that was the issue?

  $ cd mozilla-central/python/mozboot
  $ python
  >>> from mozboot.windows import WindowsBootstrapper
  >>> b = WindowsBootstrapper()
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "mozboot/windows.py", line 40, in __init__
      raise NotImplementedError('Bootstrap support for Windows is under development. For now, use MozillaBuild
  NotImplementedError: Bootstrap support for Windows is under development. For now, use MozillaBuild to set up a build environment on Windows. If you are testing Windows Bootstrap support, try `export MOZ_WINDOWS_BOOTSTRAP=1`

Comment 11

[Ralph Giles (:rillian)]

No, that wasn't the issue. The `mach boostrap` command uses MozillaBuildBootstrapper by default but it overrides `which()` just like WindowsBootstrapper does, so the behaviour should be as I expected. I tried removing `mozilla-build/msys/etc/profile.d/profile-rustup.sh` but while I had to restart my shell, it still worked.

Since I can't reproduce and have no leads, I'm going to go ahead and close. Sorry for the trouble. Please re-open if it happens to you again.