1488429 - add automatic setup of windows build environment to sccache

Status: NEW

(Firefox Build System :: General, enhancement, P3)

Description

[Nathan Froyd [:froydnj]]

Right now, for cross-compiling AArch64 Windows builds, the current setup uses a bunch of junction points (i.e. Windows symlink-alikes) to point as directories for include files, for library files, and for the appropriate bin/ directories for the host and target compilers.  The last one is done because cl.exe for the host and target are distinct, and we can't just have cl.exe in PATH: we need to have separate binaries, and absolute paths to the appropriate cl.exe (and link.exe) is the only way to accomplish this.

We use junction points rather than the pathnames directly because the actual pathnames (C:\Program Files (x86)\blah\blah\blah) have a bunch of spaces in them, and therefore don't play well with the build system.  Our automation builds mostly get around the spaces issue by using custom packages of Visual Studio, installed into directories without any spaces, and hand-assembling LIB, PATH, etc. environment variables.

I'm at a point where I can start thinking about how to package this up for other people to use.  Pointing them at a script to create the appropriate junction points + mozconfig incantations is one option.  Having bootstrap create the junction points + mozconfig incantations is another.  But wouldn't it be much nicer if something else took care of everything, and you didn't have to worry about mozconfig foo?

My initial idea was to have a virtual filesystem driver, so M:\ would point at the appropriate places.  This seems easy enough to implement with Dokany:

https://github.com/dokan-dev/dokany

but there's still some manual steps.  My second idea was to run everything under a wrapper program that sets up the translations...but we already have this fine wrapper program in sccache, so maybe we should just use that instead?  No filesystem translation tricks needed, sccache could set up all the environment variables and whatnot cl.exe needs, so we could rip out bits of configure and makefiles as well.

It's kind of scope creep for sccache, but having another wrapper around sccache to provide the right environment for cl.exe et al seems like too much futzing (though we might need wrappers for link.exe or midl.exe or similar anyway?).  Does this sound like a completely harebrained idea, or is there something actually worth pursuing here?

Comment 1

[(not currently active) Ted Mielczarek]

This seems...a bit scope creep-y, honestly. What would you expect sccache to do, exactly? We could possibly leverage the `cc` crate's functionality to locate MSVC installations:
https://docs.rs/cc/1.0.25/cc/windows_registry/index.html

...but I'm not sure how that'd help you manage multiple MSVC versions. Would you want to be able to do something like:
```
sccache c:/path/to/aarch64/cl.exe ...
sccache c:/path/to/x64/cl.exe ...
```

and have sccache figure out the proper environment for each of those invocations based solely on the path to cl.exe?

Comment 2

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

With :kmoir's consent, I'm reviewing the backlog for bugs in FFx Build System to see if we can do some things to clean up the backlog.

This bug appears to be an enhancement or feature request. If so, please set the bug's severity to `enhancement` and/or add the `feature` keyword depending on if this is an improvement to an existing thing, or a new feature/tool/system.