1641291 - Support cross-compiling from macOS host -> Windows target.

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Nick Alexander :nalexander [he/him]]

After the work of Bug 1617794, we're very close to being able to cross-compile from macOS to Windows. This ticket tracks hammering down the last few issues.

Comment 1

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1641291 - Pre: Work around logging not handling '\x' in `old-configure` output. r?#build

Windows accepts both paths, so let's just avoid the issue for now.

Comment 2

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1641291 - Part 1: Allow cross-compiling from host macOS -> Windows target. r?#build

Depends on D77116

Comment 3

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1641291 - Part 2: Make NSDISTMODE=copy impact install manifests. r?glandium

This is strictly a quality of life improvement. The most common
scenario is mounting an object directory into a Windows VM, which
generally doesn't appear to handle (absolute) symlinks gracefully.

Depends on D77117

Comment 4

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1641291 - Part 3: Ensure that paths are considered paths when compiling. r?#build

Extremely common file paths on macOS like /Users/... are interpreted
as /U... flags by clang-cl. This is so common that there's a
-Wslash-u-filename warning. Ensure that file paths are considered
paths when compiling by terminating options with --.

Depends on D77118

Comment 5

[Nick Alexander :nalexander [he/him]]

I've pushed this up for review, but I haven't checked that full builds in fact succeed. My use case is artifact builds (so that I can work on the installer/uninstaller), and that definitely works. And then I want to add a separate project for the Windows default browser agent (no ticket yet), and that definitely works.

My artifact mozconfig looks like:

mk_add_options AUTOCLOBBER=1

ac_add_options --enable-application=browser
ac_add_options --target=x86_64-pc-mingw32

ac_add_options --enable-artifact-builds

mk_add_options MOZ_WINCONSOLE=1

mk_add_options MOZ_OBJDIR="../objdirs/objdir-windows-artifact"

export MAKENSISU="/Users/nalexander/.mozbuild/nsis-3.01/makensis.exe"
export WINE="/usr/local/bin/wine"

My defaultagent mozconfig looks like:

mk_add_options AUTOCLOBBER=1

ac_add_options --enable-application=toolkit/mozapps/defaultagent
ac_add_options --target=x86_64-pc-mingw32
ac_add_options --enable-debug

mk_add_options MOZ_WINCONSOLE=1

mk_add_options MOZ_OBJDIR="../objdirs/objdir-defaultagent-compile"

export MAKENSISU="/Users/nalexander/.mozbuild/nsis-3.01/makensis.exe"
export WINE="/usr/local/bin/wine"

VSPATH=/Users/nalexander/.mozbuild/vs2017_15.8.4
VSWINPATH=/Users/nalexander/.mozbuild/vs2017_15.8.4

export WINDOWSSDKDIR="${VSWINPATH}/SDK"
export WIN32_REDIST_DIR="${VSPATH}/VC/redist/x86/Microsoft.VC141.CRT"
export WIN_UCRT_REDIST_DIR="${VSPATH}/SDK/Redist/ucrt/DLLs/x86"
export DIA_SDK_PATH="${VSPATH}/DIA SDK"
export VC_PATH="${VSPATH}/VC"

export PATH=${VSPATH}/VC/bin/Hostx64/x64:${PATH}
unset VC_PATH

export LINKER=lld-link

# There's some kind of issue with the x64 version on macOS; work around it.
MT=/Users/nalexander/.mozbuild/vs2017_15.8.4/SDK/bin/10.0.17134.0/x86/mt.exe

Comment 6

[Nick Alexander :nalexander [he/him]]

With a slight tweak to the ---before-arguments patch to not do that for assembler files -- nasm doesn't like it, at least -- this produces green builds on try.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fe38c82c2dad
Pre: Work around logging not handling '\x' in `old-configure` output. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/d830bee40b5c
Part 1: Allow cross-compiling from host macOS -> Windows target. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/0dcf604b880e
Part 2: Make NSDISTMODE=copy impact install manifests. r=glandium
https://hg.mozilla.org/integration/autoland/rev/9c0a44614576
Part 3: Ensure that paths are considered paths when compiling. r=glandium

Comment 8

[Cristina Coroiu [:ccoroiu]]

Backed out 4 changesets (bug 1641291) for build bustages and SM failures

Backout: https://hg.mozilla.org/integration/autoland/rev/689892f0856646ff97dcd37d4fa6ec777a5a2ff4

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=9c0a4461457638440c1383f3a42f51c3d18f7c69

Failure logs:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=304937107&repo=autoland&lineNumber=651
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=304937119&repo=autoland&lineNumber=8209

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:nalexander, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 10

[Nick Alexander :nalexander [he/him]]

(In reply to Release mgmt bot [:sylvestre / :calixte / :marco for bugbug] from comment #9)

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:nalexander, could you have a look please?
For more information, please visit auto_nag documentation.

This turns out to be more exciting than I thought. There are at least three things at play:

1. try builds don't involve sccache, so in this case they lie;
2. sccache doesn't itself handle -- correctly in all cases
3. cc-rs doesn't insert -- -- see cc-rs issue #513

I will land the first few commits of this series and spin out the other bit about the paths into a separate ticket.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/69389186702e
Pre: Work around logging not handling '\x' in `old-configure` output. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/e1cc07af040c
Part 1: Allow cross-compiling from host macOS -> Windows target. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/e60508a2639b
Part 2: Make NSDISTMODE=copy impact install manifests. r=glandium

Comment 12

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/69389186702e
https://hg.mozilla.org/mozilla-central/rev/e1cc07af040c
https://hg.mozilla.org/mozilla-central/rev/e60508a2639b