1557785 - Switch Windows PGO builds to the 3-tier model

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Michael Shal [:mshal]]

Linux and Android PGO builds both use the 3-tier model, but Windows still uses the old 1-tier model. We should switch Windows over as well so that all PGO builds use the same PGO logic.

Comment 1

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Add win64 PGO-instrumented task; r?tomprince

Comment 2

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Update run-profileserver.sh for Windows compatibility; r?#firefox-build-system-reviewers

The run-profileserver.sh script is a bridge between the Taskcluster task
and profileserver.py. It was originally written as a Linux-only script,
but with a few modifications it can support Windows as well. The xvfb
support needs to be optional, and the UPLOAD_PATH and PGO_RUNDIR
variables must not assume a Linux filesystem.

Depends on D34795

Comment 3

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Add Windows generate profile task; r?tomprince

Depends on D34796

Comment 4

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Hook MOZ_PGO_PROFILE_USE environment variable into common Windows mozconfig; r?#firefox-build-system-reviewers

MOZ_PGO_PROFILE_USE is set when the use-pgo attribute is defined in the
task. This environment variable is used to enable --enable-profile-use
and related configure flags.

Depends on D34797

Comment 5

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Allow LLVM_PROFDATA to work even if not explicitly set in the environment; r?#firefox-build-system-reviewers

Windows finds llvm-profdata in the PATH, in contrast to Linux or Android
builds that set LLVM_PROFDATA as an environment variable in mozconfigs.
The pgo_profile_path() configure checks should still work in this case.

Depends on D34798

Comment 6

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Fix automatic setting of LTO with clang-cl; r?dmajor

With clang-cl and PGO enabled, toolchain.configure automatically turns
on LTO for compatibility with MSVC. However, MOZ_PGO is set for both the
profile-generate and profile-use builds, but we only want LTO enabled
for the profile-use build. We can check for --enable-profile-use
explicitly and to avoid setting LTO on profile-generate.

Depends on D34799

Comment 7

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Switch backslashes to forward-slashes in JARLOG_FILE_AB_CD for Windows; r?glandium

Our version of mozmake in Windows has issues with how it shells out to
commands that are prefixed with environment variable settings. For
example, "FOO=bar $(PYTHON) script.py" behaves differently than
"$(PYTHON) script.py". In this case, packager.py is called with
JARLOG_FILE_AB_CD as a command-line argument, which gets processed
differently without the environment variable prefix. With a prefix, a
backslash gets slurped, so Z:\task/foo becomes Z:task/foo, and python
fails to open the file.

The backslash comes from the WORKSPACE variable in Taskcluster, which is
used in many places, so it seems prudent here to simply escape the
backslash as a workaround for the issue. Other possible fixes are: 1)
Convert the NO_PKG_FILES environment variable to a command-line
argument, 2) Upgrade mozmake to a newer version that fixes the shell
issues, or 3) Change WORKSPACE to use forward-slashes, and work around
any potential issues with that.

Depends on D34800

Comment 8

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Process install manifests in 3-tier Windows builds; r?#firefox-build-system-reviewers

Windows 1-tier PGO builds only partially clobber between the
profile-generate and profile-use stages, so that exports/installed files
don't have to be reprocessed. Unfortunately we can't skip the install
manifests in 3-tier PGO because the profile-generate build happens on a
different machine, so we have to differentiate between 1-tier and 3-tier
PGO builds. A new variable, MOZ_1TIER_PGO, is used for this purpose.

Eventually this logic can be cleaned up in bug 1557788 once all PGO
builds use the 3-tier model.

Depends on D34801

Comment 9

[Michael Shal [:mshal]]

Attachment: Bug 1557785 - Enable 3-tier PGO on Windows builds; r?tomprince

Depends on D34802

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1bc6ee0c6672
Add win64 PGO-instrumented task; r=tomprince
https://hg.mozilla.org/integration/autoland/rev/2989571031f9
Update run-profileserver.sh for Windows compatibility; r=firefox-build-system-reviewers,chmanchester
https://hg.mozilla.org/integration/autoland/rev/516aa5eb9732
Add Windows generate profile task; r=tomprince
https://hg.mozilla.org/integration/autoland/rev/0b6522f13bcf
Hook MOZ_PGO_PROFILE_USE environment variable into common Windows mozconfig; r=firefox-build-system-reviewers,chmanchester
https://hg.mozilla.org/integration/autoland/rev/3430c89be449
Allow LLVM_PROFDATA to work even if not explicitly set in the environment; r=firefox-build-system-reviewers,chmanchester
https://hg.mozilla.org/integration/autoland/rev/b5c65f2ea1ac
Fix automatic setting of LTO with clang-cl; r=dmajor
https://hg.mozilla.org/integration/autoland/rev/861e83cc1b9b
Switch backslashes to forward-slashes in JARLOG_FILE_AB_CD for Windows; r=glandium
https://hg.mozilla.org/integration/autoland/rev/91b34c90d2a9
Process install manifests in 3-tier Windows builds; r=firefox-build-system-reviewers,chmanchester
https://hg.mozilla.org/integration/autoland/rev/a91cc939f112
Enable 3-tier PGO on Windows builds; r=tomprince

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/1bc6ee0c6672
https://hg.mozilla.org/mozilla-central/rev/2989571031f9
https://hg.mozilla.org/mozilla-central/rev/516aa5eb9732
https://hg.mozilla.org/mozilla-central/rev/0b6522f13bcf
https://hg.mozilla.org/mozilla-central/rev/3430c89be449
https://hg.mozilla.org/mozilla-central/rev/b5c65f2ea1ac
https://hg.mozilla.org/mozilla-central/rev/861e83cc1b9b
https://hg.mozilla.org/mozilla-central/rev/91b34c90d2a9
https://hg.mozilla.org/mozilla-central/rev/a91cc939f112

Comment 12

[Florin Strugariu [:Bebe]]

== Change summary for alert #21604 (as of Wed, 26 Jun 2019 13:00:38 GMT) ==

Improvements:

3% raptor-speedometer-firefox windows10-64-shippable-qr opt 82.81 -> 85.22
2% raptor-speedometer-firefox windows10-64-shippable opt 82.50 -> 84.43

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21604

Comment 13

[Florin Strugariu [:Bebe]]

== Change summary for alert #21601 (as of Wed, 26 Jun 2019 08:22:19 GMT) ==

Improvements:

40% build times windows2012-64-shippable opt nightly taskcluster-c5.4xlarge 5,335.07 -> 3,182.93

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21601