1530908 - Various changes to artifact builds on automation

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Use an environment variable to enable LTO on automation.

This allows to disable it more easily for artifact builds.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Unify artifact builds setup between try config and try syntax.

While try syntax is approaching its EOL, the fact that using it to do
artifact builds does some things subtly differently from using try
config is not helpful.

While here, fix the try.rst doc wrt the "enabled" value for "artifacts",
which, if it's not a string, fails builds on Windows:

[taskcluster:error] TASK FAIL since the task payload is invalid. See errors:
[taskcluster:error] - env: Invalid type. Expected: string, given: integer

Depends on D21311

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Always set USE_ARTIFACT from taskcluster for artifact builds.

The artifact builds that are automatically derived using the artifact
template set the USE_ARTIFACT environment variable from taskcluster.
After the previous change, --artifact builds from try syntax do that
too.
That leaves us with only the artifact-build build not doing it, so for
consistency, do it there. That makes it not necessary to set
USE_ARTIFACT from mozconfig.artifact.automation anymore.

Depends on D21312

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Move --enable-artifact-build-symbols to mozconfig.artifact.

Depends on D21313

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Don't use different mozconfigs for artifact builds.

Artifact mozconfigs are not necessarily up-to-date wrt changes to the
nightly mozconfigs, and all in all, shouldn't be much different from
them.

It's just better to use the nightly mozconfigs (or beta on beta, etc.)
and make the mozconfigs themselves handle the few things that need to be
different when the USE_ARTIFACT environment is set (which is now
consistently set by taskcluster)

This does have the side effect of turning builds that actually don't
support artifact builds red when using --artifact on try, instead of
having them silently not be artifact builds as currently happens.

Depends on D21314

Comment 6

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Use a transform for try --artifact instead of a morph.

While the morph was changing the treeherder symbol to Ba for all jobs,
doing so with a transform fails because of the conflicting symbol check
(as multiple jobs in the same category would end up with Ba). So
instead, we append a to the existing symbol.

We also change the documentation wrt templates for try pushes, as the
artifact template is now gone (although technically, mach try will still
set params['templates']['artifacts']['enabled'] for now).

Depends on D21311

Comment 7

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Move list of supported artifact builds to an importable module.

Depends on D22054

Comment 8

[Mike Hommey [:glandium]]

Attachment: Bug 1530908 - Only enable artifacts on try builds that support them.

Currently, all tasks of kind builds are indiscriminately altered to use
artifacts, but only few of them actually support that, and the others
won't actually have the expected result when that happens. E.g. ASAN
builds with artifacts enabled end up being non-ASAN builds.

Effectively, this makes the artifact flag ignored for builds that don't
support artifacts. One could argue that those builds shouldn't happen at
all, but it feels a better use time of developer's time to just do the
full build they asked for. E.g. if they asked for ASAN with artifacts,
they still get an ASAN build, rather than an error or silently having
the task not happen after the decision task. This also allows to mix
artifact and non-artifact builds.

Further changes down the road are also modifying the artifact builds
configuration, which would actively turn those builds that don't support
artifact builds red (e.g. ASAN), so something has to be done anyways.
The alternative would be filter those builds out.

Depends on D21312

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ede80ab4fa72
Use an environment variable to enable LTO on automation. r=chmanchester
https://hg.mozilla.org/integration/autoland/rev/6b25ace491c3
Use a transform for try --artifact instead of a morph. r=tomprince
https://hg.mozilla.org/integration/autoland/rev/ecc39e015e7a
Move list of supported artifact builds to an importable module. r=chmanchester
https://hg.mozilla.org/integration/autoland/rev/61dd01ed9002
Unify artifact builds setup between try config and try syntax. r=dustin
https://hg.mozilla.org/integration/autoland/rev/ad8315f51807
Only enable artifacts on try builds that support them. r=dustin
https://hg.mozilla.org/integration/autoland/rev/c21559af66d6
Always set USE_ARTIFACT from taskcluster for artifact builds. r=chmanchester
https://hg.mozilla.org/integration/autoland/rev/8afb91736f70
Move --enable-artifact-build-symbols to mozconfig.artifact. r=chmanchester
https://hg.mozilla.org/integration/autoland/rev/0d70e7e33020
Don't use different mozconfigs for artifact builds. r=chmanchester

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/ede80ab4fa72
https://hg.mozilla.org/mozilla-central/rev/6b25ace491c3
https://hg.mozilla.org/mozilla-central/rev/ecc39e015e7a
https://hg.mozilla.org/mozilla-central/rev/61dd01ed9002
https://hg.mozilla.org/mozilla-central/rev/ad8315f51807
https://hg.mozilla.org/mozilla-central/rev/c21559af66d6
https://hg.mozilla.org/mozilla-central/rev/8afb91736f70
https://hg.mozilla.org/mozilla-central/rev/0d70e7e33020