1349180 - stylo builds do not work on try with --artifact, many tests seem to fail

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P4)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

when almost all test jobs had failures, I realized something was wrong:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&fromchange=c66e6ec8b07cb5a8af9d7315f5a2a9d9d4a0cae4

Comment 1

[Chris Peterson [:cpeterson]]

Nathan offered to take a look at the Stylo artifact build problems.

Comment 2

[(not currently active) Ted Mielczarek]

jmaher linked me to this try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e770ce01a18fcb4c9ad05e6f633d7d6fa924670b&selectedJob=85430512

Looking at the log from that particular build (build-linux64-stylo/opt), it downloads this build:
[task 2017-03-21T20:17:28.634396Z] 20:17:28     INFO -  Installing from https://queue.taskcluster.net/v1/task/IaUWDpCTSzudKp8CWuVRQQ/artifacts/public/build/target.tar.bz2

Looking at the task it's downloading from, it seems to be a non-stylo Linux64 opt build:
https://tools.taskcluster.net/task-inspector/#IaUWDpCTSzudKp8CWuVRQQ/

I think what happens here is that we have special handling for artifact builds on try:
[task 2017-03-21T20:16:00.027232Z] 20:16:00     INFO - Artifact build requested in try syntax.
[task 2017-03-21T20:16:00.027412Z] 20:16:00     INFO - Using artifact build variant "artifact".
[task 2017-03-21T20:16:00.028100Z] 20:16:00     INFO - Updating self.config with the following from /home/worker/workspace/build/src/testing/mozharness/scripts/../configs/builds/releng_sub_linux_configs/64_artifact.py:
<...>
[task 2017-03-21T20:16:00.043486Z] 20:16:00     INFO -  'src_mozconfig': 'browser/config/mozconfigs/linux64/artifact',

...and so we wind up using the artifact mozconfig, and not the stylo one here. I'm not sure if there's a straightforward way to support build variants with artifact builds on try, but we should at least make it not do *this*.

Comment 3

[Chris Peterson [:cpeterson]]

Nathan offered to take a look at the Stylo artifact build problems. I forgot to needinfo him on this bug.

Comment 4

[Nathan Froyd [:froydnj]]

(In reply to Chris Peterson [:cpeterson] from comment #3)
> Nathan offered to take a look at the Stylo artifact build problems. I forgot
> to needinfo him on this bug.

Thanks.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> I think what happens here is that we have special handling for artifact
> builds on try:
> [task 2017-03-21T20:16:00.027232Z] 20:16:00     INFO - Artifact build
> requested in try syntax.
> [task 2017-03-21T20:16:00.027412Z] 20:16:00     INFO - Using artifact build
> variant "artifact".
> [task 2017-03-21T20:16:00.028100Z] 20:16:00     INFO - Updating self.config
> with the following from
> /home/worker/workspace/build/src/testing/mozharness/scripts/../configs/
> builds/releng_sub_linux_configs/64_artifact.py:
> <...>
> [task 2017-03-21T20:16:00.043486Z] 20:16:00     INFO -  'src_mozconfig':
> 'browser/config/mozconfigs/linux64/artifact',
> 
> ...and so we wind up using the artifact mozconfig, and not the stylo one
> here. I'm not sure if there's a straightforward way to support build
> variants with artifact builds on try, but we should at least make it not do
> *this*.

Maybe the "right" thing to do is create an artifact variant for stylo builds, and have the appropriate mechanism select that instead?  We already duplicate artifact build things (mozconfigs, mozharness configs, ...) for everything else.  Seems like copy-and-pasting is the way of things for these kind of builds...

Also, why are we interested in artifact builds for Stylo?  Do we expect frontend developers to be testing things with Stylo to see if they go any faster or something?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

I was testing a few changes to tests and needed to run in linux64-stylo and my default try push uses --artifact, so all my tests failed :(   I don't know if we need this specifically, but we shouldn't have things failing when pushing with --artifact to try.

Comment 6

[(not currently active) Ted Mielczarek]

Yeah, I don't think this is really intentional. We should either make this error or just not run variant builds when you push to try with --artifact.

Comment 7

[Chris Manchester (limited bugmail, email directly)]

Unfortunately neither configure nor the artifact code are equipped to produce stylo artifact builds right now, I have a patch that will revert these to regular builds even in the presence of `--artifact` in try syntax.

Comment 8

[Chris Manchester (limited bugmail, email directly)]

Attachment: Bug 1349180 - Don't attempt an artifact build for stylo builds with --artifact in try syntax.

Our configs as well as the artifact code are not equipped to do this.

Review commit: https://reviewboard.mozilla.org/r/128698/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/128698/

Comment 9

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8856771 [details]
Bug 1349180 - Don't attempt an artifact build for stylo builds with --artifact in try syntax.

https://reviewboard.mozilla.org/r/128698/#review131166

Stamp!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/596ea55fa962
Don't attempt an artifact build for stylo builds with --artifact in try syntax. r=nalexander

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/596ea55fa962

Comment 12

[(not currently active) Ted Mielczarek]

This and bug 1356213 seem like basically the same thing. Should we flip this around and make just the opt+debug builds opt-in to artifact build support instead?

Comment 13

[Nick Alexander :nalexander [he/him]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #12)
> This and bug 1356213 seem like basically the same thing. Should we flip this
> around and make just the opt+debug builds opt-in to artifact build support
> instead?

This is sensible, because I expect the number of "esoteric" builds to grow as we push for more and finer-grained tooling.  I think I followed suit with the existing code, which was opt-out.

See https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/fx_desktop_build.py#130, which explains part of this design space -- but not whether we should be opt-in or opt-out.