1314679 - Support artifact builds on try for Windows 2012 on TaskCluster

Status: RESOLVED FIXED

(Testing :: General, defect, P2)

Description

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

See https://treeherder.mozilla.org/#/jobs?repo=try&revision=805d3f2a4af1&selectedJob=30324234 (Windows 2012 opt/debug 32/64: build-win32/debug, build-win32/opt, build-win64/debug, build-win64/opt)

This is a Tier-2 build, so it was beyond the scope of Bug 1299702.

Comment 1

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

One problem is that fx_desktop_build.py relies on the build's mozharness config to follow a certain naming convention to determine the platform. I have a try push where I fix that issue by renaming the config files for these windows jobs: https://hg.mozilla.org/try/rev/622e6d6624d5ecbf6606a32e3d4c1c630efa0d56

In turn, the jobs fail in the clone-tools step with:
> 22:05:22    FATAL - Automation Error: Can't checkout https://hg.mozilla.org/build/tools!
> 22:05:22    FATAL - Caught exception: could not find the robustcheckout Mercurial extension 

I think the problem comes from the resulting mozharness actions after we swap in the artifact config in response to --artifact in the try syntax:

> 21:52:34     INFO - Artifact build requested in try syntax.
> 21:52:34     INFO - Updating self.config with the following from Z:\task_1477517577\build\src\testing\mozharness\scripts\..\configs\builds/releng_sub_windows_configs/64_artifact.py:

64_artifact.py config specifies the following actions, and that's what the job uses with --artifact: 
'clobber',
'clone-tools',
'checkout-sources',
'build',
'sendchange',

But taskcluster_firefox_win64_opt.py specifies just:
'clone-tools',
'build',
'check-test',

So I think running clobber messes something up with clone-tools in this case. 

Maybe, in general, we actually want our list of actions under --artifact to be the set intersection of the default actions the original mozharness config used for the job and the artifact mozharness config being swapped in.

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

looking in 32_artifact.py and 64_artifact.py, those configs reset a bunch of path values to values which are only valid for buildbot builders. things like the PATH environment variable and vcs_share_base have different values on taskcluster builders. the specific hg error from the try builds mentioned above is caused by the PATH env var being reset to include the path to the old version of hg from mozilla-build instead of the recent version. i'm guessing that the only config that's really needed in those should be the values that actually differ from the base configs. eg:

32_artifact.py:

config = {
    'base_name': 'WINNT_5.2_%(branch)s_Artifact_build',
    'src_mozconfig': 'browser/config/mozconfigs/win32/artifact',
}

64_artifact.py:

config = {
    'base_name': 'WINNT_6.1_x86-64_%(branch)s_Artifact_build',
    'src_mozconfig': 'browser/config/mozconfigs/win64/artifact',
}

i tested this theory here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8a13e0ee842614a01078b9aecaa2e6c2bc7ace63
i got 16 minute builds so i'm not sure if somethings wrong or if artifact builds are supposed to be that fast.

Comment 3

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

That looks like a successful artifact build. Thanks, Rob!

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: Bug 1314679 - rename mh configs to support platform recognition for artifact builds

Review commit: https://reviewboard.mozilla.org/r/95666/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/95666/

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: Bug 1314679 - remove inherited mh config to support tc win artifact builds

Review commit: https://reviewboard.mozilla.org/r/95668/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/95668/

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8814391 [details]
Bug 1314679 - rename mh configs to support platform recognition for artifact builds

https://reviewboard.mozilla.org/r/95666/#review95694

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

since artifact builds would be very helpful with taskcluster test greening where we're only interested in test results (rather than build results) and faster builds would facilitate this, i went ahead and pushed both Maja's change and mine into review so we can hopefully get this merged soon.

Comment 8

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96540

Looks good to me, and I see it works fine on try. Thanks for the patch!

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Please ensure that the review flags are properly set in MozReview so that Autoland can push this.

Comment 10

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96608

Comment 11

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96610

Comment 12

[Rob Thijssen [:grenade (EET/UTC+0300)]]

i think autoland may have choked on my review of maja's commit (because i pushed it to review, presumably)

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/44b646931108
Rename mh configs to support platform recognition for artifact builds. r=grenade
https://hg.mozilla.org/integration/mozilla-inbound/rev/4b304f6f7c92
Remove inherited mh config to support tc win artifact builds. r=chmanchester

Comment 14

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/44b646931108
https://hg.mozilla.org/mozilla-central/rev/4b304f6f7c92