Closed Bug 1314679 Opened 3 years ago Closed 3 years ago

Support artifact builds on try for Windows 2012 on TaskCluster

Categories

(Testing :: General, defect, P2)

defect

Tracking

(firefox53 fixed)

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: maja_zf, Assigned: grenade)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

See https://treeherder.mozilla.org/#/jobs?repo=try&revision=805d3f2a4af1&selectedJob=30324234 (Windows 2012 opt/debug 32/64: build-win32/debug, build-win32/opt, build-win64/debug, build-win64/opt)

This is a Tier-2 build, so it was beyond the scope of Bug 1299702.
One problem is that fx_desktop_build.py relies on the build's mozharness config to follow a certain naming convention to determine the platform. I have a try push where I fix that issue by renaming the config files for these windows jobs: https://hg.mozilla.org/try/rev/622e6d6624d5ecbf6606a32e3d4c1c630efa0d56

In turn, the jobs fail in the clone-tools step with:
> 22:05:22    FATAL - Automation Error: Can't checkout https://hg.mozilla.org/build/tools!
> 22:05:22    FATAL - Caught exception: could not find the robustcheckout Mercurial extension 

I think the problem comes from the resulting mozharness actions after we swap in the artifact config in response to --artifact in the try syntax:

> 21:52:34     INFO - Artifact build requested in try syntax.
> 21:52:34     INFO - Updating self.config with the following from Z:\task_1477517577\build\src\testing\mozharness\scripts\..\configs\builds/releng_sub_windows_configs/64_artifact.py:

64_artifact.py config specifies the following actions, and that's what the job uses with --artifact: 
'clobber',
'clone-tools',
'checkout-sources',
'build',
'sendchange',

But taskcluster_firefox_win64_opt.py specifies just:
'clone-tools',
'build',
'check-test',

So I think running clobber messes something up with clone-tools in this case. 

Maybe, in general, we actually want our list of actions under --artifact to be the set intersection of the default actions the original mozharness config used for the job and the artifact mozharness config being swapped in.
looking in 32_artifact.py and 64_artifact.py, those configs reset a bunch of path values to values which are only valid for buildbot builders. things like the PATH environment variable and vcs_share_base have different values on taskcluster builders. the specific hg error from the try builds mentioned above is caused by the PATH env var being reset to include the path to the old version of hg from mozilla-build instead of the recent version. i'm guessing that the only config that's really needed in those should be the values that actually differ from the base configs. eg:

32_artifact.py:

config = {
    'base_name': 'WINNT_5.2_%(branch)s_Artifact_build',
    'src_mozconfig': 'browser/config/mozconfigs/win32/artifact',
}

64_artifact.py:

config = {
    'base_name': 'WINNT_6.1_x86-64_%(branch)s_Artifact_build',
    'src_mozconfig': 'browser/config/mozconfigs/win64/artifact',
}

i tested this theory here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8a13e0ee842614a01078b9aecaa2e6c2bc7ace63
i got 16 minute builds so i'm not sure if somethings wrong or if artifact builds are supposed to be that fast.
That looks like a successful artifact build. Thanks, Rob!
Comment on attachment 8814391 [details]
Bug 1314679 - rename mh configs to support platform recognition for artifact builds

https://reviewboard.mozilla.org/r/95666/#review95694
Attachment #8814391 - Flags: review?(rthijssen) → review+
since artifact builds would be very helpful with taskcluster test greening where we're only interested in test results (rather than build results) and faster builds would facilitate this, i went ahead and pushed both Maja's change and mine into review so we can hopefully get this merged soon.
Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96540

Looks good to me, and I see it works fine on try. Thanks for the patch!
Attachment #8814392 - Flags: review?(cmanchester) → review+
Please ensure that the review flags are properly set in MozReview so that Autoland can push this.
Assignee: nobody → rthijssen
Keywords: checkin-needed
Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96608
Attachment #8814392 - Flags: review+
Comment on attachment 8814392 [details]
Bug 1314679 - remove inherited mh config to support tc win artifact builds

https://reviewboard.mozilla.org/r/95668/#review96610
i think autoland may have choked on my review of maja's commit (because i pushed it to review, presumably)
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/44b646931108
Rename mh configs to support platform recognition for artifact builds. r=grenade
https://hg.mozilla.org/integration/mozilla-inbound/rev/4b304f6f7c92
Remove inherited mh config to support tc win artifact builds. r=chmanchester
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/44b646931108
https://hg.mozilla.org/mozilla-central/rev/4b304f6f7c92
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.