Closed
Bug 1185643
Opened 8 years ago
Closed 8 years ago
switch over taskcluster jobs to use gecko copy of mozharness
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(firefox43 fixed)
RESOLVED
FIXED
Tracking | Status | |
---|---|---|
firefox43 | --- | fixed |
People
(Reporter: jlund, Assigned: dustin)
References
Details
(Whiteboard: [build])
Attachments
(2 files)
thanks to progress in https://bugzil.la/1131856, a copy of mozharness is now in the gecko tree and is being used for all buildbot based jobs. We can now switch over TC to also use the gecko copy. You have a couple options: 1) if you have a copy of the tree and the rev you are build/testing on the slave: # just call the script you need from the tree: `python $gecko_tree/testing/mozharness/scripts/$script # with args` from the same cwd you have been calling mozharness from before. The work_dir (`pwd`/build) can stay the same 2) if you don't have a copy (say maybe for tests): # you can use Archiver[1] and the client[2] to download a tarball of mozharness based on a specific gecko repo + rev: `python $BUD/tools_jlund/buildfarm/utils/archiver_client.py mozharness --repo integration/mozilla-inbound --rev 0a1f292ba8cc --destination $CWD/mozharness --debug; python mozharness/scripts/$script # + args` [1] https://api.pub.build.mozilla.org/docs/usage/archiver/ [2] https://hg.mozilla.org/build/tools/file/default/buildfarm/utils/archiver_client.py
Assignee | ||
Updated•8 years ago
|
Blocks: q3-bb-tc-migration
Updated•8 years ago
|
Whiteboard: [build]
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → dustin
Assignee | ||
Comment 1•8 years ago
|
||
Wander: testing/taskcluster/tasks/build.yml has references to MOZHARNESS_* which I won't remove yet, on the assumption that b2g builds are still depending on them (bug 1187706)
Assignee | ||
Comment 2•8 years ago
|
||
From the patch I'm about to submit (more or less) https://tools.taskcluster.net/task-inspector/#24zeN42OSuKwl-ynm4z7dA/0
Assignee | ||
Comment 3•8 years ago
|
||
Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn This required moving the custom mozharness configs out of the builder image (which puts them in ~, where mozharness no longer looks) and moves them into mozharness itself.
Attachment #8639464 -
Flags: review?(winter2718)
Assignee | ||
Comment 4•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn This required moving the custom mozharness configs out of the builder image (which puts them in ~, where mozharness no longer looks) and moves them into mozharness itself.
Attachment #8639464 -
Flags: review?(wcosta)
Assignee | ||
Comment 5•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn This required moving the custom mozharness configs out of the builder image (which puts them in ~, where mozharness no longer looks) and moves them into mozharness itself.
Comment 6•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander The only issue I have with this, is that $WORKSPACE/build/src/testing is hard coded in before $MOZHARNESS_SCRIPT so if someone wants to add a custom path they'll need to add "../../../" Otherwise, heck yes. :)
Updated•8 years ago
|
Attachment #8639464 -
Flags: review?(winter2718) → review+
Comment 7•8 years ago
|
||
On second thought, we'll probably have more users wanting to just change scripts from within the tree than we will people who want to test their own MH checkout. +1
Comment 8•8 years ago
|
||
https://reviewboard.mozilla.org/r/14201/#review12819 lgtm
Comment 9•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander https://reviewboard.mozilla.org/r/14203/#review12823 Ship It!
Attachment #8639464 -
Flags: review?(wcosta) → review+
Depends on: 1188330
Assignee | ||
Comment 11•8 years ago
|
||
I'm game to change $MOZHARNESS_SCRIPT to be relative to ${topsrcdir}, but let's take one step at a time. Specifically, it'd be great to have gecko, b2g, etc. all using the same approach for that.
Assignee | ||
Comment 12•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e3c15b9f89b3&exclusion_profile=false for the sake of verifying I didn't break anything B2G
Comment 13•8 years ago
|
||
seems this broke a lot of stuff like b2g and mulet like https://treeherder.mozilla.org/logviewer.html#?job_id=12197062&repo=mozilla-inbound so backed this out and hopefully this fix the issue and we are able to re-open mozilla-inbound
Comment 14•8 years ago
|
||
Backout: https://hg.mozilla.org/integration/mozilla-inbound/rev/2a2f1d5a647b
Assignee | ||
Comment 15•8 years ago
|
||
I'm incidentally adding some debugging to 'mach'; here's some sample output: https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/7oXTBVAzSyCtTOtO2jgqoA/0/public/logs/live_backing.log
Assignee | ||
Comment 16•8 years ago
|
||
Ah, the failures I was seeing in mach are from bug 1186003, the landing of which caused a permafail for linux builds in TC. I'm not sure why the manifestation was different than in bug 1187047, but hopefully the fix to both is the same.
Depends on: 1187047
This is blocking me for getting Mulet Gu. When can I hope that this is fixed?
Blocks: 1188337
Assignee | ||
Comment 20•8 years ago
|
||
:ted -- we might be able to land this patch more-or-less as-is, even if the desktop builds still don't work (12 days ago, there were 3 or so things causing desktop builds to fail, this among them -- probably more now). I don't recall why the backout occurred.
Assignee | ||
Comment 21•8 years ago
|
||
Ah, yes -- https://dxr.mozilla.org/mozilla-central/source/testing/docker/tester/bin/entrypoint starts with `test $MOZHARNESS_REPOSITORY`, and due to the mach_commands.py changes in e3c15b9f89b3, that value is empty.
Assignee | ||
Comment 22•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander Bug 1185643: use in-tree mozharness for builds; r?wander This required moving the custom mozharness configs out of the builder image (which puts them in ~, where mozharness no longer looks) and moving them into mozharness itself.
Attachment #8639464 -
Attachment description: MozReview Request: Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn → MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander
Attachment #8639464 -
Flags: review+ → review?(winter2718)
Updated•8 years ago
|
Attachment #8639464 -
Flags: review+
Comment 23•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander https://reviewboard.mozilla.org/r/14203/#review14995 ::: testing/taskcluster/mach_commands.py:288 (Diff revision 2) > mozharness = load_mozharness_info() Can't this be removed?
Assignee | ||
Updated•8 years ago
|
No longer blocks: q3-bb-tc-migration
Comment 24•8 years ago
|
||
Comment on attachment 8639464 [details] MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander https://reviewboard.mozilla.org/r/14203/#review14997 Ship It!
Attachment #8639464 -
Flags: review+
Assignee | ||
Comment 25•8 years ago
|
||
It can't be removed yet, because testing/taskcluster/tester/bin/entrypoint still refers to MOZHARNESS_RE*
Assignee | ||
Updated•8 years ago
|
Attachment #8639464 -
Flags: review?(winter2718)
Assignee | ||
Comment 27•8 years ago
|
||
Attachment #8651094 -
Flags: review?(wcosta)
Updated•8 years ago
|
Attachment #8651094 -
Flags: review?(wcosta) → review+
Updated•5 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•