1185643 - switch over taskcluster jobs to use gecko copy of mozharness

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jordan Lund (:jlund)]

thanks to progress in https://bugzil.la/1131856, a copy of mozharness is now in the gecko tree and is being used for all buildbot based jobs.

We can now switch over TC to also use the gecko copy.

You have a couple options:

1) if you have a copy of the tree and the rev you are build/testing on the slave:

# just call the script you need from the tree:
`python $gecko_tree/testing/mozharness/scripts/$script # with args` from the same cwd you have been calling mozharness from before. The work_dir (`pwd`/build) can stay the same

2) if you don't have a copy (say maybe for tests):

# you can use Archiver[1] and the client[2] to download a tarball of mozharness based on a specific gecko repo + rev:

`python $BUD/tools_jlund/buildfarm/utils/archiver_client.py mozharness --repo integration/mozilla-inbound --rev 0a1f292ba8cc --destination $CWD/mozharness --debug; python mozharness/scripts/$script # + args`

[1] https://api.pub.build.mozilla.org/docs/usage/archiver/
[2] https://hg.mozilla.org/build/tools/file/default/buildfarm/utils/archiver_client.py

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Wander: testing/taskcluster/tasks/build.yml has references to MOZHARNESS_* which I won't remove yet, on the assumption that b2g builds are still depending on them (bug 1187706)

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

From the patch I'm about to submit (more or less)
  https://tools.taskcluster.net/task-inspector/#24zeN42OSuKwl-ynm4z7dA/0

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

Comment 6

[Morgan Phillips [:mrrrgn]]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

The only issue I have with this, is that $WORKSPACE/build/src/testing is hard coded in before $MOZHARNESS_SCRIPT so if someone wants to add a custom path they'll need to add "../../../"

Otherwise, heck yes. :)

Comment 7

[Morgan Phillips [:mrrrgn]]

On second thought, we'll probably have more users wanting to just change scripts from within the tree than we will people who want to test their own MH checkout. +1

Comment 8

[Wander Lairson Costa]

https://reviewboard.mozilla.org/r/14201/#review12819

lgtm

Comment 9

[Wander Lairson Costa]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review12823

Ship It!

Comment 10

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e3c15b9f89b3

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

I'm game to change $MOZHARNESS_SCRIPT to be relative to ${topsrcdir}, but let's take one step at a time.  Specifically, it'd be great to have gecko, b2g, etc. all using the same approach for that.

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e3c15b9f89b3&exclusion_profile=false for the sake of verifying I didn't break anything B2G

Comment 13

[Carsten Book [:Tomcat]]

seems this broke a lot of stuff like b2g and mulet like https://treeherder.mozilla.org/logviewer.html#?job_id=12197062&repo=mozilla-inbound so backed this out and hopefully this fix the issue and we are able to re-open mozilla-inbound

Comment 14

[Pulsebot]

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2a2f1d5a647b

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

I'm incidentally adding some debugging to 'mach'; here's some sample output:
  https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/7oXTBVAzSyCtTOtO2jgqoA/0/public/logs/live_backing.log

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

Ah, the failures I was seeing in mach are from bug 1186003, the landing of which caused a permafail for linux builds in TC.  I'm not sure why the manifestation was different than in bug 1187047, but hopefully the fix to both is the same.

Comment 18

[Alexandre LISSY :gerard-majax [PTO 12/12/2022-15/02/2023]]

This is blocking me for getting Mulet Gu. When can I hope that this is fixed?

Comment 20

[Dustin J. Mitchell [:dustin] (he/him)]

:ted -- we might be able to land this patch more-or-less as-is, even if the desktop builds still don't work (12 days ago, there were 3 or so things causing desktop builds to fail, this among them -- probably more now).  I don't recall why the backout occurred.

Comment 21

[Dustin J. Mitchell [:dustin] (he/him)]

Ah, yes -- https://dxr.mozilla.org/mozilla-central/source/testing/docker/tester/bin/entrypoint starts with `test $MOZHARNESS_REPOSITORY`, and due to the mach_commands.py changes in e3c15b9f89b3, that value is empty.

Comment 22

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness for builds; r?wander

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moving them into
mozharness itself.

Comment 23

[Wander Lairson Costa]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review14995

::: testing/taskcluster/mach_commands.py:288
(Diff revision 2)
>          mozharness = load_mozharness_info()

Can't this be removed?

Comment 24

[Wander Lairson Costa]

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review14997

Ship It!

Comment 25

[Dustin J. Mitchell [:dustin] (he/him)]

It can't be removed yet, because testing/taskcluster/tester/bin/entrypoint still refers to MOZHARNESS_RE*

Comment 26

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/be6ff275a904

Comment 27

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bump.patch

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/be6ff275a904