Package mozharness as part of the build package step (as opposed to the test package step)

RESOLVED FIXED in Firefox 44

Status

defect
RESOLVED FIXED
4 years ago
2 years ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

unspecified
mozilla44
Dependency tree / graph

Firefox Tracking Flags

(firefox44 fixed)

Details

Attachments

(1 attachment)

Taskcluster test jobs need to obtain a copy of the in-tree mozharness. Currently, it gets zipped up as part of 'make package-tests' [1]. Then the TC build jobs will manually copy that bundle to the artifact upload dir in a series of platform specific shell scripts.

Seeing as mozharness isn't just for tests, it would make more sense to just do this packaging/uploading as part of the build package step, alongside the symbols, app package, etc.. This way, we don't need to handle this in every individual class of test jobs.

A side-effect is that buildbot jobs will also have a mozharness.zip in their build dir. This could either be seen as a good thing (opens up new avenues) or a bad thing (wasted space/compute time). Currently buildbot jobs obtain mozharness through a service called 'archiver' (which for some reason I don't understand we decided not to use for taskcluster). Making them use mozharness.zip instead of archiver could be a potential follow-up (disclaimer: I know nothing about archiver or its pros/cons).

I'm open to feedback on this, but in my limited understanding, this seems like the current best option.

p.s The reason I'm looking into this is that TC linux64 builds are different from TC mulet/emulator builds and fixing this would automatically make the mozharness.zip available to linux64 test jobs.

[1] https://dxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#431
Please don't bit rot bug 1208320 :)
Depends on: 1208320
Crap, wish I saw that. My patch does pretty much exactly the same thing as:
https://reviewboard.mozilla.org/r/20339/diff/3#index_header

Oh well, I'm not in a huge rush to land this.. so I can wait for that to land and rebase my patch on top of it. I'll upload what I have for now anyway.
Comment on attachment 8667425 [details]
MozReview Request: Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps

Bug 1209628 - Package mozharness as part of the build
Comment on attachment 8667425 [details]
MozReview Request: Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps

Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps
Attachment #8667425 - Attachment description: MozReview Request: Bug 1209628 - Package mozharness as part of the build → MozReview Request: Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps
Attachment #8667425 - Flags: review?(gps)
Here's a try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=260b04d1b3b7

Could make an argument that mozharness packaging doesn't belong in mozbuild/action/test_archive.py anymore, but I don't think it's a big enough deal to care about.
Comment on attachment 8667425 [details]
MozReview Request: Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps

https://reviewboard.mozilla.org/r/20745/#review19023

::: toolkit/mozapps/installer/package-name.mk:136
(Diff revision 3)
> +MOZHARNESS_PACKAGE = $(PKG_BASENAME).mozharness.zip

There's nothing build configuration specific about the mozharness archive: there really isn't a need for $(PKG_BASENAME) here. Convince me this isn't a cargo cult.
Attachment #8667425 - Flags: review?(gps) → review+
It is. I'll change it.
Comment on attachment 8667425 [details]
MozReview Request: Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps

Bug 1209628 - Package mozharness alongside other build artifacts rather than as part 'package-tests', r=gps
https://hg.mozilla.org/mozilla-central/rev/2f7150d0e0fa
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
Depends on: 1221356
See Also: → 1232887
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.