1300901 - TC builds use MOZ_SIMPLE_PACKAGE_NAME

Status: RESOLVED WONTFIX

(Taskcluster :: General, defect)

Description

[Geoff Brown [:gbrown]]

Test tasks need to download artifacts created by their associated build tasks, but don't have a convenient way of determining the names of those artifacts. Most current TC builds use the mozilla build environment variable MOZ_SIMPLE_PACKAGE_NAME so that builds use well-known, constant names for artifacts, like target.tar.bz2 or target.apk; then tests can download "https://.../target.apk" without concern.

In contrast, most buildbot build jobs do not use MOZ_SIMPLE_PACKAGE_NAME and generate artifacts like firefox-51.0a1.en-US.linux-x86_64.tar.bz2. The release number changes over time of course.

I think TC builds should not use MOZ_SIMPLE_PACKAGE_NAME because:
 - it's not the same as buildbot
 - release, locale, and platform information is "lost" / no longer strongly associated with artifacts
 - MOZ_SIMPLE_PACKAGE_NAME does not work with Windows builds

Comment 1

[Geoff Brown [:gbrown]]

I'll take this if no one else is interested; feel free to take it while I'm gone if you like! :)

I had an idea that perhaps the test tasks could inspect https://queue.taskcluster.net/v1/task/<build-task-id>/runs/0/artifacts/ and search for '*.test_packages.json' or '*.tar.bz2', etc. Unfortunately, I only get "InsufficientScopes" when I try to read that /artifacts/ url.

I wonder if I could use an index instead?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

I know the folks working on nightlies and releases are worried about this.  On the other hand, there are a number of people using the index to find builds right now, and I imagine it's a lot easier to find a build with a static name.

The tasks are all generated by the same process, which runs in the gecko tree, so if that process can calculate the stem of the name then it can just insert that everywhere.  That's probably the best approach here.  But it does seem like a lot of work, and involves a lot of tying distant things together.. does `mach taskgraph` have to parse mozconfigs??

Comment 3

[Pete Moore [:pmoore][:pete]]

:gbrown - see https://bugzilla.mozilla.org/show_bug.cgi?id=1265537#c3 - I think it makes sense to set MOZ_SIMPLE_PACKAGE_NAME to `{{build_product}}.{{build_product_locale}}.{{build_name}}` - this way you lose the version number, but keep the other information which is static per task, varied across tasks. I believe the only mistake in the full version naming was inclusion of the version number.

It might make sense to hang bug 1265537 from here to cover the Windows builds - I'll let you decide. For now I just made it "See Also". :-)

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Geoff Brown [:gbrown] (PTO Sept 7-12) from comment #1)
> I had an idea that perhaps the test tasks could inspect
> https://queue.taskcluster.net/v1/task/<build-task-id>/runs/0/artifacts/ and
> search for '*.test_packages.json' or '*.tar.bz2', etc. Unfortunately, I only
> get "InsufficientScopes" when I try to read that /artifacts/ url.

Maybe you see bug 1295949 and try to access artifacts with encoded slashes?

Comment 5

[Justin Wood (:Callek)]

FWIW, on my L10n-in-tc work, I've opted to NOT use MOZ_SIMPLE_PACKAGE_NAMES so far...

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I really like the simple names as a rule.  They certainly make taskgraph generation easier, but they also make it easier for external users to find builds given an index name, without enumerating artifacts and guessing based on regexes.

The queue does support "reference" artifacts, so in cases where it's necessary or better to upload fully-named artifacts, we could potentially *also* upload reference artifacts with a simple name, pointing to the full name.  Best of both worlds!

Note that, as far as I know, the workers do not support this yet, so we would have to figure out how best to represent it (maybe translate symlinks to references??)

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

fwiw, i'm leaning towards keeping complex names. i don't have a good justification for my preference, i just find it easier to live with, than explaining why i simplified the name.

in the tc windows test efforts (on try), i've now started getting the version from the filesystem in the decision task (/home/worker/checkouts/gecko/browser/config/version.txt) and use that to build the artifact url strings. i'm happy to be convinced otherwise, but this is the direction i'm leaning today.

see: https://hg.mozilla.org/try/rev/fc0ffe8510563a28f97baaa2624dfe6249867c0d#l10.128

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

I'm OK with that, but please do so consistently across all platforms and task types!

It will probably make sense to land a patch for linux (and mac cross) builds on this bug, then base your tc windows work on that.

Comment 9

[Geoff Brown [:gbrown]]

Thanks everyone for the discussion here...and sorry for the long delay in responding!

Comment 10

[Geoff Brown [:gbrown]]

Attachment: use 'make echo-variable-PACKAGE' to retrieve package name

I found this alternative way for reliably determining the package name in generate_build_stats(). It solves one problem blocking non-constant package file names and seems like a good improvement regardless of how we proceed on other issues.

Comment 11

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #10)
> Created attachment 8796189 [details] [diff] [review]
> use 'make echo-variable-PACKAGE' to retrieve package name

Forgot to mention, it works well on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2992d7f5d21dcccda8ee6a3f9b3da5248b62b7fb

Comment 12

[(not currently active) Ted Mielczarek]

Let's not add more dependencies on build system internals (particularly not make, which we'd like to get rid of). If we don't like `MOZ_SIMPLE_PACKAGE_NAME` as a solution, what would you think if we added a different variable to just leave version numbers out of the filenames? That would be very straightforward to do, and allow you to have package filenames that would not change, but would vary per-build platform and otherwise match the existing buildbot builds.

Comment 13

[(not currently active) Ted Mielczarek]

You'd lose the ability to sort downloads of multiple nightly builds by version, but honestly I've hit the situation many times where I've downloaded multiple nightly builds from the same release cycle and don't actually know which is which without looking inside the archive.

We'd also clearly need to put the version number back in for uploading Nightly and Release builds, but that feels like something that release/nightly promotion is totally capable of.

Comment 14

[(not currently active) Ted Mielczarek]

I should compose all my thoughts into *one* bug comment before hitting submit, honestly.

As a counterpoint here: I've historically always been annoyed by the version number in the filenames of nightly builds, since it meant that trying to write a script to download builds had to scrape the list of files instead of just using well-known names. I'd rather not have to do the same thing forever with the shiny new Taskcluster index.

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8796189 [details] [diff] [review]
use 'make echo-variable-PACKAGE' to retrieve package name

Review of attachment 8796189 [details] [diff] [review]:
-----------------------------------------------------------------

clearing review in light of ted's comments

Comment 16

[Geoff Brown [:gbrown]]

It is fairly easy to not use MOZ_SIMPLE_PACKAGE_NAME (use the default build behavior and generate filenames with version, locale, and platform) and produce successful build and test tasks:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7109c9550345e5c4d68ed3ea6c5b066d2fed63d9


The main problem with the default package naming is that the build system determines the name of the package, while other entities - like test jobs or tasks - need to use that name. MOZ_SIMPLE_PACKAGE_NAME (or another environment variable or similar mechanism) allows an external party (like buildbot or the decision task) to specify the name, enabling tests or other entities running in the same, or possibly derived, environment (test tasks) to access the name...but it doesn't help anything outside of that environment, like Ted's build-downloading scripts (comment 14).

We could have the build echo PACKAGE into a well-known named file, say target.json, and read target.json to get the name of the package file...but that seems too convoluted.

There is also pmoore's idea to consider, from comment 3:

> set MOZ_SIMPLE_PACKAGE_NAME to `{{build_product}}.{{build_product_locale}}.{{build_name}}`

Without the version number, that's less variable, but it's still more difficult to work with than 'target.zip'.

I opened this bug from a "taskcluster migration" perspective, which for me means ensuring that our taskcluster builds and tests look and act the same as they have on buildbot. In this case, perhaps that's not important, or at least out-weighed by the advantages of having constant, well-known names.

Let's WONTFIX this, keep the existing 'target.xxx' names, and sort out the Windows problems (bug 1265537) as a separate issue.