1150348 - Add metadata to all builds pointing to url of symbols/build/tests/etc...

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect)

Description

[James Lal [:lightsofapollo]]

(I am sure this is the wrong component)

---

Particularly when running tests out of the tree (gaia/third party etc...) there are times when you want to figure out

 - Where the build came from
 - What type of build it is
 - Where are the symbols (crash reporting)

We have all kinds of "interesting" ways of figuring out this information but it would be nice to have a canonical source of this metadata which we can embed in the builds (the actual things we distribute) at minimum for non-release.

Comment 1

[James Lal [:lightsofapollo]]

Can you help me route this? I see a bright future where we can automatically detect symbols (at minimum) and lots of other fun things after we land this... I am planning to prototype this at minimum for TC controlled builds.

Comment 2

[James Lal [:lightsofapollo]]

(above comment for mshal)

Comment 3

[James Lal [:lightsofapollo]]

Some context into what the json would look like (ideally)

{
  "build": "https://queue.taskcluster.net/..../firefox-build.yay.tar.gz",
  "symbols": "https://queue.taskcluster.net/..../symbols.tar.gz",
}

Comment 4

[(not currently active) Ted Mielczarek]

James and I talked about this on IRC a bit, I think the right place for this is to dump an artifacts.json...next to the artifacts. We can stick it in the build archive if that makes people's lives easier. I wouldn't try to shoehorn this into any of our existing files.

We generate other json files that get uploaded in packager.mk:
https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/packager.mk#93

I'd just write a simple Python script that dumps the JSON, like:
https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/informulate.py?from=informulate.py#1

package-name.mk already has all the filenames in Makefile variables:
https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/package-name.mk#124

If you want full URLs obviously you'll need to pass the base URL in from the CI system.

Comment 5

[Michael Shal [:mshal]]

If we're just creating a new file, that definitely simplifies things. I was thinking application.ini or mach_build_properties.json would potentially work here, but they would both need changing.

In order to pass the base URL in, we'll need to tweak mozharness a bit. Right now all of the taskcluster uploading logic happens after the build.

Comment 6

[Wander Lairson Costa]

This is a really ugly proof of concept. The code is far from pretty and some things are hardcoded.

gecko code:
https://github.com/walac/gecko-dev/commit/b45e8df6001f117e0261903671b792b679c2f8c9

mozharness:
https://github.com/walac/build-mozharness/commit/a960622fe81a90c7cbb7f8dae07a3089e1de7849

TC build:
https://tools.taskcluster.net/task-inspector/#5qjjBHITRz24yzMcvN7Syw/1