1622948 - Add buildhub.json files incorporating gecko and android-components revision information for fenix

Status: NEW

(Release Engineering :: General, task)

Description

[William Lachance (:wlach)]

buildhub has proven a very useful tool for desktop-- it lets us see whether a buildid we see in telemetry corresponds to an official Firefox release from us.

The mechanism it uses is to produce a buildhub.json file along with the build with a bunch of metadata (see bug 1443873 for implementation details), which are then consumed by other systems. I think we could use a similar approach for Fenix, which would (eventually) not only let us use the excellent buildhub web ui for browsing releases, it would also let us get a printable regression range if we were ever to add support for it to mozregression (bug 1556042).

After supporting Fenix, it would be nice to actually have this type of data for every product that we build (e.g. Firefox reality). But I think Fenix would be a priority

For more information see Johan Lorenzo's helpful comment here:

https://bugzilla.mozilla.org/show_bug.cgi?id=1556042#c15

Comment 1

[Johan Lorenzo [:jlorenzo]]

:bhearsum and I met yesterday and we chatted about how we can get buildhub.json generated. In the gecko world, it's done by informulate[1], which is called as part of the build[2].

buildhub.json contains this data, for example[3]:

{
  "build": {
    "as": "/builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang -std=gnu99",
    "cc": "/builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang -std=gnu99",
    "cxx": "/builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ -std=gnu++17",
    "date": "2020-03-16T13:00:52Z",
    "host": "x86_64-pc-linux-gnu",
    "id": "20200316130052",
    "target": "x86_64-pc-linux-gnu"
  },
  "download": {
    "date": "2020-03-16T13:16:31.710775Z",
    "mimetype": "application/octet-stream",
    "size": 73025002,
    "url": "target.tar.bz2"
  },
  "source": {
    "product": "firefox",
    "repository": "https://hg.mozilla.org/integration/autoland",
    "revision": "e144f3d78dfaeaf1b26a4997051935b60902b4a0",
    "tree": "autoland"
  },
  "target": {
    "channel": "default",
    "locale": "en-US",
    "os": "linux",
    "platform": "linux-x86_64",
    "version": "76.0a1"
  }
}

• build contains compiler metadata. We won't use clang to compile Fenix, but gradle instead. We might want to add gradle metadata for it. Taskgraph interact with gradle so it can handle it.
• source and target is metadata taskgraph knows and can pass down to the build task.
• if I understand correctly, download is metadata about the final artifact. We somehow need a script to generate it.

Speaking of script, informulate.py is really gecko-centric. Both :bhearsum and I don't think we should bother porting it to Fenix. Instead, we may just want to reuse the bits that generate the download section, since that's the only part gradle can't control.

Finally, we may want Fenix to be at parity with gecko before adding geckoview and android-components metadata. The latter may require some changes on the buildhub side.

[1] https://searchfox.org/mozilla-central/rev/7dafc35406b9c945189c617d427f5458933fd3fb/toolkit/mozapps/installer/informulate.py
[2] https://searchfox.org/mozilla-central/rev/bc3600def806859c31b2c7ac06e3d69271052a89/toolkit/mozapps/installer/packager.mk#119
[3] https://firefoxci.taskcluster-artifacts.net/Lf_yN9B7RBSEfcb8gsm9-Q/0/public/build/buildhub.json

Comment 2

[William Lachance (:wlach)]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)

Finally, we may want Fenix to be at parity with gecko before adding geckoview and android-components metadata. The latter may require some changes on the buildhub side.

This makes sense to me, but note that it is what is most required for mozregression. :) Just getting a regression range for fenix itself wouldn't be very useful IMO.

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

Quick touch base on this as I've seen this resurrected in https://github.com/mozilla-releng/buildhub2/issues/628. Just to confirm, what's the priority of this? We don't currently have too much bandwitdh in RelEng for tackling it but I want to understand what a reasonable timeframe for it might be, if any?

Comment 4

[Pascal Chevrel:pascalc]

For crash-stop, calixte found a workaround so emergency on the relman side

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Pascal Chevrel:pascalc from comment #4)

For crash-stop, calixte found a workaround so emergency on the relman side

Confirmed over Matrix that this is not an emergency for RelMan side.

Comment 6

[William Lachance (:wlach)]

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #3)

Quick touch base on this as I've seen this resurrected in https://github.com/mozilla-releng/buildhub2/issues/628. Just to confirm, what's the priority of this? We don't currently have too much bandwitdh in RelEng for tackling it but I want to understand what a reasonable timeframe for it might be, if any?

I don't think it's an emergency, but I think implementing this would open the door to a few things that are ~impossible right now:

1. Much easier bisections of problems / bugs that we notice in Firefox for Android via mozregression (you can see that there is a fair amount of usage of mozregression for desktop Firefox, this isn't a product people use for fun so continued use shows its value: https://sql.telemetry.mozilla.org/queries/70610#177730)
2. For products like GLAM, it would allow us to show platform changes that led to a quality metric changing: https://github.com/mozilla/glam/issues/1124

I suspect there are many more examples, e.g. performance automation like perfherder but these are the two I know of off the top of my head.
tl;dr: I think we're really suffering for not having this, as it makes understanding how/why something changed in the product much more difficult in many dimensions.

Comment 7

[William Lachance (:wlach)]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #6)

2. For products like GLAM, it would allow us to show platform changes that led to a quality metric changing: https://github.com/mozilla/glam/issues/1124

Another GLAM issue seems to be that non-official builds are getting mixed into the results, which this could help with: https://github.com/mozilla-mobile/fenix/issues/18092

Comment 8

[Andrew Halberstadt [:ahal]]

Now that :jlorenzo has left, we have very little mobile experience. I was going to try and take a stab at this but am quite out of my depth. Aki was suggesting we could build a wrapper around gradle that generates the buildhub.json file. Does that sound reasonable :wlach? Or do you have any pointers on a good place to integrate something like this?

Comment 9

[Andrew Halberstadt [:ahal]]

Another idea Aki had was to insert a && generate_buildhub_json.py into the task command:
https://github.com/mozilla-mobile/fenix/blob/master/taskcluster/fenix_taskgraph/job.py#L52-L64

Maybe that would be cleaner.

Comment 10

[William Lachance (:wlach)]

(In reply to Andrew Halberstadt [:ahal] from comment #8)

Now that :jlorenzo has left, we have very little mobile experience. I was going to try and take a stab at this but am quite out of my depth. Aki was suggesting we could build a wrapper around gradle that generates the buildhub.json file. Does that sound reasonable :wlach? Or do you have any pointers on a good place to integrate something like this?

Unfortunately I don't have much experience with building Android apps. I would lean towards the second option personally but that could just be because I'm scared of anything to do with gradle.

:mdroettboom, :tlong -- sorry to rope you into this but you have more relevant experience here than I do and this is technically a data problem. Do you have any feedback/input on what approach might work best here? Comment 1 has a description of the data we would generate.

Comment 11

[Travis Long [:travis_]]

It seems like this could be done with Gradle, perhaps as a plugin. I admit my Gradle expertise isn't the greatest but the ability to invoke a python script generate_buildhub_json.py isn't all that much different than invoking the glean_parser. As to who would be someone who could tackle something like this? That I can't answer, but perhaps someone on the Android team (thinking of Sebastian) might be able to point you in the right direction for Gradle expertise.

Comment 12

[William Lachance (:wlach)]

(In reply to Travis Long [:travis_] from comment #11)

It seems like this could be done with Gradle, perhaps as a plugin. I admit my Gradle expertise isn't the greatest but the ability to invoke a python script generate_buildhub_json.py isn't all that much different than invoking the glean_parser. As to who would be someone who could tackle something like this? That I can't answer, but perhaps someone on the Android team (thinking of Sebastian) might be able to point you in the right direction for Gradle expertise.

Good point, I believe the gradle code which invokes the glean_parser is defined here: https://github.com/mozilla/glean/tree/main/gradle-plugin (was looking at the other day when investigating something else)

Comment 13

[Michael Droettboom [:mdroettboom]]

Seems like a reasonable solution for projects that are using Glean, since they have a Python interpreter handy. Non-Glean projects don't necessarily have a Python, but that's probably fine for now since this buildhub ask is sort of in service of the data platform and therefore of Glean right now.