1552583 - Teach Mozilla buildhub2 to answer queries about GeckoView versions used in downstream consumer Apps

Status: NEW

(GeckoView :: General, enhancement, P3)

Description

[Nick Alexander :nalexander [he/him]]

Right now, it's difficult to determine from an Android App that consumes GeckoView exactly which version of GeckoView it is consuming. For example, builds of Fenix capture that information in buildSrc here, but it's pretty difficult to determine that. I generally have to go from: Fenix APK -> mozilla-mobile/fenix git sha -> Gecko.kt code -> extract version -> buildhub to look up mozilla-central hg sha.

How can we make that easier? Could we leverage buildhub to capture that org.mozilla.fenix with android:versionCode X and android:versionName Y used GeckoView with build ID Z? Could we use an alternate service to capture and query this?

More generally, could we capture all of the dependencies in some way like this? Gradle builds provide this in a variety of forms; the question is how to report them and query them. It would be helpful to answer the same question for the version of Android Components and Application Services packages in the most recent Fenix build, for example.

Comment 1

[Nick Alexander :nalexander [he/him]]

:willkg -- thoughts on this? A quick glance at https://buildhub.readthedocs.io/en/latest/api.html#more-about-the-release-record-id suggests that this is a bit of a stretch for the current buildhub schema, but you may know otherwise.

Comment 2

[Nick Alexander :nalexander [he/him]]

I've put this in the GeckoView product because it's part of Mozilla's android ecosystem that pivots around GeckoView; happy to re-home if there's a better place.

Comment 3

[Nick Alexander :nalexander [he/him]]

NI for https://bugzilla.mozilla.org/show_bug.cgi?id=1552583#c1.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Two things:

First, we have a Buildhub and a Buildhub2. Buildhub is probably going to get decomissioned once we get Buildhub users moved over to Buildhub2. Buildhub2 is at https://buildhub.moz.tools/ .

Second, having said that, the problem is still the same. Both Buildhub and Buildhub2 (and Socorro) pull their build information from archive.mozilla.org from buildinfo.json files generated during the build process and they're only looking at specific information.

Is Fenix information on archive.mozilla.org? I didn't think it was, but maybe I just don't know where it is.

For products that use different build/release systems than Firefox, we have a couple of options. One option is the product release systems adds data in the appropriate directories and formats to archive.mozilla.org and then we fix Buildhub2 to slurp that up. The other option is that we add accounts, API tokens, an API to Buildhub2 so that product release systems can push that information into Buildhub2 when the release is done.

Off the top of my head, growing an API seems the most flexible, but I haven't really given this much thought. This feels like a multi-month project.

I wrote up this issue for it: https://github.com/mozilla-services/buildhub2/issues/542

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #4)

Two things:

First, we have a Buildhub and a Buildhub2. Buildhub is probably going to get decomissioned once we get Buildhub users moved over to Buildhub2. Buildhub2 is at https://buildhub.moz.tools/ .

Thanks for clarifying that.

Second, having said that, the problem is still the same. Both Buildhub and Buildhub2 (and Socorro) pull their build information from archive.mozilla.org from buildinfo.json files generated during the build process and they're only looking at specific information.

Ah, I see. I didn't expect this.

Is Fenix information on archive.mozilla.org? I didn't think it was, but maybe I just don't know where it is.

It is not, to the best of my knowledge. Presumably it will be at some point, because a.m.o backs many things, including our download sites. (I think.)

For products that use different build/release systems than Firefox, we have a couple of options. One option is the product release systems adds data in the appropriate directories and formats to archive.mozilla.org and then we fix Buildhub2 to slurp that up. The other option is that we add accounts, API tokens, an API to Buildhub2 so that product release systems can push that information into Buildhub2 when the release is done.

Off the top of my head, growing an API seems the most flexible, but I haven't really given this much thought. This feels like a multi-month project.

You're right, this does sound like a multi-month project. (I thought this was how buildhub{2} would work now.)

I wrote up this issue for it: https://github.com/mozilla-services/buildhub2/issues/542

Sweet! Given that buildhub{2} is a crawler, not an API itself, this does sound like a much bigger lift than I anticipated. It might be possible to populate a.m.o and then grow dependency information, but that sounds like a strange route to take to get this information. There may be off-the-shelf services that can answer these questions for us... I will try to have a look.

Thanks for the snappy response, :willkg!

Comment 6

[Nick Alexander :nalexander [he/him]]

Just for the record, I added an HG SHA to the GV POM files: see https://bugzilla.mozilla.org/show_bug.cgi?id=1570764#c8.

Comment 7

[Chris Peterson [:cpeterson]]

Tasks and enhancements should have severity N/A.