1184524 - [Go Faster] Q3 Goal Tracking - Prototype a lightweight build, test, and release pipeline for go-faster

Status: RESOLVED WORKSFORME

(Core Graveyard :: Tracking, defect)

Comment 1

[Morgan Phillips [:mrrrgn]]

I've changed the name, since I don't think implementing a pipeline is feasible or wise, when we haven't decided yet what an addon is going to look like. Instead, I'm labeling it as prototype: the idea being that we can give you some degree of certainty that we have proper tools in place to get something shipped once it's available [to be shipped].

Comment 2

[Robert Helmer [:rhelmer]]

I've done some basic prototyping on this and discussed with involved folks - I think that we can use existing infrastructure and not have to build out something new:

1) the Firefox build and add-on packaging happen as part of the build process. The add-on lives in mozilla-central (./browser/extensions/loop/) and is bundled into the Firefox application directory, tests are run by "./mach test", etc.

Here's an example build run through Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=150066c728b3

2) if we want to serve the add-on via addons.m.o, then AMO needs to have a HTTP REST API to allow this. Alternatively, we could simply use the AMO signing service and host the add-on elsewhere (still needs an API)

I'm working through issues with the validator and signing service, for the moment I uploaded an add-on manually that works with the above Try builds:
https://people.mozilla.org/~rhelmer/[e-mail]

3) Our standard update service, Balrog, delivers the set of system add-ons to Firefox. This is where throttling, targeting specific locales/versions/etc. could happen.

As mentioned above, system add-ons do not need to be hosted on AMO (Balrog can point to any URL, as we do with MARs) but they do need to be signed. The build process does not need to block on this.

The advantage to using existing infra is that we have very few blockers (AMO needs an API anyway, and Balrog needs to support delivering the system add-on set), and we can take advantage of infra changes as they roll out (for instance we can move the add-on upload in #2 to a taskcluster job per 1196052, #1 will ultimately be handled by taskcluster, etc.)

I'll round up the dependent bugs for build system/AMO/Balrog, in the meantime if anyone has any questions or concerns let me know! Happy to change course if this isn't workable.

Comment 3

[[e-mail] (:bhearsum)]

(In reply to Robert Helmer [:rhelmer] from comment #2)
> I've done some basic prototyping on this and discussed with involved folks -
> I think that we can use existing infrastructure and not have to build out
> something new:
> 
> 1) the Firefox build and add-on packaging happen as part of the build
> process. The add-on lives in mozilla-central (./browser/extensions/loop/)
> and is bundled into the Firefox application directory, tests are run by
> "./mach test", etc.
> 
> Here's an example build run through Try:
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=150066c728b3
> 
> 2) if we want to serve the add-on via addons.m.o, then AMO needs to have a
> HTTP REST API to allow this. Alternatively, we could simply use the AMO
> signing service and host the add-on elsewhere (still needs an API)
> 
> I'm working through issues with the validator and signing service, for the
> moment I uploaded an add-on manually that works with the above Try builds:
> https://people.mozilla.org/~rhelmer/[e-mail]
> 
> 3) Our standard update service, Balrog, delivers the set of system add-ons
> to Firefox. This is where throttling, targeting specific
> locales/versions/etc. could happen.
> 
> As mentioned above, system add-ons do not need to be hosted on AMO (Balrog
> can point to any URL, as we do with MARs) but they do need to be signed. The
> build process does not need to block on this.
> 
> The advantage to using existing infra is that we have very few blockers (AMO
> needs an API anyway, and Balrog needs to support delivering the system
> add-on set), and we can take advantage of infra changes as they roll out
> (for instance we can move the add-on upload in #2 to a taskcluster job per
> 1196052, #1 will ultimately be handled by taskcluster, etc.)
> 
> I'll round up the dependent bugs for build system/AMO/Balrog, in the
> meantime if anyone has any questions or concerns let me know! Happy to
> change course if this isn't workable.

I haven't seen anything that isn't already supported by Balrog. Dave and I talked last week and I think we can use the exact same structure we use for GMP updates, which returns responses like:
<updates><addons>
    <addon id="gmp-eme-adobe" URL="https://cdmdownload.adobe.com/firefox/win/x86/primetime_gmp_win_x86_beta_28032.zip" hashFunction="sha512" hashValue="e41bd3287ca708d125b6095e196b0e532a2e18a65a5e71df6d1c2e791e977c8992f31251dbb4673e111ea74ea335c90d8139a0e1bf68a6e547f9668923ff513b" size="3678938" version="13"/>
    <addon id="gmp-gmpopenh264" URL="http://ciscobinary.openh264.org/openh264-win32-4adf9cd6dd1358eefe3cbb5eddbfbf4ff06cfa65.zip" hashFunction="sha512" hashValue="c490589b3325bbe8a4179cfb242b639e8ca78b760f7a4ce74d1f4beb3474491206d43495b8fd842efa63154a24f164a1fa2b1c325f1ab29a4292d683f1af5b79" size="319475" version="1.4"/>
</addons></updates>

Comment 4

[Benjamin Smedberg]

We're shipping things now, so I think this bug is beyond its useful lifetime.