1464807 - Investigate whether we should upload GeckoView on an S3 bucket

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Uploading, enhancement)

Description

[Johan Lorenzo [:jlorenzo]]

The signing plan defined in bug 1405129 is little bit fragile in term of signature verification. We may want to control the maven-like server instead. Let's investigate how hard it is to provide all the files required by maven/gradle. A good starting point: https://blog.packagecloud.io/eng/2017/03/09/how-does-a-maven-repository-work/#maven-metadataxml

Comment 1

[Johan Lorenzo [:jlorenzo]]

This is fairly doable in a reasonnable amount of time. 


1. TESTING PROTOCOL

In one terminal, prepare the maven-like server:
> mkdir my-maven-repo
> cd my-maven-repo
> python -m http.server 8000

In another terminal, make a small gradle project, put your own repo as the only one:

> allprojects {
>     repositories {
>         maven {
>             url "http://localhost:8000/maven2"
>         }
>     }
> }

Then build the gradle project.

For each missing dependency (either a pom or a jar), download it from one of these repo:

> GOOGLE_PATH='https://dl.google.com/dl/android'
> JCENTER_PATH='http://jcenter.bintray.com'
> MAVEN_CENTRAL_PATH='http://central.maven.org'

and place under the same folder layout. Rebuild the project and see gradle now complaning about the missing deps anymore. If your testing project has too many underlying deps (like mine), you can put jcenter as a fallback repo:

> allprojects {
>     repositories {
>         maven {
>             url "http://localhost:8000/maven2"
>         }
>         jcenter()
>     }
> }


2. RESULTS

I didn't have too much to do to set up a functional repo. Gradle did fetch the deps from my custom-made repo (per the logs of the http server). Python's http server is simple enough to be fairly certain it doesn't contain any maven fancy feature. However, it allows to list documents in directories, which I think S3 doesn't do by default. 


3. UNTESTED ELEMENTS

Even though I defined it, I don't think the super pom.xml worked. It's documented in the blob posted in comment 0. Nonetheless, I don't see it on http://central.maven.org/maven2/. I don't think we need it though. As far as I understand, this is to define a proxy to another repo. I don't see us willing to do so.

The Snapshots haven't been tested either. This part worries me a little bit because it can hide race conditions dependending on how we use it.
If we host maven-metadata.xml on S3, then Beetmover has to download the latest version of this file, edit it, and upload it back to S3. Let's imagine we have more than 1 geckoview to upload at the same time, we may end up with a corrupted maven-metadata.xml. 

Sebastian, do you know how often we want to upload a SNAPSHOT of geckoview or an Android component?

Comment 2

[Sebastian Kaspari (:sebastian; :pocmo)]

> Sebastian, do you know how often we want to upload a SNAPSHOT of geckoview or an Android component?

Sorry for the late reply. I look rarely in Bugzilla these days. Please just ping me additionally on IRC/Slack if I'm too slow.

Ideally we'd have a Nightly release for GeckoView (whether we call it a snapshot or not is something that we can define..) and a snapshot release of the components for every master push. This would allow us to test landed patches in apps with a snapshot release immediately.

Comment 3

[Johan Lorenzo [:jlorenzo]]

I tested a maven repo on top of an S3 bucket: this is doable!


:catlee had set up [1]. This is an upload of the artifacts of a task like [2]. 

> 3. UNTESTED ELEMENTS
> 
> Even though I defined it, I don't think the super pom.xml worked.
> I don't think we need it though.

No super pom.xml has ever been defined in the S3 bucket and gradle remained happy. Let's not bother with it.


> The Snapshots haven't been tested either.

I chatted with Sebastian on Friday. We don't think SNAPSHOT are a use case we want to support. The reason is: SNAPSHOTs allow to keep following the latest build of a given version. GeckoView has always been pinned to a single version (for instance [3]). This enables reproducible builds, which we aim to have. Therefore, we don't have to support SNAPSHOTs.

Sebastian also suggested to have different artifactIDs per Gecko branch: 
> 16:04:44 UTC <•sebastian> Pro different Ids:
> 16:04:44 UTC <•sebastian> 1) I browse the repository to see what the latest versions are: http://jcenter.bintray.com/org/mozilla/components/feature-search/
> 16:04:44 UTC <•sebastian> 2) You can do things like (even though it is discouraged) ...geckoview:62.* to get the latest 62 release build. That may break if there are other versions under the same artifact id

So let's have: 
* geckoview-${platform}
* geckoview-beta-${platform}
* geckoview-nightly-${platform}

${platform} being (at the moment): arm64-v8a, armeabi-v7a, and x86.

Thanks to this split, version numbers for Nightly packages can be like 63.0.20180625120000 (major.minor.buildId). This notably avoids to compare a nightly (63.0.20180625120000) and a release build (63.0) - the former being considered more recent by regular semantic versioning.


> This part worries me a little bit because it can hide race conditions
> depending on how we use it.

:catlee found a solution to this. We can use an Amazon Lambda function. Everytime somebody/something uploads a new geckoview artifact, the S3 function would be triggered and regerate the metadata. I wrote [4] to regenerate maven-metadata.xml[5] and its checksums. It basically looks at the content of the directory and gets data from it. This saves us from race conditions as the last run will get the data uploaded from every previous runs. Thus, we can upload any artifact at any pace.

We just have to figure out how to maintain this (currently simple) function and how to deploy it. Worst case scenario, AWS Lambda already support version control.



[1] http://geckoviewtest.s3-website-us-west-2.amazonaws.com/
[2] https://tools.taskcluster.net/groups/ZeVkONGGR8-6VBGYs152-w/tasks/ewbtOCGHTTui6mI_xMeOpg/runs/0/artifacts
[3] https://github.com/mozilla-mobile/focus-android/blob/95e9841e1dbceb935f12623ce879f3568bfeffa2/build.gradle#L10
[4] https://us-west-2.console.aws.amazon.com/lambda/home?region=us-west-2#/functions/populateMavenMetada/versions/$LATEST?tab=graph
[5] http://geckoviewtest.s3-website-us-west-2.amazonaws.com/org/mozilla/geckoview-beta-x86/maven-metadata.xml