Closed Bug 1297519 Opened 3 years ago Closed 3 years ago

Generate multilocale Firefox snaps in beta

Categories

(Release Engineering :: Release Automation: Other, defect)

defect
Not set

Tracking

(firefox51 fixed)

RESOLVED FIXED
Tracking Status
firefox51 --- fixed

People

(Reporter: rail, Assigned: rail)

References

Details

Attachments

(4 files, 1 obsolete file)

Snaps should include all shipped locales. We can probably schedule only nightly CI and release tasks for these (because it would require to have all repacks and XPIs ready).
Assignee: nobody → rail
Summary: Generate multilocale Firefox snaps → Generate multilocale Firefox snaps in beta
Attached patch snapcraft.diff (obsolete) — Splinter Review
The idea is have a docker image and a script to generate snap images.

I put all related files into a single directory. We can either generate an image and publish it to dockerhub, or generate it runtime. 

There will be a task generating a snap and its checksum, a task signing the checksum, and a beetmover task to copy these to the candidates directory.

Snaps will be enabled per branch (another branch config variable passed down to releasetasks).

Something like this.
Attachment #8786073 - Flags: feedback?(jlund)
Do we need to remove the firefox-l10n.js pref file that sets en-US? IIRC, the matchOS pref and the general.useragent.locale need both to be tweaked for matchOS to work.
I tested it and it works without any further customization, LANG=xx_YY /snap/bin/firefox works just fine.
The easiest way to test it is to run the following command and then install the generated snap file:

docker run -e VERSION=49.0b7 -e BUILD_NUMBER=1 \
  -e CANDIDATES_DIR=https://archive.mozilla.org/pub/firefox/candidates \
  -v $(pwd):/snapcraft -v $(pwd)/home:/home \
  -ti rail/snap-dev sh -c "cd /snapcraft && bash runme.sh"

This will generate ./home/worker/artifacts/firexo*.snap, which can be installed by running sudo snap install ./home/worker/artifacts/firexo*.snap
Comment on attachment 8786073 [details] [diff] [review]
snapcraft.diff

from an overall approach perspective, this looks sane to me. ++ to having what you have defined in tree.

we discussed over video of using hg.m.o subdir archive url locations for retrieving this since we only need to grab this once per release.

have you tested this either through tc task creator yet? Locally maybe? Be cool to see inputs/outputs/logs for this.
Attachment #8786073 - Flags: feedback?(jlund) → feedback+
There is nothing using this flags yet, but soon will be. Just need to land these before I can use them in releasetasks.
Comment on attachment 8791825 [details]
Bug 1297519 - Generate multilocale Firefox snaps

The changes in release-runner make sense to me.
Attachment #8791825 - Flags: review?(jlorenzo) → review+
Comment on attachment 8791824 [details]
Bug 1297519 - Generate multilocale Firefox snaps

So do the variable initialization in buildbot-configs
Attachment #8791824 - Flags: review?(jlorenzo) → review+
Comment on attachment 8791825 [details]
Bug 1297519 - Generate multilocale Firefox snaps

https://hg.mozilla.org/build/tools/rev/46ece28c68a5
Attachment #8791825 - Flags: checked-in+
Comment on attachment 8791824 [details]
Bug 1297519 - Generate multilocale Firefox snaps

https://hg.mozilla.org/build/buildbot-configs/rev/4354ce6042f5
Attachment #8791824 - Flags: checked-in+
Comment on attachment 8792039 [details]
Bug 1297519 - Generate multilocale Firefox snaps , a=release DONTBUILD

https://reviewboard.mozilla.org/r/79290/#review77838

I'm not too familair with snap/snapcraft, but the overall logic, while fragile, looks sane [and no less fragile than many of our other tooling in this area].

::: release/docker/firefox-snap/Dockerfile:5
(Diff revision 1)
> +FROM ubuntu:16.04
> +
> +RUN apt-get update -q &&  apt-get install -qy software-properties-common && apt-get clean
> +RUN add-apt-repository ppa:snappy-dev/snapcraft-daily
> +RUN apt-get update && apt-get install -qy snapcraft bzip2 curl && apt-get clean

not as enticed by a ppa for -dev/-daily and then installing that package without any sort of version pinning.

Feels like we'd be asking for trouble on docker regens (since an unrelated change could take a newer unstable snapshot that breaks us).

I won't block on this since we're still getting our feet wet here. :-)

::: release/docker/firefox-snap/Makefile:9
(Diff revision 1)
> +
> +build:
> +	docker build -t $(FULL_IMAGE_NAME) --no-cache --rm .
> +
> +push:
> +	docker push $(FULL_IMAGE_NAME):latest

can we tag each push with some sort of buildID as well, so rollbacks are easy?
Attachment #8792039 - Flags: review?(bugspam.Callek) → review+
(In reply to Justin Wood (:Callek) from comment #15)
> not as enticed by a ppa for -dev/-daily and then installing that package
> without any sort of version pinning.

It needed this at some point to get latest snap/snapcraft. I can check if Ubuntu got the updates in the main repo, so we can get rid of the PPA.

> can we tag each push with some sort of buildID as well, so rollbacks are
> easy?

We won't be using tags in any case (we use sha256). Pushing :latest is just faster (no need to push everything).
https://reviewboard.mozilla.org/r/79290/diff/1-2/ addresses the PPA comment
Pushed by raliiev@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4606672d8efe
Generate multilocale Firefox snaps r=Callek, a=release DONTBUILD
https://hg.mozilla.org/mozilla-central/rev/4606672d8efe
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Forgot the magic words
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Attached file PR for releasetasks
Attachment #8794390 - Flags: review?(bugspam.Callek)
Attachment #8786073 - Attachment is obsolete: true
Comment on attachment 8794390 [details] [review]
PR for releasetasks

r+ed in the PR
Attachment #8794390 - Flags: review?(bugspam.Callek)
Attachment #8794390 - Flags: review+
Attachment #8794390 - Flags: checked-in+
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.