855067 - Localized Lightning builds using Thunderbird Infrastructure

Status: RESOLVED FIXED

(Calendar :: Build Config, defect)

Description

[Philipp Kewisch [:Fallen][📱📆]]

This bug is to track setting up l10n builds on the Thunderbird infrastructure, which will ultimately mean localized releases.

I need to figure out the next steps right now, one of them will be finally continuing on bug 812299.

Comment 1

[Philipp Kewisch [:Fallen][📱📆]]

Looks like I have something that works. Tryserver builds running at https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=20766daedb7b

Comment 2

[Philipp Kewisch [:Fallen][📱📆]]

Seems windows doesn't like pattern rules and normal rules together on the left side, new try run: https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=99dda64cc7b3.

The other builds completed normally. The warning are probably just because check-sync-dirs complains that the mozilla mozconfig.common doesn't include --enable-calendar

Comment 3

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: Fix - v1

Here we go, this takes care!

Comment 4

[Mark Banner (:standard8)]

Comment on attachment 730658 [details] [diff] [review]
Fix - v1

Do we know roughly how long the repack step takes? I'm wondering if we should (and are able to) do it on nightly/release builds only, to save a bit of builder time.

It would mean a bit of difference between dep and nightly builds, but I think I'd prefer that to taking up unnecessary time on the dep builds.

Also adding John here to get releng's feedback on these thoughts.

Comment 5

[Philipp Kewisch [:Fallen][📱📆]]

The repackage for one locale goes pretty quickly, here are the times on my machine:

make wget-en-US unpack:
real	0m1.128s
user	0m0.316s
sys	0m0.197s

make clobber-de langpack-de repackage-zip-de:
real	0m3.086s
user	0m1.981s
sys	0m0.751s

Some extra time for uploads of course.

Comment 6

[Philipp Kewisch [:Fallen][📱📆]]

Its totally fine for me to do this on nightly/release builds only, I don't see a need for each dep build to be localized.

Joey, whats your take? Can you tell us if its possible to do so?

Comment 7

[Mark Banner (:standard8)]

So actually, can I check what this is doing (not having tested yet)? Is it repacking all lightning locales into the lightning build, or is it just doing the langpack-* targets and producing lightning-ab-CD.xpi builds?

Comment 8

[Philipp Kewisch [:Fallen][📱📆]]

This bug is about the langpack-* targets only. There is no step I can hook into where it would make sense to repack all locales. My idea is to run an extra builder or script that does nothing other than download all locale builds and package them to one all-locales xpi.

A new patch is coming up that will allow l10n repacks to also work on release builds.

Comment 9

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: Fix - v2

This is it. I have tested it on stage, modulo some cleanup in lightning-packager.mk and the moz.build changes needed for this to apply on comm-central.

With this patch we should be able to:

* Build nightly,beta,release builds of Lightning and upload them correctly
* Create one-locale-per-xpi builds of respective builds uploaded correctly

As mentioned, the merging will happen in a different bug. The build time hasn't changed notably from what I mentioned before.

My plan is this:

1) Land on comm-central, make sure l10n and builds still work
2) Backport to comm-aurora, this should work just as well as for central
3) Backport to comm-beta when a beta build is being created, I will be there to fix things in case of unexpected problems
4) I will also be there when the release builds are created.

If possible, I'd appreciate if we could land this on central/aurora soon (this week?). Maybe we can manage building the first or second beta and the next release with this patch?

Comment 10

[Mark Banner (:standard8)]

Comment on attachment 749294 [details] [diff] [review]
Fix - v2

Review of attachment 749294 [details] [diff] [review]:
-----------------------------------------------------------------

Ok, this generally looks fine, I can't see any issues in it, but I've not gone through in depth.

::: mail/config/mozconfigs/linux64/l10n-mozconfig
@@ +5,5 @@
>  ac_add_options --enable-profiling
>  ac_add_options --with-l10n-base=../../l10n
>  
> +# Build lightning locales
> +ac_add_options --enable-calendar

Just a side note, this won't be enough for release/beta channels until bug 758149 is fixed.

Comment 11

[Philipp Kewisch [:Fallen][📱📆]]

(In reply to Mark Banner (:standard8) from comment #10)
> Ok, this generally looks fine, I can't see any issues in it, but I've not
> gone through in depth.
Great! So whats the next step? We should probably coordinate to make sure I am around when the builds are done in case something goes wrong.

> > +# Build lightning locales
> > +ac_add_options --enable-calendar
> 
> Just a side note, this won't be enough for release/beta channels until bug
> 758149 is fixed.
Yes, known. buildbot-configs patch something like this: http://hg.mozilla.org/users/jhopkins_mozilla.com/buildbot-configs/rev/8f650f65b14f

Comment 12

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 749294 [details] [diff] [review]
Fix - v2

As per IRC discussion with Standard8, I am landing this on beta first. I will follow up aurora and central in the beginning of next week when the builds succeed.

https://hg.mozilla.org/releases/comm-beta/rev/c177c5ec418

Comment 13

[Philipp Kewisch [:Fallen][📱📆]]

Sorry, missing character on that URL: 

https://hg.mozilla.org/releases/comm-beta/rev/c177c5ec418d

Comment 14

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: buildbot-configs mozconfig changes - v1

Here are the mozconfigs for beta/release

Comment 15

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 752191 [details] [diff] [review]
buildbot-configs mozconfig changes - v1

https://hg.mozilla.org/build/buildbot-configs/rev/940bc528bc35

Comment 16

[Armen [:armenzg]]

Live in production.

Comment 17

[Philipp Kewisch [:Fallen][📱📆]]

Backported to releases/comm-beta changeset 3d7d2af9a03d

Comment 18

[Philipp Kewisch [:Fallen][📱📆]]

Sorry, last comment was from my autopush script. Reopening until this is green and checked in everywhere.

Comment 19

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 749294 [details] [diff] [review]
Fix - v2

I'm going to go ahead and push this to c-c and when thats green also c-a since the next merge is coming up. It seems to have worked fine on beta!

Comment 20

[Philipp Kewisch [:Fallen][📱📆]]

https://hg.mozilla.org/comm-central/rev/05d059079026

Comment 21

[Philipp Kewisch [:Fallen][📱📆]]

c-c builds are green, I have now transplanted to aurora:

https://hg.mozilla.org/releases/comm-aurora/rev/1eda7c30ee58