1588668 - Package multi-locale Lightning at build-time for upload to ATN

Status: RESOLVED FIXED

(Calendar :: Build Config, task)

Description

[Geoff Lankow (:darktrojan)]

I want Lightning packaging happening as a build-time task so that pulling localised artifacts across a slow network isn't necessary. Note this bug doesn't include uploading to ATN, which has been discussed.

Comment 1

[Geoff Lankow (:darktrojan)]

Attachment: 1588668-lightning-repack-1.diff

Here's what I just ran on Try to get a build of Lightning 68.1.2 (sans ja-JP-mac, which I added manually due to problems with the Mac L10n task). I'm going to push the same thing now on comm-central to check I haven't broken it there since I first wrote it.

It's not perfect and I expect there's probably things to complain about. Right now it's set to not run unless somebody asks for it (run-on-projects: []) but it could run on any build that already does L10n for little cost.

Comment 2

[Geoff Lankow (:darktrojan)]

I've put the line to extract the ja-JP-mac .dmg in the wrong place, but apart from that all still works fine on comm-central.

Comment 3

[Óvári]

https://github.com/flathub/org.mozilla.Thunderbird/issues/88

Comment 4

[Rob Lemley [:rjl]]

Comment on attachment 9101022 [details] [diff] [review]
1588668-lightning-repack-1.diff

Review of attachment 9101022 [details] [diff] [review]:
-----------------------------------------------------------------

Nothing major jumps out at me. Out of curiosity, do you run your bash scripts through shellcheck? I plan to enable those tests at some point.

::: calendar/lightning/repack/all-locale-repack.sh
@@ +1,1 @@
> +# This Source Code Form is subject to the terms of the Mozilla Public

#!/bin/bash

@@ +26,5 @@
> +for l in *
> +do
> +  if [[ $l == ja-JP-mac ]]
> +  then
> +    prepath='Thunderbird/Thunderbird.app/Contents/Resources'

This might not work with Daily builds. IIRC it gets named "Thunderbird Daily.app'

@@ +80,5 @@
> +python3 "$GECKO_PATH/comm/calendar/lightning/repack/modify-manifest.py"
> +touch -d 20100101 manifest.json
> +zip lightning.xpi manifest.json
> +
> +mkdir ../artifacts

$UPLOAD_DIR maybe?

::: taskcluster/comm_taskgraph/__init__.py
@@ +145,5 @@
>  
>          yield job
> +
> +
> +def add_l10n_dependencies(config, jobs):

Can you add a documentation comment please?

Comment 5

[Geoff Lankow (:darktrojan)]

(In reply to Rob Lemley [:rjl] from comment #4)

Nothing major jumps out at me. Out of curiosity, do you run your bash
scripts through shellcheck? I plan to enable those tests at some point.

I had no idea that was a thing but I like it.

• prepath='Thunderbird/Thunderbird.app/Contents/Resources'

This might not work with Daily builds. IIRC it gets named "Thunderbird
Daily.app'

You're right. Now I need to figure out how to know if we're building Daily.

Comment 6

[Geoff Lankow (:darktrojan)]

Attachment: 1588668-lightning-repack-2.diff

With changes. Setting r? again for peace of mind.

Comment 7

[Geoff Lankow (:darktrojan)]

Comment on attachment 9102020 [details] [diff] [review]
1588668-lightning-repack-2.diff

Scratch that. The ancient version of 7z on the builders fails to extract the .dmg.

Comment 8

[Geoff Lankow (:darktrojan)]

Attachment: 1588668-lightning-repack-3.diff

Now using Mozilla's dmg tools because the version of 7z on the builders is ancient and can't handle what I wanted it to.

Comment 9

[Jorg K (CEST = GMT+2)]

You want that backported? To TB 68 perhaps?

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/0618eba9d869
Package multi-locale Lightning at build-time for upload to ATN. r=rjl DONTBUILD

Comment 11

[Geoff Lankow (:darktrojan)]

Comment on attachment 9102033 [details] [diff] [review]
1588668-lightning-repack-3.diff

Yes I do. Bug 1588659's got to go with it.

Comment 12

[Jorg K (CEST = GMT+2)]

TB 68.2.0 ESR / Cal 7.0:
https://hg.mozilla.org/releases/comm-esr68/rev/e6946b1a24880faee6831cc74c9d438f562185fc

Comment 13

[Geoff Lankow (:darktrojan)]

The pushed patches have lost the file mode of all-locale-repack.sh. The very first line should say new file mode 100755, as I posted them, not new file mode 100644.

Comment 14

[Jorg K (CEST = GMT+2)]

Maybe a Windows thing? Can you push a fix with DONTBUILD?

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/c6e68117d8ac
follow-up - Reapply correct file mode to all-locale-repack.sh; rs=bustage-fix DONTBUILD

Comment 16

[Jorg K (CEST = GMT+2)]

Do we need that on the branches?

Comment 17

[Geoff Lankow (:darktrojan)]

Yes.

Edit: I've done this:
https://hg.mozilla.org/releases/comm-beta/rev/1f79b263bbbb5adb59effdb52a517646a33ff41a
https://hg.mozilla.org/releases/comm-esr68/rev/299add060019cc07203c878c98b430a7f6bffa49

Comment 18

[Jorg K (CEST = GMT+2)]

After bug 1576707 landed, and with the typo fixed that they landed (bug 1576707 comment #15), we get:
Exception: addon-calendar-l10n can't fetch toolchain-linux64-libdmg artifacts because there are no tasks with label toolchain-linux64-libdmg in kind dependencies!

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/7c4a9a0edb69
Follow-up: Add missing toolchain kind dependency required after bug 1576707. r=me DONTBUILD

Comment 20

[Geoff Lankow (:darktrojan)]

Well that was unfortunate timing!