Closed Bug 1490598 Opened 6 years ago Closed 10 months ago

Enable automatic language pack upload to ATN via Taskcluster.

Categories

(Thunderbird :: Add-Ons: General, defect)

defect
Not set
major

Tracking

(thunderbird_esr60+, thunderbird_esr102 wontfix, thunderbird115 fixed)

RESOLVED FIXED
116 Branch
Tracking Status
thunderbird_esr60 + ---
thunderbird_esr102 --- wontfix
thunderbird115 --- fixed

People

(Reporter: franz.trischberger, Assigned: rjl)

References

(Blocks 1 open bug, )

Details

(Keywords: intl, regression, Whiteboard: [regression:TB60])

User Story

https://support.mozilla.org/en-US/questions/1231936
https://support.mozilla.org/en-US/questions/1241978
https://support.mozilla.org/en-US/questions/1240358
https://support.mozilla.org/en-US/questions/1240455

Attachments

(2 files, 3 obsolete files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
Build ID: 20180830143136

Steps to reproduce:

I tried to install Deutsch Language Pack.


Actual results:

It does not work (install button deactivated) as the Language Pack is only compatible with 52.0 and not the most recent 60.0 Release.


Expected results:

The Deutsch Language Pack should be available for 60.0, too (which was released more than a month ago).
This affects all language packs by Mozilla on addons.thunderbird.net. The German one can be found at https://addons.thunderbird.net/en-US/thunderbird/addon/deutsch-de-language-pack-thund/

Andreas, do you know whom to talk to to upload the new language packs.

Franz, for now you can download the language packs from http://ftp.mozilla.org/pub/thunderbird/releases/60.0/win64/xpi/ de.xpi is German. Right-click and save the links.
Status: UNCONFIRMED → NEW
Component: de / German → Add-Ons: General
Ever confirmed: true
Flags: needinfo?(awagner)
Product: Mozilla Localizations → Thunderbird
Summary: Thunderbird-60.0: Deutsch Language Pack not compatible → Language packs on addons.thunderbird.net not updated for Thunderbird 60
Sebastian: Thx for the reply.
I tried to get it working with the linked de.xpi. Still all in English. Tried it.xpi, same issue.
They get copied to default_profile/extensions and renamed (looks like in firefox) but apparently not used.
It is a different issue, I think. Do you know if it is already reported?
How can I debug this issue?
You still need to manually activate the language pack:

1. Go to Thunderbird's Options.
2. Go to "Advanced" > "General"
3. Open the "Config Editor" and confirm the warning.
4. Right-click the list and choose "New" > "String".
5. Use as name intl.locale.requested and set the value to de
6. Close Thunderbird and launch it again.
Thank you very much! That did the trick.
And sorry for the dumb questions. Never had to use the advanced config editor before.
Flags: needinfo?(awagner)
Flags: needinfo?(philipp)
This is a little weird. I've fixed the issue preventing web extensions from uploading to addons.thunderbird.net, but they can't be approved right now because addons-server tries to sign them, and we don't have a working signing server nor do I think we can even have one.

I don't know if I can just disable signing in the code or if that will produce language packs that won't enable by default or what.
I got this to upload by disabling signing, but addons-server only recognizes web extensions as compatible with Firefox by default, so you have to manually force compatibility for Thunderbird on the versions page (https://addons.thunderbird.net/en-US/developers/addon/deutsch-de-language-pack-thund/versions/2468594) otherwise it won't be installable for Thunderbird and in fact the page will break because it tries to redirect you to AMO, since the addon is only compatible with Firefox, creating a circular redirect.
Also, all the versions have a buildid string in them, which seems less than ideal. "Version 60.0buildid20180731173940" is what is displayed after upload, rather than Version 60.0.
Thx Andrei! The language pack is available now and auto update from 52 to 60 worked.
Unfortunately the GUI is still in English. I have to manually set intl.locale.requested in the advanced config editor.
Is it normal to have to manually enable these? I don't really know how they're supposed to work.
Before the 60.0 switch it usually was enough to install and enable the language pack addon.
So for Firefox, there is some releng magic that creates and uploads the language packs from automation, hence the 60.0buildid20180731173940 version number. We may have to do something like this as well. ni? rjl to figure this out.

It would also be great if on addons.thunderbird.net we could assume that WebExtensions are (only) compatible with Thunderbird by default.
Flags: needinfo?(philipp) → needinfo?(rob)
(In reply to Philipp Kewisch [:Fallen] [:📆] from comment #11)
> It would also be great if on addons.thunderbird.net we could assume that
> WebExtensions are (only) compatible with Thunderbird by default.

If we do this, it does mean that you will be able to upload any Firefox web extension, it will be auto approved and available for download/install, but probably won't work. Maybe that's not a big deal, but it is a regression from legacy extensions, because Web Extensions don't have any application compatibility information beyond "gecko". I dunno if this is intentional, if we're supposed to be guessing the application compatibility based on the permissions and chrome_resources used by the extension, but it seems like it'd be a lot easier if we could just require addiong Thunderbird to the application list for all TB Web Extensions manifests. Maybe that goes against the design intention though, I don't really understand how this was supposed to work.
I took a real quick look, and there are some taskcluster tasks defined for Firefox that deal with signing and uploading langpack addons:
release-sign-and-push-langpacks
release-beetmover-signed-langpacks
release-beetmover-signed-langpacks-checksums

Those would need to be ported over to Thunderbird. Additionally, new scopes will need to be created within Taskcluster to support uploading to addons.thunderbird.net.
Flags: needinfo?(rob)
Severity: normal → major
Keywords: intl, regression
Whiteboard: [regression:TB60]
Counting up some of what I am guessing are the most used languages, I get 600,000 users. We failed to track this for release. So we need to prioritize fixing this. Rob, how long might this take and can it go to the top of your list?
Assignee: nobody → rob
Flags: needinfo?(rob)
I don't have a good sense of how long this will take unfortunately. A lot depends on ATN and what it's capabilities are, and at the moment it's a black box for me. I'll try to pull together what needs to be done. I'll likely need input/help from :sancus.
In the short-term, can we set up a knowledgebase article or wiki page with the alternate install instructions described earlier in this bug to grab the XPI from the ftp site and install that way? (See comment 1 and comment 3) I realize it's not ideal, but we can at least get people going that way.
Flags: needinfo?(rob)
(In reply to Wayne Mery (:wsmwk) from comment #15)
> Counting up some of what I am guessing are the most used languages, I get
> 600,000 users.
Wayne, can you please clarify this. What are these language packs used for? Everyone can download a localised version of TB with the language pack already installed. In TB 60 it's quite cumbersome to activate an installed language pack or switch between them. Somehow I'm missing the urgency of this.
I'm not an intl user, so I have no clue to answer either question of what and urgency. But there are two bug reporters who can be asked, plus whoever is in SUMO such as https://support.mozilla.org/en-US/questions/1240358

Perhaps there is a lot of overlap, or many users don't care. Cut 600k in half and it still a high percentage of users, so this at least deserves a strong assessment by us.
Even if we fix this, those 600.000 users will then run into bug 1437480. They'll need to set pref "intl.locale.requested". Exposing this in the UI has only arrived in TB 63 in bug 1481852 which we can't backport.
The language packs are often used for:
- portable version of TB
- Linux distros if they do not provide langpack as a package in their repository
- if someone installs English version of TB and consider installing different version not-safe or too difficult
- or if they simply do not know it is possible (usually other sw allows to switch languages seamlessly compared to TB).
(In reply to Rob Lemley [:rjl] from comment #16)
> I don't have a good sense of how long this will take unfortunately. A lot
> depends on ATN and what it's capabilities are, and at the moment it's a
> black box for me.

Firefox has a job that does this for AMO, so you should be able to copy a lot from there. Essentially there is a user that does the upload via the API, and the secret is stored with the others. Hope it helps!
> Firefox has a job that does this for AMO,  

Back to Rob :)
Flags: needinfo?(rob)
User Story: (updated)
I will see what I can come up with.
Flags: needinfo?(rob)
There is work being done on the Firefox langpack addons process in bug
1566298, so this is an opportune time to get these working on Thunderbird,
uploading to addons.thunderbird.net.

First pass at porting the Taskcluster kind files from Firefox.
Comment on attachment 9078323 [details] [diff] [review]
tb_langpack_ATN_taskcluster.patch

Note: This reflects the more-recent work I'm doing here -- however it does not yet work (I need to get autograph support for a new signing user that can sign addons, AND I need to modify the AMO submission user to be @mozilla.com -- you may want to chat with me before implementing :-)

Thanks Rob. Great to see this making progress. Intl issues sometimes don't get enough attention.

Version: unspecified → 60
Summary: Language packs on addons.thunderbird.net not updated for Thunderbird 60 → Language packs on addons.thunderbird.net not updated for Thunderbird 60/68
Version: 60 → 68
Blocks: 1577142

Did someone manually upload German (Deutch) language pack for TB60? All others I've checked are still compatible with TB52. If this bug can't be easily fixed, I'm willing to add all language packs to ATN, if someone gives me the rights to upload them…

Now that all the language packs that are available for TB 68 have been manually uploaded and are available on ATN (thank you sancus!), maybe also multilingual can be enabled? I can confirm that if I toggle intl.multilingual.enabled and intl.multilingual.downloadEnabled to true, I can search for new languages in TB68, I can download them and I can switch to them!

See Also: → 1590206

It was enabled in bug 1590206.

Not sure if the languages pack upload are broken again, or never worked completely, however at least the Czech language pack is missing on ATN, see https://addons.thunderbird.net/thunderbird/language-tools/. IIRC it's supposed to be here https://addons.mozilla.org/thunderbird/addon/czech-cz-language-pack-thunder/.

Czech is definitely the only one, Slovak is missing too. Both are available on FTP https://archive.mozilla.org/pub/thunderbird/releases/68.8.0/linux-x86_64/xpi/.

Getting this out of my queue until ATN is able to support this automatically.

Assignee: rob → nobody

Still the same situation - Thunderbird 78. In the options there is no possibility to add another language. When trying to search for new language, nothing appears. I am interested in Czech version.

Until they are uploaded to ATN, you can download them from https://archive.mozilla.org/pub/thunderbird/releases/78.2.1/linux-x86_64/xpi/. (78.2.1 is the latest at time of comment).

Assignee: nobody → justdave

If someone can catch me up real quick here, it looks (from comment 24) like this actually got implemented at one point, but didn't actually work? Or did that not actually get committed? Is it worth building on top of that and debugging it or should I start over copying what Firefox does now and trying to port that over?

Flags: needinfo?(rob)

(In reply to Dave Miller [:justdave] (justdave@thunderbird.net) from comment #35)

If someone can catch me up real quick here, it looks (from comment 24) like this actually got implemented at one point, but didn't actually work? Or did that not actually get committed? Is it worth building on top of that and debugging it or should I start over copying what Firefox does now and trying to port that over?

I would start with what Firefox has today. I don't think the work I had based that patch on got finished, or else has changed significantly since I did that.

Flags: needinfo?(rob)
Summary: Language packs on addons.thunderbird.net not updated for Thunderbird 60/68 → Language packs on addons.thunderbird.net not updated for Thunderbird 60/68/78

FYI over the next couple weeks there will be significantly more interest from users because automatic updates will be resuming, hopefully in large numbers.

Hi I just updated TB in Debian Buster to 78.3.1 (that's the version that's now in Debian Buster's package repository). That installed also
the package thunderbird-l10n-de 1:78.3.1-2~deb10u2
for German language support. But TB is in English, and German language is not selectable under preferences menu.

Then I went to TB Addons, there was written the installed German language pack would not be compatible with TB 78. Then I clicked "manage"->add from file and selected /usr/share/thunderbird/extensions/langpack-de@thunderbird.mozilla.org.xpi from the Debian package.
Now I have German language in TB 78, it says updated last 2020-10-07 so today. And the new OpenPGP strings are translated, so it is in the Debian package, but was not updated automatically.

There is no need to download the xpi file in this case, it is in the Debian package correct, but the language is not updated automatically what I would have assumed.

We indeed don't have them on the addons server yet (soon!) but if they're pushing it out as a package anyway then the failure to use it sort of sounds like a package bug? If it's something in TB it might be worth filing a separate bug for that, but I'd have the Debian maintainers make sure it's not a bug in the packaging first.

See Also: → 1670435

The language packs that exist on ATN have been updated to their 78 versions respectively. However, the taskcluster automation is not yet finished, so I'm changing this bug to refer to that since it has some relevant history etc.

There is also some cleanup of Language Pack listings necessary. Since that is a website issue, it will be tracked here: https://github.com/thundernest/addons-server/issues/152

Summary: Language packs on addons.thunderbird.net not updated for Thunderbird 60/68/78 → Enable automatic language pack upload to ATN via Taskcluster.

(In reply to Andrei Hajdukewycz [:sancus] from comment #43)

The language packs that exist on ATN have been updated to their 78 versions respectively.

That turned out to be that only the ones that already had listings on ATN had gotten updated. I just went through and created listings for all the ones Mozilla builds that weren't there and uploaded them, so they should all be there for real now.

Did the automation get finished?

Assignee: justdave → rob
Assignee: rob → daniel

The script currently used for this is here: https://gist.github.com/Sancus/58c7a4c5174eab3b513f7a9c876a17b5

It contains the code that handles the annoying parts of interacting with the ATN upload API(the token authorization, basically).

Attachment #9250839 - Attachment is obsolete: true
Attachment #9250845 - Attachment is obsolete: true
Attachment #9078323 - Attachment is obsolete: true

Pushed by elizabeth@thunderbird.net:
https://hg.mozilla.org/comm-central/rev/81a318158885
ATN Uploader docker image. r=dandarnell
https://hg.mozilla.org/comm-central/rev/192ec4650383
Upload langpacks to ATN on release. r=dandarnell

Status: ASSIGNED → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED

Comment on attachment 9338372 [details]
Bug 1490598 - Upload langpacks to ATN on release. r=#thunderbird-build-system-reviewers

[Approval Request Comment]
Upload langpacks to ATN automagically on release. Both patches.

Attachment #9338372 - Flags: approval-comm-beta?

Comment on attachment 9338371 [details]
Bug 1490598 - ATN Uploader docker image. r=#thunderbird-build-system-reviewers

[Triage Comment]
Approved for beta - assuming you also want this

Flags: needinfo?(rob)
Attachment #9338371 - Flags: approval-comm-beta+

Comment on attachment 9338372 [details]
Bug 1490598 - Upload langpacks to ATN on release. r=#thunderbird-build-system-reviewers

[Triage Comment]
Approved for beta

Attachment #9338372 - Flags: approval-comm-beta? → approval-comm-beta+
Flags: needinfo?(rob)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: