422636 - Assign target locales to dictionaries and language packs

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Administration, defect)

Description

[Fred Wenzel [:wenzel]]

While the current target locales for dictionaries were imported, this was not possible for language packs, so in order to make the language support page look and work as expected, the following steps are necessary:

- consolidate dictionaries' target locales so they show up under the same language name where appropriate, and are disambiguated by the additional description field only
- identify language packs that should show up, change their addon type to ADDON_LPAPP (= 5) if necessary (requires SQL, so we should make a list of addon ids, then run some SQL)
- assign target locales (and possibly additional info) to these language packs
- afterwards, verify that every line on the language tools page has an English and local name assigned, fix the missing ones, if necessary by having clouserw augment the global product details arrays for moz.com.

Comment 1

[Fred Wenzel [:wenzel]]

Basil, per bug 416779 comment 3, could you help with this if you find time? The main thing to be done is identifying the language packs that are supposed to show up and determining their target locales, the rest is relatively minor cleanup.

A wiki page or so would be great, I can help boil that down to an SQL query afterwards.

Thanks!

Comment 2

[Basil Hashem [:baz]]

OK, I've added a wiki page with the content you require Fred. I have the contents of this page as an Excel document so if you need me to send you just a column or some other variation, let me know. It's pretty easy.

http://wiki.mozilla.org/Update:LanguagePacks

Comment 3

[Fred Wenzel [:wenzel]]

Attachment: SQL to fix language packs according to wiki page

Thanks, Basil. That was very helpful. I made a bunch of SQL queries from the wiki page, and it works well. We need to have it run on production (preview uses the same database, right?) before we can proceed with the rest of the steps here.

Comment 4

[Fred Wenzel [:wenzel]]

Attachment: Language packs together with dictionaries

This is what it looks like now; note that only very few lang packs show up in the same line as a dictionary, which is why we need to put the ones together that belong together after the SQL fix lands.

Comment 5

[Fred Wenzel [:wenzel]]

Oremj, could you run some SQL for us again? attachment 309392 [details] from comment 3 will fix the addon types for a bunch of language packs. Thanks!

Comment 6

[Wil Clouser [:clouserw]]

(In reply to comment #5)
> Oremj, could you run some SQL for us again? attachment 309392 [details] from comment 3
> will fix the addon types for a bunch of language packs. Thanks!
> 

the target_locale column hasn't been added in production yet so this won't work. :(

Comment 7

[Fred Wenzel [:wenzel]]

Argh, dang it. I forgot about that. Looks like we need to do that first. I guess I put this SQL on the "to be run" list on the wiki then and we should determine a date for these and the other changes to be run?

Comment 8

[Fred Wenzel [:wenzel]]

All right, bug 423188 has landed.

Basil, if you want to help a) combining similar dictionaries under the same hood and b) help identify the ones whose names are still wrong/missing, that'd be appreciated, otherwise I'll go ahead and try my best to make these decisions.

Luckily, we don't need SQL run this time.

Comment 9

[Basil Hashem [:baz]]

In order to test more, I need to move more langpacks from the sandbox but am blocked on bug 423901. I'll try to move the lang packs on production out of the sandbox but will still a db refresh on preview in order to see the changes.

Here are the entries that need to get renamed:
1) Corrector de Galego/España (gl-ES) ->
   English Name: Galician
   Native Name: Galego (gl-ES)
2) Danish Dictionary (danish)
   English Name: Danish
   Native Name: Dansk
3) Eesti keele pakk (et) & Eesti keele speller (et-EE) need to be combined
   English Name: Estonian
   Native Name: Eesti keele
4) Furlan Language Pack (fur-IT)
   English Name: Friulian
   Native Name: Furlan
5) Germany entries, need to look like English (*) entries
   German (Austria)     Deutsch (de-AT)  	
   German (Germany)     Deutsch (de-DE)
   German (Germany)     Deutsch (de-DE-alt)
   German (Germany)     Deutsch (de-DE-comb)
   German (Switzerland)	Deutsch (de-CH)
6) Latin Dictionary (la)
   English Name: Latin
   Native Name: Latina (la)
7) Srpski (srpski-LATINICA)
   English Name: Serbian
   Native Name: Srpski (sr)
8) Svensk ordlista (sv)
   English Name: Swedish
   Native Name: Svenska (sv)
9) Tatarça Tíl Paketí (İQTElif) [Tatar Language Pack] (tt-RU)
   English Name: Tatar
   Native Name: Tatarça (tt-RU)
10) Ukrainian entries should be combined
11) Upper Sorbian spelling dictionary (hsb)
   English Name: Upper Sorbian
   Native Name: Hornjoserbsce (hsb)
12) Valencian Language (RACV) Spellchecker (roa-ES-val)
   English Name: Valencian
   Native Name: valencià (roa-ES-val)
13) Vietnamese Language Pack (vi-VN)
   English Name: Vietnamese
   Native Name: tiếng Việt (vi-VN)

Hope that helps.

Comment 10

[Fred Wenzel [:wenzel]]

That sure does help, however, it may require SQL again after all; before, I thought that changes to preview would affect the production DB. But since they don't propagate and prod doesn't have UI for target locales, it looks like I need to whip up some SQL again.

Comment 11

[Basil Hashem [:baz]]

Just a few more changes that I've run across:

- The following should be entries should be made Lang Packs for Galician/Galego : AMO_ID=5876 for Firefox, AMO_ID=5877 for Thunderbird, AMO_ID=5879 for Sunbird
- AMO_ID=6251 is for Thunderbird (arabic lang pack)
- Valencian, should be removed from the list since they are not true lang packs, they should be marked as extensions.

Comment 12

[Fred Wenzel [:wenzel]]

Attachment: SQL: More language pack locale assignments

This SQL makes the changes that can be done in the database (particularly assigning the right language codes).

Other changes have to be done in the mozilla-wide language list, which I will attach also shortly.

Note that I replaced srpski-LATINICA by sr-Latn (for "Serbian, spelled with the Latin alphabet") as proposed by RFC 4646.

Comment 13

[Fred Wenzel [:wenzel]]

As additions to the global language list (http://viewvc.svn.mozilla.org/vc/libs/product-details/localeDetails.class.php?view=markup) I identified the following. I think clouserw will have to do this, because I doubt I have write access to the product-details repository.
---

add:
	locale: gl
	English Name: Galician
   	Native Name: Galego

add:
	locale: et
	English Name: Estonian
   	Native Name: Eesti keel

add:
	locale: fur-IT
   	English Name: Friulian
   	Native Name: Furlan

add:
	locale: la
   	English Name: Latin
   	Native Name: Latina

add (following rfc 4646):
	locale: sr-Latn
   	English Name: Serbian
   	Native Name: Srpski

add:
	locale: tt-RU
   	English Name: Tatar
   	Native Name: Tatarça

add:
	locale: hsb
	English Name: Upper Sorbian
   	Native Name: Hornjoserbsce

add:
	locale: roa-ES-val
   	English Name: Valencian
   	Native Name: valencià

add:
	locale: vi
	English Name: Vietnamese
   	Native Name: tiếng Việt

add:
	locale: de-DE
	English Name: German (Germany)
   	Native Name: Deutsch (Deutschland)

add:
	locale: de-AT
	English Name: German (Austria)
   	Native Name: Deutsch (Österreich)

add:
	locale: de-CH
	English Name: German (Switzerland)
   	Native Name: Deutsch (Schweiz)

Comment 14

[Fred Wenzel [:wenzel]]

Attachment: Product details patch

This patch adds the languages/locales to the global locale array. clouserw, please review and apply the patch to the repository if it's all right.

Comment 15

[Wil Clouser [:clouserw]]

Comment on attachment 311428 [details] [diff] [review]
Product details patch

in SVN

Comment 16

[Wil Clouser [:clouserw]]

I'm in contact with the nb-NO/nn-NO dictionary authors and am trying to get them to split the dictionary into two packages since our new code doesn't support combined dictionaries(!).  It looks like there might be a German dictionary that is combined also that we should get in touch with.

Comment 17

[Fred Wenzel [:wenzel]]

Thanks for contacting them.

The combined German one consists of two spelling variants (classic vs. reformed spelling); the other two on the page are these variants separately. So I don't see a problem with that one, in particular because I marked them with an additional info text.

Comment 18

[Fred Wenzel [:wenzel]]

This is fixed, since the big bulk changes have landed (note that the SQL that combines a bunch of dictionaries is in production, but not on preview yet). Please file individual bugs if you notice additional things that we need to do.

Comment 19

[Fred Wenzel [:wenzel]]

FYI: I just sandboxed the Greek-English mixture dictionary in production (add-on ID 3451). According to its description text it really just is a combination of English and Greek.

Comment 20

[Wil Clouser [:clouserw]]

(In reply to comment #19)
> FYI: I just sandboxed the Greek-English mixture dictionary in production
> (add-on ID 3451). According to its description text it really just is a
> combination of English and Greek.
> 

Did you let the authors know?  I've been avoiding doing anything in production until we have policy to point people to.

Comment 21

[Fred Wenzel [:wenzel]]

As far as I got it, Basil just pushed a bunch of dictionaries public and this one was among them but shouldn't have been? But maybe I misunderstood this.