551886 - [km] [Fx,Tb] New localization: Khmer

Status: RESOLVED FIXED

(Mozilla Localizations :: Registration & Management, defect)

Description

[Vannak Eng]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; km-KH; rv:1.9.2) Gecko/20100115 Firefox/3.6
Build Identifier: 

Localizing FireFox, Thunderbird into Khmer

Reproducible: Always

Comment 1

[Seth Bindernagel]

Hi Vannak,  Please now attach the localization .XPI for review.

Comment 2

[Axel Hecht [:Pike]]

Technical question, we're trying to keep locale code as short as possible. Is 'km' descriptive enough or are there regional variants of Khmer that would benefit from the KH region code?

Comment 3

[Vannak Eng]

@sethb: Attachments is allowed only .txt... so what am I gonna do? change the .xpi to .txt in order to attach it?

@Axel: I think only just km is descriptive enough for khmer language code.

Comment 4

[Axel Hecht [:Pike]]

We'll need a source to review, not a xpi. If you don't have a source, gandalf might have something to help you.

Attaching should work, just leave the mimetype to autodetect, and it'll do the right thing for tar.gz or .zip.

Comment 5

[Vannak Eng]

Attachment: Khmer Langpack for FireFox 3.6

Comment 6

[Vannak Eng]

Attachment: Khmer Langpack for Thunderbird 3.0.1

Comment 7

[Vannak Eng]

Hi Axel, I already attached two langpacks, one is for firefox and the other is for thunderbird. please review it. thanks

Comment 9

[Axel Hecht [:Pike]]

Gandalf, can you help out converting the langpack to a source here?

Comment 10

[Gervase Markham [:gerv]]

gandalf: ping?

Gerv

Comment 11

[Axel Hecht [:Pike]]

Comment on attachment 432710 [details]
Khmer Langpack for Thunderbird 3.0.1

Marking this attachment obsolete, it's a langpack, and it's not for a current version of thunderbird.

Comment 12

[Axel Hecht [:Pike]]

Comment on attachment 432709 [details]
Khmer Langpack for FireFox 3.6

Making it clear that there's not "r+" on a xpi, by denoting an r-.

Comment 13

[Seth Bindernagel]

Did Gandalf ever get a chance to try to convert Vannak's xpi to a source format, as we asked in Comment 9?  Now I see an r- in Comment 12, but I don't know if I ever saw the source posted by Gandalf.

Comment 14

[Vannak Eng]

@Seth, @Axel Can I attach the source .po for you guys to see? and how long does the review process take?

Comment 15

[Seth Bindernagel]

Please attach the PO files.  Gandalf or I can work on converting it to the proper Mozilla source format.

Comment 16

[Vannak Eng]

Attachment: PO files of Firefox

Comment 17

[Axel Hecht [:Pike]]

For reference, I want our sources for review. One goal of that review process is to make sure that the contributors are in shape to work on the sources we use for build, and there's no point in landing .po files.

Comment 18

[Seth Bindernagel]

Hi Eng,  It appears that you and Dietrich have been chatting about how to get the Mozilla l10n-drivers team something that we can review since our communication has been somewhat confusing.  Sorry about that.  I hope I can help clear this up for you so you can provide us something that we can review.

We often ask localizers to give us their work in "Source Format".  That means the entire set of DTD and property files in the proper structure that we ask for all of our locales.  The process on how to begin a new localization is all documented here:

https://developer.mozilla.org/en/create_a_new_localization

At this point, we will need to convert your set of .po files to the "source format", which includes DTD and property files in the proper structure that Mozilla use to generate a locale.  That format has to match the format that our source code base uses.

First question needing a response:  How did you come to localize Firefox using .po files?  Did you use a tool?  Which one? 

Second question needing a response:  Have you ever used the Translate Toolkit?  This can help you convert your work into the Mozilla source format, which included DTD and property files.

http://translate.sourceforge.net/wiki/toolkit/features

Can you try to convert your files to Mozilla source format and attach them for review?

Comment 19

[Vannak Eng]

Hi Seth,
for the first question, I'm using python script mo2po to convert en-us xpi file then convert it back by using po2mo.

second, I'm using Kbabel in openSUSE to do the localization.

Cheers,
Vannak

Comment 20

[Vannak Eng]

Attachment: Firefox .DTD .Property

Comment 21

[Axel Hecht [:Pike]]

Comment on attachment 478971 [details]
Firefox .DTD .Property

This is the source of a langpack. What we're looking for is a directory structure like 
browser/chrome/browser/...
toolkit/chrome/...

etc like you can see in http://mxr.mozilla.org/l10n-mozilla1.9.2/source/en-GB/ (you won't have all those dirs, some of them are for other apps).

gandalf should have a script that can shuffle the files around as much as possible, to help with the transition.

How to create en-US pot files in that very directory structure is something that someone else will need to explain to you, I've never done that. I've glanced at http://translate.sourceforge.net/wiki/guide/mozilla, and I didn't spot any bugs in https://translate.svn.sourceforge.net/svnroot/translate/src/trunk/tools/mozilla/get_moz_enUS.py or http://translate.sourceforge.net/wiki/toolkit/creating_mozilla_pot_files. Apart from the fact that we're not using CVS for ages anymore.


One more comment, did anyone break the news to you that we're not actually taking new locales for 3.6 anymore? It hasn't been a pleasant release to work on due to the many updates, and 4.0 is around the corner (dagnabbit, don't prove me wrong). Sadly, 4.0 at this point is a good 1000 more strings compared to 3.6. The current en-US sources for that are in mozilla-central, that's the repository mentioned in the translate.sf.net docs above, so that should get you the right pots. 4.0 is not string frozen yet, and it's going to change the hg repository, but for the next few days, mozilla-central is the place where to get the en-US files. Updates to that will be posted to .l10n, http://www.mozilla.org/about/forums/#dev-l10n.

Comment 22

[Vannak Eng]

Hi Axel,
I just cloned the mozilla-1.9.2 from releases/mozilla-1.9.2 by using hg and creating .pot files already. and I wonder what should I do for this step?

Cheers,
Vannak

Comment 23

[Dietrich Ayala (:dietrich)]

Attachment: source locale

i used gandalf's map script to convert the langpack for 3.6 to the source directory for 4.0 (thanks for the help gandalf and axel!).

what are the next steps from here?

Comment 24

[Axel Hecht [:Pike]]

Nice, the attachment 479727 [details] gives us a good head-start.

This is the current stats from compare-locales:

errors: 1
missing: 507
missingInFiles: 1229
keys: 743
unchanged: 182
changed: 3832
obsolete: 434
66% of entries changed


That one error is in toolkit/chrome/global/aboutSupport.dtd
            ERROR: mismatched tag at line 4, column 241 within aboutSupport.pageSubtitle
It's missing the <a id='supportLink'> that's in http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/en-US/chrome/global/aboutSupport.dtd#4.


Next up is translating the 1700 missing strings. I can only guess on the workflow, but I expect that you'd convert the DTD/properties back to PO now that they're in the right place, and then let the po-magic loose with the pot files you created from mozilla-central. (I read in dietrich's irc comments that you moved over to central for en-US, right?)


We'll look into creating an hg repo and actually landing this after we branched for mozilla-2.0, that way you only have one repository to shuffle around locally.

Comment 25

[Axel Hecht [:Pike]]

One more thing, going back to our discussion here in the bug about km vs km-KH, the attachment was still for km-KH, so if you'd do that migration locally while you're at it, that'd be great.

Comment 26

[Seth Bindernagel]

Hey Vannak,  Just checking in here to see if there is any progress on your translations or anything we can help with.  Please ask any questions if you have them.

Comment 27

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #25)
> One more thing, going back to our discussion here in the bug about km vs km-KH,
> the attachment was still for km-KH, so if you'd do that migration locally while
> you're at it, that'd be great.

(In reply to comment #24)
> Nice, the attachment 479727 [details] gives us a good head-start.
> 
> This is the current stats from compare-locales:
> 
> errors: 1
> missing: 507
> missingInFiles: 1229
> keys: 743
> unchanged: 182
> changed: 3832
> obsolete: 434
> 66% of entries changed
> 
> 
> That one error is in toolkit/chrome/global/aboutSupport.dtd
>             ERROR: mismatched tag at line 4, column 241 within
> aboutSupport.pageSubtitle
> It's missing the <a id='supportLink'> that's in
> http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/en-US/chrome/global/aboutSupport.dtd#4.
> 
> 
> Next up is translating the 1700 missing strings. I can only guess on the
> workflow, but I expect that you'd convert the DTD/properties back to PO now
> that they're in the right place, and then let the po-magic loose with the pot
> files you created from mozilla-central. (I read in dietrich's irc comments that
> you moved over to central for en-US, right?)
> 
> 
> We'll look into creating an hg repo and actually landing this after we branched
> for mozilla-2.0, that way you only have one repository to shuffle around
> locally.

As I understand it, next steps here are:

* change from "km-KH" to "km"
* fix the "missing: 507" 
* fix the "missing: 1229"
* fix the dtd syntax error. 
* post new patch?

The timing of branch point for mozilla-20 has changed, so I'm not sure what the plans are for setting up this repo. However, we can attend to these fixes in the meanwhile.

Comment 28

[Axel Hecht [:Pike]]

The repo is actually up, builds are failing on missing files, https://l10n-stage-sj.mozilla.org/dashboard/compare?run=102498 and http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla-l10n-km.

In terms of bugs to file, I'm waiting on updates to our messaging for the recent changes in search, etc. Don't want to send out confusing messages.

Comment 29

[Axel Hecht [:Pike]]

patches or hg bundles with fixes for the reported issues would be very welcome, too.

Comment 30

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #28)
> The repo is actually up, 
Sweet, thanks Axel.


> builds are failing on missing files,
> https://l10n-stage-sj.mozilla.org/dashboard/compare?run=102498 and
> http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla-l10n-km.
> 
> In terms of bugs to file, I'm waiting on updates to our messaging for the
> recent changes in search, etc. Don't want to send out confusing messages.

I know there's a DTD error to be fixed. 

Once the DTD error is fixed, aiui, the l1n-merge would let us generate a Khmer nightly build with en-US strings for any missing Khmer strings?

Comment 31

[Axel Hecht [:Pike]]

Sadly, l10n-merge doesn't fix everything yet. Didn't get to fixing that yet, bug 525438.

Comment 32

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #31)
> Sadly, l10n-merge doesn't fix everything yet. Didn't get to fixing that yet,
> bug 525438.

Axel: Which specific things does l10n-merge not handle right now? If you give us an explicit list, we'll focus on fixing those, and the broken DTD, first, so we can unblock using l10n-merge as quickly as possible.

Comment 33

[Dietrich Ayala (:dietrich)]

Axel: What is the error, and how are you seeing it? Are you applying the patch, and then running the app? How would someone who is not you reproduce what you are seeing?

Comment 34

[Axel Hecht [:Pike]]

http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla-l10n-km&hours=24 has plenty of failed builds, with logs.

I'll not go in and create a detailed list of failures now as I'm spending that time on just making those problems go away in the other bug.

Comment 35

[Dietrich Ayala (:dietrich)]

Ah ok, we can see the errors that you're referring to in those logs, great.

What other bug are you referring to?

Comment 36

[Axel Hecht [:Pike]]

bug 525438 should make l10n-merge cover more grounds.

Comment 37

[Seth Bindernagel]

Hey Vannak,  Checking in here to see if there is any progress on your
translations or anything we can help with.  Please provide an update.  Thanks.

Comment 38

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Attachment: add standard template files

This patch at least fixes the current error on nightly builds. Once this lands, we can see what errors we hit, even with running l10n-merge.

...
make[1]: *** No rule to make target `/builds/moz2_slave/cen-lnx-l10n-ntly/build/l10n-central/km/browser/README.txt', needed by `libs'.  Stop.
make[1]: Leaving directory `/builds/moz2_slave/cen-lnx-l10n-ntly/build/mozilla-central/browser/locales'
make: *** [libs-km] Error 2
program finished with exit code 2

Comment 39

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #38)
> Created attachment 509008 [details] [diff] [review]
> add standard template files
> 
> This patch at least fixes the current error on nightly builds. Once this lands,
> we can see what errors we hit, even with running l10n-merge.
> 
> ...
> make[1]: *** No rule to make target
> `/builds/moz2_slave/cen-lnx-l10n-ntly/build/l10n-central/km/browser/README.txt',
> needed by `libs'.  Stop.
> make[1]: Leaving directory
> `/builds/moz2_slave/cen-lnx-l10n-ntly/build/mozilla-central/browser/locales'
> make: *** [libs-km] Error 2
> program finished with exit code 2


(note: I asked Armen for the r? on this patch, because he's been helping me so far, he's a recognized Mozilla localizer (hy-AM), its not really locale-specific and its unclear who is the formal reviewer for this new locale. If there is someone else who would be better for this r?, please let me know.)

Comment 40

[Armen [:armenzg]]

Comment on attachment 509008 [details] [diff] [review]
add standard template files

You have a couple of mentions of Armenian localizers in the patch. I know we are famous but we don't speak Khmer ;)
r+ when removing those.

You might want to push your change to an hg user repo or bitbucket.org.

In the long term for km you could request km to be created in Narro [1].
From there you can export any new translations and push the changes to your user repo/bitbucket until there is a team officially is created (see bug 559702 as an example).
Once a team is officially recognized a Mercurial account on Mozilla's network gets granted and the changes on your user repo

[1] https://l10n.mozilla.org/narro/narro_project_list.php?l=km

Comment 41

[Axel Hecht [:Pike]]

Comment on attachment 509008 [details] [diff] [review]
add standard template files

That'd be an r-, then.

Let's not pile plans on top of discussions that happen outside of this bug, please.

Comment 42

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #40)
> Comment on attachment 509008 [details] [diff] [review]
> add standard template files
> 
> You have a couple of mentions of Armenian localizers in the patch. I know we
> are famous but we don't speak Khmer ;)
> r+ when removing those.

From irc, Armen was confused and thought that I left Armenian localisers here. The names are Vannak (one Khmer localizer) and Armen (who has done great behind-the-scenes work to help). I can remove Armen if its a problem.

Comment 43

[Seth Bindernagel]

Hi Vannak:  

We have a number of contributors (contacts of John O'Duinn :joduinn who is commenting inside this bug) who are eager to help with the Khmer translation of Firefox.  We need to see your work attached to this bug ASAP or we will begin outreach to those new contributors to begin.  If you respond, :joduinn would like to introduce you to all of his Khmer-speaking contacts who are interested in contributing.  Can you please acknowledging receipt of this message?  

And, can you let us know

1) the status of your work
2) how :joduinn and his group can contact you to understand where they may help

If you do not respond, we will ask the new contributors to begin their work.  I do hope we hear from you.  We need to have *much* more responsiveness from you as a volunteer if you intend to lead a localization effort.  Not responding to this bug or other forms of contact from Mozilla means that we cannot work with you.  Localizing and releasing software is deadline-driven work and requires timely responses.  We wouldn't be able to grow Mozilla if we did not have volunteers who respond quickly to our messages.

-Seth

Comment 44

[Vannak Eng]

Hi Seth,
Thank you for this.
1) The status of my work: I have tried many times to get the souce cloned by using hq clone but it always fails.
2) They can contact me by sending email to [e-mail] or [e-mail] or hp 012305031.
Cheers,
Vannak

Comment 45

[Seth Bindernagel]

(In reply to comment #44)
> Hi Seth,
> Thank you for this.
> 1) The status of my work: I have tried many times to get the souce cloned by
> using hq clone but it always fails.

Can you describe what problems you are experiencing?  I understand from Dietrich that he has tried to help you by sending a zip file of the latest strings from beta 9.  We are now on to beta 11, so there have been some minor changes since then.  You will need to get a proper Hg setup to be able to continue.  We should try to figure that out for you asap.

Do you have or know of IRC as a communication tool?  Many of us at Mozilla use that and can be found on the IRC server (irc.mozilla.org) on the channel #l10n.  You should try to set up your IRC client and join us to get this Hg problem fixed or we will not be able to proceed.

Also, have you considered using a tool like Narro? (https://l10n.mozilla.org/narro/narro_project_list.php?l=en-US) That may allow you to work as we figure out together what problems you are having with Hg.  If you would like to use this tool, I can put you in touch with the leader of the Narro project who can set you up quickly and you can begin work, including importing anything that you may have already done.

> 2) They can contact me by sending email to [e-mail] or
> [e-mail] or hp 012305031.

Thanks for this contact information.  I am certain that John O'Duinn (:joduinn) will send an email that introduces the new contributors.

Thank you for responding quickly.  I look forward to the next timely response.

Comment 46

[Vannak Eng]

Hi seth, I think working in Narro is better and faster for me. as I'm now lost in the process of getting the source and converting it.

Bests,
Vannak

Comment 47

[Vannak Eng]

Attachment: Please help to convert this to source locale

Comment 48

[Dietrich Ayala (:dietrich)]

This file is conversion from XPI using mo2po. What translation tool did you use after that to generate this?

Comment 49

[Vannak Eng]

i used wordforge translation tool (this tool is developed from Pootle)

Comment 50

[Dietrich Ayala (:dietrich)]

Vannak, can you post to the l10-dev list, and see if anyone knows the best way to convert Pootle output to source locale?

http://groups.google.com/group/mozilla.dev.l10n/topics

Comment 51

[Vannak Eng]

i did it already thank Dietrich.

Comment 52

[Dwayne Bailey]

(In reply to comment #47)
> Created attachment 531578 [details]
> Please help to convert this to source locale

I ran these files against my PO build scripts.  You have only 88 PO files, yet the current Aurora branch has 228.  I think you've missed some files.

Could you please point me to ALL the po files that you have.  My scripts can easily convert from any PO layout to the correct Mozilla layout.

I've put the corrected PO files up here: http://pootle.locamotion.org/km/firefox/ - these follow the prioritisation order that we've established for the other languages using Pootle.  But like I said I think too much is missing from your PO ZIP that you uploaded.

So if you can point me to the correct PO files then I can easily convert these.  You are welcome to translate on Pootle as we have a number of languages working there.  The checks from Translate Toolkit are integrated into Pootle so you can ensure that you don't make too many technical errors.

Comment 53

[arky [:arky]]

Attachment: Please review code for landing

Comment 54

[Vannak Eng]

Please help to review the translation from Arky attachment. Thanks

Comment 55

[Dietrich Ayala (:dietrich)]

l10n@ - do you have an ETA on reviewing this?

Comment 56

[Vannak Eng]

Attachment: Thunderbird

Comment 57

[Axel Hecht [:Pike]]

Comment on attachment 576395 [details]
Please review code for landing

Thanks for the patience, there was a lot going on here, mail dying included:

This looks great, r=me with a few nits:

There are a lot of CSS specs translated, the new version of compare-locales finds those. There's also a parsing error in extensions/update.dtd, the quotes in http://a.maimult.ro/lmo/translate.php?l=km&p=28&f=/toolkit/chrome/mozapps/extensions/update.dtd&t=6&s=Click+Finish+to+continue&o=&h=1&m=10&i=0#i9115479 break. Same in profile/createProfileWizard.dtd.

In browser/defines.inc, the contributors look wrong. In toolkit/defines.inc, the language name sounds wrong, clearly this isn't Khmer as spoken in the US?

Please revert browser/chrome/browser-region/region.properties to its en-US form, changes to that file will need to get actual reviews. That'll also catch the technical bugs in that file.

Do not add the en-US searchplugin files, they'll be picked up from the en-US sources by the build anyway. The list.txt is fine, though.

In toolkit/chrome/global/intl.properties:
general.useragent.locale should be km
intl.accept_languages should be localized to list km, and other languages likely spoken by people in myanmar, if applicable. Arky, can you assist here?
intl.menuitems.insertseparatorbeforeaccesskeys should be true, not a translation thereof.

The rest looks great, I'll file more bugs to get the uplift started in a few.

Comment 58

[Axel Hecht [:Pike]]

Congratulations, filed bug 709138 to get this landed and shipped. Marking this bug FIXED.

Comment 59

[arky [:arky]]

Thank you!

Comment 60

[Justin Dolske [:Dolske]]

Comment on attachment 577884 [details]
Thunderbird

Clearing old, unowned review request from closed bug. If this is still an issure, please file a new bug or reassign the review to someone specific. Thanks!