366316 - AMO Localization (tr)

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Localization, defect)

Description

[Ahmet Serkan Tıratacı]

This's the bug for localisation of AMO into Turkish.

Comment 1

[Ahmet Serkan Tıratacı]

Attachment: Turkish localisation

Turkish localisation ready for preview.

Comment 2

[Wil Clouser [:clouserw]]

Thanks for the translation.  It looks like we're going to need to make some code changes to handle some issues with how php handles Turkish.  Shaver found some php bug links that we'll need to look through to get a solution (php bugs 18556, 29955, and others...)

Since this affects the core code, and will require changes to cake, I'm setting the target milestone to 3.0 (when we will have upgraded cake).  We might have to go to 3.1 if cake 1.2 doesn't solve the issues.

Comment 3

[Wil Clouser [:clouserw]]

Updating bug title to be in line with localizers' guide (http://wiki.mozilla.org/Update:Localizers)

Comment 4

[Ahmet Serkan Tıratacı]

Has there been any improvement for last 4 months? We are really discouraged.

Comment 5

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Turkish continues to pose a problem for AMO, due to bad interactions with PHP and the framework we use.  It will require someone to do detailed debugging, and likely fixing, of the Cake framework at the least, and we don't have anyone to hand with the cycles to do that.  If there is someone in the Turkish localization community with experience making PHP applications play nicely with the unique characteristics of the Turkish language (largely case interactions, I believe), we would very much appreciate their assistance.

We share your discouragement here, and would very much like to see Turkish supported.  We're going to need help doing it, though. ;(

Comment 6

[Ahmet Serkan Tıratacı]

We are also looking for someone to solve this. Can you tell what exactly the problem is, give us details of the issue or link to bug that might have been opened to solve it?

Comment 7

[Wil Clouser [:clouserw]]

Ahmet, I'm sorry this takes so long - I felt bad when you posted yesterday, so I dug into it a bit.  I was planning on coming up with a solution for this in the AMO meeting today, but since things are started here, I'll comment.  I don't think this will be too hard to fix (for our case).

The problem is, the php string functions use the locale's alphabet when converting characters.  That means if there is no representation for the letter in the alphabet, it's skipped.  The problem we're running into here is that cake uses the string functions to determine which files to include, eg. uses('Image') in a file will include image.php (notice 'I' was converted to 'i' which Turkish doesn't have.)

So, solutions that I came up with yesterday that I would be happy to have input on:

1) Run through the cake libraries (or at least the inflector), and replace all the string functions with their mb_ counterparts.  The multibyte string functions don't use the set locale, so they'll work on any letters that have an upper/lower case counterpart (including stuff with umlauts, etc.)

2) The only place that we seem to be running into this problem is Inflector::underscore().  We could manually s/I/i/ in that function and call it a day.  Note: There is no guarantee this is the only place that the problem exists, but when I make that substitution, a test subset of pages render for me with no issues.

3) Overload the standard string functions in apache with the multibyte ones automatically by putting mbstring.func_overload in the .conf.  This would make calls to all the string functions automatically use the multibyte ones no matter what.  This makes me nervous...

Any other ideas?  Comments?

No matter which course we take, I think we're going to be happy we've got a pile of unit tests to make sure things still work.

Comment 8

[Wil Clouser [:clouserw]]

Adding Jeremy for his opinion on option #3.  This page:

http://php.oregonstate.edu/manual/en/ref.mbstring.php#mbstring.overload

Specifically says "Note:  It is not recommended to use the function overloading option in the per-directory context, because it's not confirmed yet to be stable enough in a production environment and may lead to undefined behaviour."  

Does that mean you have to enable it for the entire server, or you can split it off for each vhost, or just keep it out of the .htaccess?  Jeremy, do you have any ideas/experience with this?

If Jeremy's response isn't positive, I'll just do #2 as a fix to get Turkish working.

Comment 9

[Ahmet Serkan Tıratacı]

Wil, thanks for paying attention and investigating. I'm spreading word to get assistance.

For those who want to help, I advise you to read about Turkish alphabet and notice Turkish dotted and dotless i before you begin.

See http://en.wikipedia.org/wiki/Turkish_dotted_and_dotless_I for reference.

Comment 10

[Jesper Kristensen]

> The problem is, the php string functions use the locale's alphabet when
> converting characters.

Is localized character conversion needed in AMO? If not, it might work to

-    var $_lc_category      = LC_ALL;
+    var $_lc_category      = LC_ALL - LC_CTYPE;

in site/app/config/language.php on line 58

Comment 11

[Wil Clouser [:clouserw]]

(In reply to comment #10)
> Is localized character conversion needed in AMO? If not, it might work to
> 
> -    var $_lc_category      = LC_ALL;
> +    var $_lc_category      = LC_ALL - LC_CTYPE;
> 
> in site/app/config/language.php on line 58

The LC_ constants can't subtract themselves like that, so we actually have to make 5 separate calls to setlocale().  That said, it's working on my dev site, and I committed the change for testing on the staging site:

http://remora.stage.mozilla.com/tr/firefox/

Comment 12

[Ahmet Serkan Tıratacı]

(In reply to comment #11)
> (In reply to comment #10)
...
> The LC_ constants can't subtract themselves like that, so we actually have to
> make 5 separate calls to setlocale().  That said, it's working on my dev site,
> and I committed the change for testing on the staging site:
> 
> http://remora.stage.mozilla.com/tr/firefox/


I haven't noticed any problems with Turkish characters so far.

Comment 13

[Wil Clouser [:clouserw]]

We didn't find any problems with the way we're setting the locale now, so I'm going to reassign this bug to Ahmet.  Ahmet, if you haven't read the localizer guide yet, please do ( http://wiki.mozilla.org/Update:Localizers ).

Thanks Jesper for the fix. :)

Comment 14

[Jesper Kristensen]

The fix gives me a bunch of warning messages on each page. The LANG environment variable is used instead of LC_MESSAGES on Windows. Can you please put some kind of if-statement around setlocale(LC_MESSAGES, $lang); ?

Maybe:
if (!defined('WINDOWS')) setlocale(LC_MESSAGES, $lang);

This is what I get:

Notice: Use of undefined constant LC_MESSAGES - assumed 'LC_MESSAGES' in ***\site\app\config\language.php on line 191

Warning: setlocale() [function.setlocale]: Passing locale category name as string is deprecated. Use the LC_* -constants instead. in ***\site\app\config\language.php on line 191

Warning: setlocale() [function.setlocale]: Invalid locale category name LC_MESSAGES, must be one of LC_ALL, LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, or LC_TIME. in ***\site\app\config\language.php on line 191

Comment 15

[Wil Clouser [:clouserw]]

(In reply to comment #14)
> The fix gives me a bunch of warning messages on each page. The LANG environment
> variable is used instead of LC_MESSAGES on Windows. Can you please put some
> kind of if-statement around setlocale(LC_MESSAGES, $lang); ?
> 
> Maybe:
> if (!defined('WINDOWS')) setlocale(LC_MESSAGES, $lang);
> 
> This is what I get:
> 
> Notice: Use of undefined constant LC_MESSAGES - assumed 'LC_MESSAGES' in
> ***\site\app\config\language.php on line 191
> 
> Warning: setlocale() [function.setlocale]: Passing locale category name as
> string is deprecated. Use the LC_* -constants instead. in
> ***\site\app\config\language.php on line 191
> 
> Warning: setlocale() [function.setlocale]: Invalid locale category name
> LC_MESSAGES, must be one of LC_ALL, LC_COLLATE, LC_CTYPE, LC_MONETARY,
> LC_NUMERIC, or LC_TIME. in ***\site\app\config\language.php on line 191
> 

That sounds more like you don't have the gettext libraries installed or something.  It's saying LC_MESSAGES is not a constant (it's "undefined").  

I'm adding fligtar because he has a windows install I think.

Comment 16

[Justin Scott [:fligtar]]

(In reply to comment #15)
> I'm adding fligtar because he has a windows install I think.
> 

I'm on a mac now and my windows setup is very far away. Can you just make the changes in your local copy?

Comment 17

[Jesper Kristensen]

I can just comment it out in my local copy, but it would make it easier if AMO can run on Windows out of the box.

> That sounds more like you don't have the gettext libraries installed or
> something.  It's saying LC_MESSAGES is not a constant (it's "undefined").  

I guess it is some sort of magic that make the gettext functions work on my machine. :)

Comment 18

[Wil Clouser [:clouserw]]

Jesper: Looking around on google, it looks like LC_MESSAGES isn't defined on windows (surprise), and it looks like most people do something like:

if (defined('LC_MESSAGES')) {
  setlocale(LC_MESSAGES, $lang);
}

I'm not sure how windows knows to translate messages if this isn't set, but if you can verify that this code works on windows, we can add it in.  It shouldn't affect our production/staging/preview servers.

Comment 19

[Jesper Kristensen]

The selected locale for gettext is controlled by:

                putenv("LANG={$lang}");

There is already a comment that say this, but it is not very clear:

        // Set LANG environmental variable. This is not optional for Windows.

Comment 20

[Wil Clouser [:clouserw]]

(In reply to comment #19)
> The selected locale for gettext is controlled by:
> 
>                 putenv("LANG={$lang}");
> 
> There is already a comment that say this, but it is not very clear:
> 
>         // Set LANG environmental variable. This is not optional for Windows.
> 

Update to r4318 and let me know if it works.  I put in the defined() checks.

Comment 21

[Jesper Kristensen]

It works for me.

Comment 22

[Wil Clouser [:clouserw]]

Hi,

This is just a friendly reminder that I'm sending to all the AMO L10n bugs:

The 3.2 redesign is going live in just over a week and a half.  If you can't make the schedule or you have any questions please let me know.  If you've already updated your locales, thanks so much - they're looking great. :)

Comment 23

[Rail Aliiev [:rail]]

Hi all,
is there any changes regarding using Turkish in AMO?
If you still have some troubles I would be happy to help with this issue.

Comment 24

[Wil Clouser [:clouserw]]

Mass-close of L10n bugs.  We're using the newsgroups to notify localizers about updates so there isn't much of a reason to leave these open.  If you have a specific case, feel free to reopen.

Thanks for all your help! :)