844097 - LC_ALL environment variable is not honored to start the application with a specific locale

Status: RESOLVED WORKSFORME

(Testing :: Mozbase, defect)

Description

[[e-mail]]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.84 Safari/537.22

Steps to reproduce:

LC_ALL=tr_TR.utf8 mozmill
or 
export LC_ALL=tr_TR.utf8
mozmilla


Actual results:

Always getting firefox with en_EN.utf8 


Expected results:

should get firefox with defined locales

executing just firefox works just fine, does not mater if locales are exported or prepended

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So Tom is seeing this on Linux Fedora with the per default shipped Firefox binary and a couple of other locales installed. Most likely via the system packaging system.

This scenario is not something we support yet. Usually you have to run a Firefox version for the given locale. We never handled locale XPIs.

Would be nice if we could get this fixed but it's not high on our priority list.

We might wanna make it a mentored project.

Comment 2

[[e-mail]]

Seems I got a workaround, executing

  LC_ALL=zu_ZA.utf8 mozmill -a /usr/lib64/firefox/langpacks/[e-mail]

works for me.

Comment 3

[theod]

I can confirm this bug. When executing:

LC_ALL=de_DE.utf8 mozmill

Won't get the localized version (will get English version)

Though, on the same system, executing:

LC_ALL=de_DE.utf8 firefox

Will get Firefox in the right localized version...

So it seems it comes from Mozmill itself / its way of launching Firefox.

Isn't mozmill launching Firefox in a kind of isolated / sandboxed environment?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

How does passing through of environment variables work for different processes? Shouldn't the child processes automatically inherit them? Or need this to be done by the wrapper itself?

Comment 5

[theod]

According to the fork() documentation, environment variables are supposed to be inherited... unless some changes are made by the caller of fork().

I think the fact the Firefox process does not inherits the environment variable comes from the way it is instanciated. As far as I have seen, mozmill does not just run the "firefox" command, else it would not open a brand-new Firefox process even when Firefox is already running but it would open a new window in the existing Firefox.

I tried to locate the exact execution of Firefox in the Mozmill code but as far as I have seen it uses the Mozrunner package. Thus, it might actually be an issue related to mozrunner itself and not mozmill. Further tests could certainly tell us whether it's the case. I will do some, if I have the time (and manage to understand how to use mozrunner).

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Theod, mozrunner has the same command line options as mozmill itself. So running the following command should give us a hint:

mozrunner -b %path_to_firefox%

If it's indeed a mozrunner issue we will have to move the bug into the right component.

Comment 7

[theod]

I just tested and this seems to be an issue related to mozrunner.

Running:

`LC_ALL=de_DE.utf8 firefox`

Will spawn a localized version of Firefox.

Whereas running, in the same environment :

`LC_ALL=de_DE.utf8 mozrunner -b /usr/bin/firefox`

Will just spawn the English version.

BTW, I tried with other locales, and it did exactly the same.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I wonder where it could be located in mozrunner. I could assume it's how we start the process. Jeff, do you have some input regarding mozprocess?

Comment 9

[Jeff Hammel]

Is this for mozmill 1.5 or 2.x?  For 2.x, this SHOULD work at casual glance (untested).  For 1.5, there is no mozprocess.

Comment 10

[theod]

From what I've tested, the mozrunner affected is at least the 2.5.x. 
> $ mozrunner --version
> mozrunner  2.5.15

I don't know if this means mozmill 2.x of 1.5.x . The tests I ran were against mozmill, though, are against the 1.5.21 version.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozrunner 2.5.15 is the latest version for Mozmill 1.5.x. Mozmill 2.0 makes use of Mozrunner 5.15.

First thing I would check is the list of environment variables Mozrunner is able to see. Try to do it in its __init__ method. If LC_ALL is present it might be related how we launch Firefox.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I think a good read on how we could get this fixed can be found here:
http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/

Comment 13

[Jeff Hammel]

(In reply to Henrik Skupin (:whimboo) from comment #12)
> I think a good read on how we could get this fixed can be found here:
> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-
> your-enemy/

filed mozprocess issue : https://bugzilla.mozilla.org/show_bug.cgi?id=861951

Comment 14

[theod]

(In reply to Henrik Skupin (:whimboo) from comment #12)
> I think a good read on how we could get this fixed can be found here:
> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-
> your-enemy/

What would be the relation between our environment variables-related issue and the hanging PIPE when the input in greater than 65ko?

Anyway, I have just tested adding :


> print "------------------------------------"
> print os.environ
> print "------------------------------------"

At line 55 of __init__.py in my mozrunner/ directory.

And... I do not paste the complete output here, by I can see the LC_ALL variable when running :

`LC_ALL=de_DE.utf8 mozrunner -b /usr/bin/firefox`

And.... yes, it has the right value : 'LC_ALL': 'de_DE.utf8'

However, the Firefox launched is still in English...

So mozrunner DOES see the env variables... but just does not pass them to the subprocess, for some reason...

Note: I did not comment on the other issue because the title does not seem to be related to those LC_ALL issues... so, not sure how it is related.

Comment 15

[theod]

OK, so I decided to investigate a bit further... And, this is beginning to be a bit weird.

I tried to look at the env variables until the Popen() command itself and everything is all right. The subprocess even gets the LC_ALL variable.

I tried to replace the command run by killableprocess.Popen by : ["/usr/bin/bash", "-c" , "echo $LC_ALL"]

And, you know what? Here's what it displays : de_DE.utf8

So, the environment is perfectly passed. Bash can get it. But Firefox cannot.

Either this might be an issue related to the profile we pass to Firefox, either this is related to LC_ALL being a bash-specific thing?

I am going to try to find out more information.

Comment 16

[theod]

OK, this is it. It comes from the content that is being put inside the temporary profile that is created by mozrunner/mozmilll/any of those ones.

If I run firefox through bash, in the same Popen(): 

> cmd2 = ['/usr/bin/bash', '-c']
> cmd2.append(' '.join(cmd))

--> This will run : bash -c "/usr/bin/firefox -profile /tmp/tmp0ZNWyR.mozrunner" and Firefox will not be localized to LC_ALL

If I run :

> cmd2 = ["/usr/bin/bash", "-c", "/usr/bin/firefox -profile /tmp/blo"]

(with /tmp/blo being a newly created/fresh empty directory)

--> This will run : bash -c "/usr/bin/firefox -profile /tmp/blo" and Firefox will be localized.

And the only difference is the profile that Firefox is given.

I am not a prefs.js pro so I won't try any further investigation right now. Anyone has an idea?

Comment 17

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Could you compare the two profiles and see what's different in the MozRunner generated one? We set several preferences [1], but I don't see any that would influence the locale.

[1] https://github.com/mozilla/mozbase/blob/master/mozprofile/mozprofile/profile.py#L257

Comment 18

[John Vandenberg]

I think you need to add (untested)

'intl.locale.matchOS' : True,

http://kb.mozillazine.org/Intl.locale.matchOS

Comment 19

[Juan C. Mojica Roa (:jcmojicar)]

further investigation on this showed me this:

1 - Its necessary to add 'intl.locale.matchOS' : True, to profile.py as stated in comment #18.

2- I could make it work (mozrunner and therefore mozmill) with a localized version of firefox by changing how the language pack add-on is installed. 

     - When installed under the user home (/home/<username>/.mozilla/extensions), there is a need to set the following in profile.py, to enable firefox to get the add-on from the profiles created by the user running mozrunner, as it appears on https://developer.mozilla.org/en-US/Add-ons/Installing_extensions:

                   'extensions.enabledScopes' : 7,
                   'extensions.autoDisableScopes' : 8,

     - To avoid changing the scopes in profile.py, the language pack can be installed under <firefox binary path>/distribution/bundles as described on http://mike.kaply.com/2012/02/09/integrating-add-ons-into-firefox/ in the Scenario 1, this way just adding the 'intl.locale.matchOS' will do the trick.

I tested this on Firefox 26.0, mozrunner 5.27, mozmill 2.1-dev

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I'm not that happy with enabling 'intl.locale.matchOS' by default. Mostly because of the comments on bug 331779. As long as it is not clear which side-effects this preference can cause nowadays, we shouldn't make use of it. Also I wouldn't change the scope preferences for extensions as mentioned above. We strictly set those a couple of months ago, to disallow the installation of any system-wide extension. Those always caused us troubles on Ubuntu.

Given that this bug only applies to Firefox versions with installed XPI locale packages, I would propose that we leave the code as it is now. If you want to use another locale you should specify the location to the XPI file for the locale to install. With the LC_ALL environment variable you will be able to select it as the current locale on startup. This is what Tom Pelka mentioned in comment 2 already. So here the possible CLI command for zu_ZA:

LC_ALL=zu_ZA.utf8 mozmill -a /usr/lib64/firefox/langpacks/[e-mail]

I'm closing this bug as WFM given the solution from above. Thanks everyone for the help in investigating this problem!