Status

P1
critical
RESOLVED FIXED
8 years ago
8 years ago

People

(Reporter: ozten, Assigned: brez)

Tracking

unspecified

Details

(Reporter)

Description

8 years ago
Analogous to Markup's Bug#652912, we need proper locale detection and L10n support.

This can be fixed after Bug#650033, since gettext is used currently and that would not change.
(Reporter)

Updated

8 years ago
Severity: normal → critical
Priority: -- → P1
Whiteboard: claudia@barbariangroup.com,ericac@barbariangroup.com,gjimenez@mozilla.com,jbresnik@barbariangroup.com,stephen.donner@gmail.com,tdickson@barbariangroup.com
Target Milestone: --- → 1.0
(Reporter)

Comment 1

8 years ago
(In reply to comment #0)

> This can be fixed after Bug#650033

This is a comment on priority. Please don't wait until that bug is closed to start.
(Reporter)

Updated

8 years ago
Whiteboard: claudia@barbariangroup.com,ericac@barbariangroup.com,gjimenez@mozilla.com,jbresnik@barbariangroup.com,stephen.donner@gmail.com,tdickson@barbariangroup.com
(Reporter)

Comment 2

8 years ago
Following the changes applied to Markup, we should use tower, commons middleware, etc.

Tower bug split out into Bug#662846.
Depends on: 662846
Summary: Fix locale detection etc by upgrading to tower, commons middleware, etc → Fix locale detection
(Assignee)

Updated

8 years ago
Assignee: nobody → jbresnik
(Assignee)

Comment 3

8 years ago
Both tower and commons middleware are currently enabled in webifyme.. i.e. what exactly is the request here?  Please clarify - thanks
(Assignee)

Comment 4

8 years ago
Did find some errors while processing though..

./manage.py makemessages --locale=es
processing language es
Error: errors happened while running xgettext on tests.py
./vendor/src/django/tests/regressiontests/i18n/tests.py:92: warning: internationalized messages should not contain the `\r' escape sequence

Will chk into that
(Assignee)

Comment 5

8 years ago
And extract throws:


ImportError: No module named translate.storage

So I'm guessing its there (as previously stated) but not working - anyhow looking into it.
(Reporter)

Comment 6

8 years ago
What is the full stacktrace?

Perhaps I missed adding a dependency? Is your vendor and/or virtualenv up to date?

What version of Python are you running against? I think translate.storage is from the standard library. I'm using 2.6.
(Assignee)

Comment 7

8 years ago
Full stacktrace / version info:


$ ./manage.py extract
/Users/jbresnik/Projects/mozilla/webifyme/ffenv/lib/python2.6/site-packages/jinja2/__init__.py:31: UserWarning: Module django was already imported from /Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/__init__.pyc, but /Users/jbresnik/Projects/mozilla/webifyme/ffenv/src/django is being added to sys.path
  __version__ = __import__('pkg_resources') \
/Users/jbresnik/Projects/mozilla/webifyme/ffenv/lib/python2.6/site-packages/jinja2/__init__.py:31: UserWarning: Module tower was already imported from /Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/tower/tower/__init__.pyc, but /Users/jbresnik/Projects/mozilla/webifyme/ffenv/src/tower is being added to sys.path
  __version__ = __import__('pkg_resources') \
Traceback (most recent call last):
  File "./manage.py", line 46, in <module>
    execute_manager(settings)
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/core/management/__init__.py", line 261, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/core/management/__init__.py", line 67, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/django/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/Users/jbresnik/Projects/mozilla/webifyme/ff4/vendor/src/tower/tower/management/commands/extract.py", line 15, in <module>
    from translate.storage import po
ImportError: No module named translate.storage
$ python -V
Python 2.6.1
(Assignee)

Comment 8

8 years ago
Yea youre missing the translate-toolkit - adding now..
(Reporter)

Comment 9

8 years ago
(In reply to comment #8)
I mispoke, agreed not standard library.

translate-toolkit==1.6.0 is already in ff4/requirements/dev.txt

This is not needed at runtime, only by stas at build time to extract the strings. Please don't add to vendor.
(Reporter)

Comment 10

8 years ago
(In reply to comment #4)
(In reply to comment #5)
(In reply to comment #7)
Based on these comments, you are testing the work done in Bug#662846. That is cool, but unnecessary for fixing this bug.


Just to clarify a part of a test plan...

Reproduction Steps:

Locale detection Use Case
1) Switch primary language of Firefox 4 to French
2) https://webifyme-dev.allizom.org/

Actual
Not redirected from https://webifyme-dev.allizom.org/.

Expected
Redirected to https://webifyme-dev.allizom.org/fr/

I'm pretty sure we *don't need* ff4/locale/fr/LC_MESSAGES/django.po.

Fixing this is in the middleware, which doesn't use po files.

Work to be done. Compare this app to Markup
* Need middleware from https://github.com/mozilla/markup/tree/master/ffdemo/commons
* Need settings from https://github.com/mozilla/markup/blob/master/ffdemo/settings.py
** Settings such as KNOWN_LANGUAGES_DEV, KNOWN_LANGUAGES_PROD, etc
(Assignee)

Comment 11

8 years ago
All set - redirection is working now - essentially there were a lot of updates to the config - implemented the same way markup is.. 

https://github.com/mozilla/webifyme/commit/4c09547cb8baac17dde6e2986ee00b61dbf6d35e
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
(Assignee)

Comment 12

8 years ago
Updating vendor as well..
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 13

8 years ago
All set

https://github.com/mozilla/webifyme-lib/commit/a2942f13696a2cf54f5f8abd14c489f3c4bbe50f
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 14

8 years ago
(In reply to comment #13)
https://webifyme-dev.allizom.org/en-US/ is 500.

I've filed Bug#664603 and will take a stab at fixing it.
I think that the locale detection is too greedy right now. 

With (de, en-US, en) in my browser settings, I'm redirected to

https://webifyme-dev.allizom.org/de/

and the text is in German.  When I manually go to 

https://webifyme-dev.allizom.org/en-US

the text is still in German.


Interestingly, with (en-US, en, de) I'm redirected to 

https://webifyme-dev.allizom.org/en-US/

but the text is in German too. /me puzzled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 17

8 years ago
Well how it works is the browser sets the 'Accept-Language' header to whatever you set it to [1] (or in your case a range of preferred languages) - the reason it isn't changing is probably related to your browser's cache or the server's cache (etc )- I guess the real question is, is switching languages during a single session a realistic user scenario?   If so I can dig into the django details and figure out a way to force it with every request but let's make sure this is really needed first. 


[1] http://www.apps.ietf.org/rfc/rfc3282.html
I agree that (In reply to comment #17)
> I guess the real question is, is switching
> languages during a single session a realistic user scenario?

I agree that it's not a realistic scenario, so if caching can be used, that's good.  However, since this is a dev server, should caching get in the way of testing if features work?

Also, we can try to detect user's preference, but whatever we detect should have lower priority than the locales explicitly specified in the URL.  So even if my prefs say (de, en-US), I should see English when I go to /en-US.

QA folks: I'd appreciate any help here to verify that the locale detection is working fine.  See comment 16 for testing scenarios.
(Assignee)

Comment 19

8 years ago
Yea I'll keep digging to figure out what's going on here..
(Assignee)

Comment 20

8 years ago
Been digging into this and discovered a few things: 

The LocaleURLMiddleware that we use ('commons' not the Django version) deliberately strips out the Locale as part of processing the request - so here's a scenario that explains what is going on by design:

1. I have my locale in the browser preferences set to es-mx,ru (mexican spanish, then russian) - this is confirmed by looking at the header in the incoming request for /fr/quiz

(Pdb) request.META['HTTP_ACCEPT_LANGUAGE']
'es-mx,ru;q=0.5'

Also the URL path:

(Pdb) prefixer.locale
'fr'

Which activates 'fr' for this request - problem is, since the locale is stripped out of the path, further requests default to en-mx (which I've specified in the browser preferences). 

So not sure if that's what we want or not..

Still looking into why the proper translation isn't being activated based on the URL locale setting.. more to come.
(Assignee)

Comment 21

8 years ago
Ok so the problem was really just be misconfigured - you can now type in a locale into the URL and see that language (tested this many times) *but once you navigate to a different link it will return to what you have defaulted in your browser - this is *not a bug but in fact how the commons library is implemented. Also any unsupported locale (defined as *not having an associated po / mo file) will default to en-US, this is also *not a bug but how the commons library is implemented. 

https://github.com/mozilla/webifyme/commit/00070e41bd72860b66a3e6c43ef2d7ab30a49a49
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Assignee)

Comment 22

8 years ago
Ok so the problem was really just *locale be misconfigured
You need to log in before you can comment on or make changes to this bug.