647352 - Add tower to requirements

Status: VERIFIED FIXED

(Websites Graveyard :: markup.mozilla.org, defect, P1)

Description

[Staś Małolepszy :stas]

To enable l10n, we need tower (with its dependencies: babel and translate-toolkit).

> -e git://github.com/clouserw/tower.git@79b2e9d0c9f2c1a6ee8ae0453d8ca0bf3474b308#egg=tower-0.3.4-py2.6-dev

It will also require having settings.ROOT defined (which should be the same as the currently defined settings.PROJECT_PATH).

Lastly, there should be settings.DOMAIN_METHODS set up. See the following examples:

  https://github.com/mozilla/playdoh/blob/base/settings.py#L201
  https://github.com/mozilla/spark/blob/master/settings.py#L298

Comment 1

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

This is considered a blocking bug for the Markup project.

Comment 2

[Staś Małolepszy :stas]

We usually use tower because it plays nicely with Jinja2 trans blocks and also exposes handy management commands like 'extract' to extract all the strings from the code into a POT file.

If you intended to use a different approach for l10n, please let us know what it was. I'm not saying that we must be using tower, although it is something that we support and are familiar with, so I'd describe it as a preferred solution :)

Comment 3

[Adam Miller]

This seems like an enhancement, not a bug or blocker. Is that correct? If that's not correct, why am I only hearing about this now? 

We didn't know about this preference until now, so we didn't include it.

Comment 4

[Staś Małolepszy :stas]

Do you have a different approach for extracting the strings then?  Can you document it here please?

Comment 5

[Adam Miller]

l10n is enabled and we're using django's gettext for small strings and blocktrans for larger ones. Think that will be ok?

Comment 6

[Laura Mesa [:lmesa] [:lvmesa]]

Stas--any comment here?

Comment 7

[Staś Małolepszy :stas]

It looks like I can use Django's ./manage.py makemessages to extract the strings.

However, Django's default i18n system has many shortcomings, which made us develop Tower.  For example, it doesn't strip whitespace from strings, and in consequence, multiline strings need to have zero indentation, or this happens;

> #: templates_orig/base.html:98
> #, python-format
> msgid ""
> "\n"
> "\t\t\t\t\tExcept where otherwise noted, content on this site is licensed "
> "under the <a href=\"%(cclicenseurl)s\" target=\"_blank\">Creative Commons "
> "Attribution Share-Alike License v3.0</a> or any later version.\n"
> "\t\t\t\t\t"
> msgstr ""

Localizers don't have to keep all the whitespace characters in the translations, but they make it hard to read the string and are confusing.

I'd prefer to use tower, but I'm not sure if it makes sense to spend time right now to add it.  Ryan, what do you think?

Comment 8

[Fred Wenzel [:wenzel]]

I'll take a look if I can point the tower library onto the django templates. If it works, that's great, otherwise we'll have to live with the default Django string extraction.

Comment 9

[Staś Małolepszy :stas]

(In reply to comment #8)
> I'll take a look if I can point the tower library onto the django templates.

Fred, any luck with that?

Comment 10

[Fred Wenzel [:wenzel]]

(In reply to comment #9)
> (In reply to comment #8)
> > I'll take a look if I can point the tower library onto the django templates.
> 
> Fred, any luck with that?

Only a little. I got tower to *extract* the normalized strings now, but that doesn't mean they're being used yet.

Comment 11

[Fred Wenzel [:wenzel]]

Okay, Stas: Here's a patch for you. Can you try it?

https://github.com/mozilla/markup/commit/80296f9

Several notes:
- I am quite certain it is extracting strings right. However, it produces some "invalid" msgid at the moment (i.e., they contain UTF-8). Perhaps we just need to fix those strings in the template files?
- I also think I monkeypatched django's blocktrans right, so that it uses gettext with stripped whitespace.

However, after translating some strings and compiling the .mo file, I accessed /de/about/evan-roth/ and saw nothing at all translated (even if I remove the monkeypatch again and translate short strings only that should not be affected by the tower stuff).

Do you want to poke at it a little?

Comment 12

[Fred Wenzel [:wenzel]]

I think I fixed it. https://github.com/mozilla/markup/commit/48cbce7

Both the tower extraction commands and the compile script should work as expected.

Stas, please test this and let us know what else you need.

Comment 13

[Staś Małolepszy :stas]

Fred, you're my personal idol of today.  I filed bug 652899 and bug 652908 as follow-ups, but you nailed it anyways.  Thanks!