686984 - Compiling messages.po files on affiliates-dev failing

Status: VERIFIED FIXED

(Firefox Affiliates Graveyard :: affiliates.mozilla.org, defect)

Description

[Osmose [:osmose, :mkelly]]

affiliates-dev does not appear to be compiling the messages.po files to .mo files properly. Translations do not appear on the site, and there are no .mo files in the locale subdirectories.

cshields, jakem: From what I've been able to see, it looks like the update script is run on another server and the results are pushed to a git repo that affiliates-dev updates from. Is this correct?

affiliates-app/locale/compile.sh is running (stas had it output some debug messages yesterday and today), but, assuming I'm right about the git repo syncing, the .mo files aren't being added to the repo. Is this the case?

Comment 1

[Corey Shields [:cshields]]

I'll let jcrowe (jd on irc) look into this on the server side.

Jason, this is on the engagement cluster.  there should be a "compile.sh" directive in the local dir that may to be working, but check to see that the .mo files are being created in locale in /src/ then that they are being synced to /www/ and that they get checked in to the git repo there

Comment 2

[Jason Crowe [:jd]]

Michael,

You will need to take the '*.mo' line out it the '.gitignore' file in the web root directory.  This ignore is causing the .mo files to not get pushed out to the production cluster.  The files are getting generated on the admin host.

Comment 3

[Jake Maul [:jakem]]

This should never be necessary.

Developers commit to github and we pull from that to the src/ directory on the admin node. In there should exist any .gitignore files they want.

Then from src we rsync to www, and specifically exclude .git and .gitignore (or at least, we're supposed to). The copy in www should have only the .gitignore we've put there, if any.

The reason for this is we then commit to our own local git repo, using our own .gitignore rules... obviously, there's really nothing we'd want to ignore at that step.

I suspect what has happened is that at some point someone made a raw copy of src->www, and got all the upstream .gitignore files. It should be safe to remove any .gitignore files at this level and lower:

/data/<environment>/www/<site>/.gitignore

The .gitignore files at right at the www/.gitignore levels are good, though, those are ours. It's only the deeper ones that should not exist.

Comment 4

[Jake Maul [:jakem]]

This should be fixed now, for all environments and sites on the engagement cluster. Please let us know if it looks fixed for you. I see some .mo files on prod now, so I believe this should be working.

For the record, here's roughly what I did... it's the same thing I did on addonsadm.private.phx1 yesterday, for the very same type of issue (not locale files, but a similar case of .gitignore getting in the way):

find /data/engagement/www -mindepth 2 -type f -iname .gitignore 
find /data/engagement-dev/www -mindepth 2 -type f -iname .gitignore 
find /data/engagement-stage/www -mindepth 2 -type f -iname .gitignore

... then just rm any results however you prefer. You can then re-run the update/deploy job for a site that had .gitignore files and see that they don't come back. Affililates-dev is this:

/data/engagement-dev/src/affiliates-dev.allizom.org/update

Comment 5

[Corey Shields [:cshields]]

(In reply to Jake Maul [:jakem] from comment #3)
> Then from src we rsync to www, and specifically exclude .git and .gitignore
> (or at least, we're supposed to). The copy in www should have only the
> .gitignore we've put there, if any.

excluding the latter was pulled from the rsync command lately because of the way that devs were using .gitignore to force git to push an empty directory.  IMO this is a bad practice but we got around that by not excluding .gitignore, which passed their .gitignore through to our repo, thus causing the .mo files to be ignored.

So, while I am happy to exclude .gitignore in the deploy script, we should get with webdev and make sure they are using a different placeholder for empty directories.

Comment 6

[Fred Wenzel [:wenzel]]

(In reply to Corey Shields [:cshields] from comment #5)
> So, while I am happy to exclude .gitignore in the deploy script, we should
> get with webdev and make sure they are using a different placeholder for
> empty directories.

Shyam and a few of us already sorted this out (we can just use .gitkeep as a placeholder) -- but I don't know what places exactly are affected by this. So if you have apps that you specifically want me to fix, I am all ears.

Comment 7

[Jake Maul [:jakem]]

I will see about making a list of affected directories and opening a new bug for this. Thanks!

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Post-fix screenshot

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; at least, French and other locales (I spot-checked a few), are showing up, now: https://affiliates-dev.allizom.org/fr/