1257812 - iOS XLIFF: Clean up output files before committing them to repository

Status: RESOLVED FIXED

Description

[Francesco Lodolo [:flod]]

Per discussion on IRC: this is the script used to update locales when new strings are added to en-US
https://github.com/mozilla/firefox-ios-build-tools/blob/master/scripts/update-xliff.py

Right now when Pontoon commits files it creates a huge diff which is not usable on GitHub. It would be great to have this script run on files before committing them.

We fixed Pootle in https://github.com/translate/mozilla/issues/1

And there's also a pending PR to upstream an update to the script to pass a list of locales as parameters, and this might turn out to be useful for Pontoon too
https://github.com/mozilla/firefox-ios-build-tools/pull/22

Comment 1

[Matjaz Horvat [:mathjazz]]

AFAICT the script at https://github.com/mozilla/firefox-ios-build-tools/blob/master/scripts/update-xliff.py does 3 things for each localized XLIFF file:

1. Takes en-US file and updates/removes/adds targets.

2. Updates target-language (with locale mapping).

3. Fixes indent.

Running the script every time after our XLIFF serializer finishes its job would be a hack in our sync system and also suboptimal (e.g. in terms of I/O operations).

Instead, we should modify our existing serializer to match the functionality of the script. With the exception of locale maping (which would still need to be a bit hacky), other projects using XLIFF file format would also benefit from that approach.

Comment 2

[Francesco Lodolo [:flod]]

Uhm, what's the question? ;-)

I can confirm that the script does exactly what you're saying. Running the script on the output before commit would definitely be an hack (but also fast to implement), improving the serializer output would be a better/cleaner solution. The only limit is that we don't know how other systems using XLIFF (besides iOS) would create their files in the future.

I don't know how Pontoon stores the data internally before creating the XLIFF file, but it would be interesting to see if the diff is usable just applying the indent function.

Comment 3

[Matjaz Horvat [:mathjazz]]

(In reply to Francesco Lodolo [:flod] from comment #2)
> Uhm, what's the question? ;-)

Even if you don't know it, you answered it. ;) Sorry for not being more specific, I wanted to make sure I understand what the script does and hear your opinion on the proposed solution.

I'll add locale mapping to Pontoon and propose indent fix improvement to translate-toolkit. Before it's released, we'll use our custom fork.

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/fcfaeca080202a01883982de3a359b2cc852a7d6
Bug 1257812: Map locale codes for iOS

https://github.com/mozilla/pontoon/commit/09d95a456384ddbbb7c4853259cec15db76dd30a
Fix bug 1257812: Use translate toolkit fork with fix indent

https://github.com/mozilla/pontoon/commit/d9a90341dbb5b57a39ad9f39a2fe9cfa6d61c8fe
Merge pull request #375 from mathjazz/bug-1257812-xliff-output

Bug 1257812 xliff output

Comment 5

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/2df4c7011143065b397c05599c978104eaa1dc7a
Bug 1257812: Fix translate-toolkit and python-l20n egg fragments

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/9a4de43a36cc271f41c2314fa6ea7a6e23170cb2
Bug 1257812: Remove approved and state tags from .xliff