Developers should use Verbatim in the same way for every project

RESOLVED INVALID

Status

Webtools Graveyard
Verbatim
RESOLVED INVALID
6 years ago
2 years ago

People

(Reporter: JasnaPaka, Unassigned)

Tracking

Trunk
x86_64
Windows 7

Details

(Reporter)

Description

6 years ago
First I wanted to write it as a comment for Bug 703095 but maybe new bug is better because it is different problem. At this moment it is real problem that every project use Verbatim in different way and it is source of problems.

Two examples:
* SUMO updates strings in Verbatim automatically, for Add-ons you must update from SVN manually.
* Some projects use primary SVN (or different VCS) and from this source are strings updated in Verbatim. Typical project which do it this way: Mozilla Add-ons. 

But some project do it in the different way. They use Verbatim as a primary place for new strings and from Verbatim are strings commited to the VCS by translators. Typical project which do it this way: Mozillians. 

Real problem is that you dont know if you should update strings from VCS for some project before you start with translation. Overwrite such operation new strings in Verbatim or not? Or try to imagine situation when you work with SVN directly (my situation for Mozillians project), you think all is translated but it isn't because someone updated strings in Verbatim and you don't know about it.

Ref:
* http://groups.google.com/group/mozilla.dev.l10n.web/browse_thread/thread/10932c95d5f59ba2#

Comment 1

6 years ago
maybe just a do work around to instruct people on how the system works, but maybe we can also think of a way for verbatim to keep track of how the system is in use and surface in the UI.
A workaround could be simply mentioningin the doc that some projects require you to manually update while others do not. In the new doc that I've created, I've separated out a section on manually updating the .po files for each project. If we know the differences between each project (automated update vs manual update) I can just identify which are which in that generic doc.

We could also determine some "updating guidelines" or "cautions" within the doc to avoid those circumstances in which some one has updated while another is working off-line on the project.

I think that ultimately it is in our best interest to standardize Verbatim's use across all projects, but for the time being we can use the doc to avoid these problems.

Pavel, can you email me the following:

1) a couple of cautions with Verbatim or updating guidelines (e.g., "If your team uses an external application for translating Verbatim projects your team may not capture all new, untranslated content when updating your project files from VCS. Be sure to coordinate all content updates with your localization team to avoid this problem."

2) A list of automatically updated projects & manually updated projects or someone I can talk to about gathering this info.
(Reporter)

Comment 3

6 years ago
(In reply to jbeatty from comment #2)
> A workaround could be simply mentioningin the doc that some projects require
> you to manually update while others do not. In the new doc that I've
> created, I've separated out a section on manually updating the .po files for
> each project. If we know the differences between each project (automated
> update vs manual update) I can just identify which are which in that generic
> doc.

Problem is that this is just a workaround. Maybe we will see new project in Verbatim next week (just small example, I don't know about such project) and question will be the same. I don't expect that developers will inform us about place where translations are primary updated. My expectation is the same process for every project. I'm too optimistic... I know.

> We could also determine some "updating guidelines" or "cautions" within the
> doc to avoid those circumstances in which some one has updated while another
> is working off-line on the project.

What about some education for developers?

> I think that ultimately it is in our best interest to standardize Verbatim's
> use across all projects, but for the time being we can use the doc to avoid
> these problems.

Agree.

> Pavel, can you email me the following:
> 
> 1) a couple of cautions with Verbatim or updating guidelines (e.g., "If your
> team uses an external application for translating Verbatim projects your
> team may not capture all new, untranslated content when updating your
> project files from VCS. Be sure to coordinate all content updates with your
> localization team to avoid this problem."
> 
> 2) A list of automatically updated projects & manually updated projects or
> someone I can talk to about gathering this info.

Ok.
(In reply to Pavel Cvrcek (Mozilla.cz) [:JasnaPaka] from comment #3)
> (In reply to jbeatty from comment #2)
> > A workaround could be simply mentioningin the doc that some projects require
> > you to manually update while others do not. In the new doc that I've
> > created, I've separated out a section on manually updating the .po files for
> > each project. If we know the differences between each project (automated
> > update vs manual update) I can just identify which are which in that generic
> > doc.
> 
> Problem is that this is just a workaround. Maybe we will see new project in
> Verbatim next week (just small example, I don't know about such project) and
> question will be the same. I don't expect that developers will inform us
> about place where translations are primary updated. My expectation is the
> same process for every project. I'm too optimistic... I know.

Hahaha, not too optimistic, just realistic :-) This, of course, is the ideal situation, but we have to provide a workaround until Verbatim can unify the processes (specifically with regards to updates).
> 
> > We could also determine some "updating guidelines" or "cautions" within the
> > doc to avoid those circumstances in which some one has updated while another
> > is working off-line on the project.
> 
> What about some education for developers?

I'm not sure I see how education for developers would be useful here. The people that need the education would be the localizers, until Verbatim has been modified to treat updating each project in one standard way (either automated or manual udpates, not both). 

> 
> > I think that ultimately it is in our best interest to standardize Verbatim's
> > use across all projects, but for the time being we can use the doc to avoid
> > these problems.
> 
> Agree.
> 
> > Pavel, can you email me the following:
> > 
> > 1) a couple of cautions with Verbatim or updating guidelines (e.g., "If your
> > team uses an external application for translating Verbatim projects your
> > team may not capture all new, untranslated content when updating your
> > project files from VCS. Be sure to coordinate all content updates with your
> > localization team to avoid this problem."
> > 
> > 2) A list of automatically updated projects & manually updated projects or
> > someone I can talk to about gathering this info.
> 
> Ok.
Thanks for your help Pavel! Also, I've made some updates to that doc tutorial to reflect some of these challenges. If you get a chance to see it, let me know what you think.

https://developer.mozilla.org/en/Localizing_with_Verbatim
(In reply to jbeatty from comment #4)
> (In reply to Pavel Cvrcek (Mozilla.cz) [:JasnaPaka] from comment #3)
> > (In reply to jbeatty from comment #2)
> > > A workaround could be simply mentioningin the doc that some projects require
> > > you to manually update while others do not. In the new doc that I've
> > > created, I've separated out a section on manually updating the .po files for
> > > each project. If we know the differences between each project (automated
> > > update vs manual update) I can just identify which are which in that generic
> > > doc.
> > 
> > Problem is that this is just a workaround. Maybe we will see new project in
> > Verbatim next week (just small example, I don't know about such project) and
> > question will be the same. I don't expect that developers will inform us
> > about place where translations are primary updated. My expectation is the
> > same process for every project. I'm too optimistic... I know.
> 
> Hahaha, not too optimistic, just realistic :-) This, of course, is the ideal
> situation, but we have to provide a workaround until Verbatim can unify the
> processes (specifically with regards to updates).
> > 
> > > We could also determine some "updating guidelines" or "cautions" within the
> > > doc to avoid those circumstances in which some one has updated while another
> > > is working off-line on the project.
> > 
> > What about some education for developers?
> 
> I'm not sure I see how education for developers would be useful here. The
> people that need the education would be the localizers, until Verbatim has
> been modified to treat updating each project in one standard way (either
> automated or manual udpates, not both). 
> 
> > 
> > > I think that ultimately it is in our best interest to standardize Verbatim's
> > > use across all projects, but for the time being we can use the doc to avoid
> > > these problems.
> > 
> > Agree.
> > 
> > > Pavel, can you email me the following:
> > > 
> > > 1) a couple of cautions with Verbatim or updating guidelines (e.g., "If your
> > > team uses an external application for translating Verbatim projects your
> > > team may not capture all new, untranslated content when updating your
> > > project files from VCS. Be sure to coordinate all content updates with your
> > > localization team to avoid this problem."
> > > 
> > > 2) A list of automatically updated projects & manually updated projects or
> > > someone I can talk to about gathering this info.
> > 
> > Ok.

Also, I created a new wiki page for this info, as I didn't feel that it belonged in a tutorial for beginning localizers but more as a reference for more advanced (or intermediate) users. Once I can get this info, I'll finish the page. Here's the URL. https://developer.mozilla.org/en/L10n_Projects_on_Verbatim

> Thanks for your help Pavel! Also, I've made some updates to that doc
> tutorial to reflect some of these challenges. If you get a chance to see it,
> let me know what you think.
> 
> https://developer.mozilla.org/en/Localizing_with_Verbatim

Comment 6

6 years ago
Let me explain a few things, mainly repeating what I said to Pavel on IRC:

(In reply to Pavel Cvrcek (Mozilla.cz) [:JasnaPaka] from comment #0)
> First I wanted to write it as a comment for Bug 703095 but maybe new bug is
> better because it is different problem. At this moment it is real problem
> that every project use Verbatim in different way and it is source of
> problems.

As I said on IRC, there's no much chance for developers to update translation in `their own way`, 'cause there's really only one way. I'll explain further down this post.

> Two examples:
> * SUMO updates strings in Verbatim automatically, for Add-ons you must
> update from SVN manually.

Not sure what you mean by automatically/manually, but if you're referring to a script that Wil uses to add new strings, then it's pretty much the same for sumo, it's just that SUMO team runs the script manually when needed, and AMO team has a cronjob, IIRC.

> * Some projects use primary SVN (or different VCS) and from this source are
> strings updated in Verbatim. Typical project which do it this way: Mozilla
> Add-ons. 
>
> But some project do it in the different way. They use Verbatim as a primary
> place for new strings and from Verbatim are strings commited to the VCS by
> translators. Typical project which do it this way: Mozillians.

This is not true. So, Verbatim works with translations that are on the same virtual box as Verbatim itself(or Pootle installation, if you will), and only a few people, none of which are developers of any projects on Verbatim, have ssh access to it, which is needed for them to be able to avoid updating SVN before updating Verbatim.

So, the process is following: Web developer uses a script to extract new strings into locale PO files, which merges with the current PO files, and then they commit the changes to SVN. After that, web developer pings someone from l10n-drivers team to update locales on Verbatim for given project, or they do it themselves, if they have admin privileges on Verbatim.

Now, when one updates a language for a project from SVN, Pootle not just updates the local SVN checkout of the project, but also merges new strings with existing files(the one on Verbatim box). So, the only thing that can go wrong is that Verbatim messes up something and marks some of the strings as `fuzzy`, but those strings will still be visible to a localizers, and they will be able to make them regular translations by just un-checking `fuzzy` box by the string translation. 

> Real problem is that you dont know if you should update strings from VCS for
> some project before you start with translation. Overwrite such operation new
> strings in Verbatim or not? Or try to imagine situation when you work with
> SVN directly (my situation for Mozillians project), you think all is
> translated but it isn't because someone updated strings in Verbatim and you
> don't know about it.

As I said before, you should always, no matter what, update strings from VCS in Verbatim before you start working. Updating strings from VCS, in a worst scenario can only mess up things in Verbatim, which we can possibly resolve, but it won't, under any circumstances, do anything to a remote repo if you don't specifically press the Commit to VCS button.

So, I'd say that you should always, when you come to Verbatim to work on translating a project, do the following:

1) Update from VCS
2) Check if your language has proper number of strings(check against templates - e.g. https://localize.mozilla.org/templates/affiliates/LC_MESSAGES/ )
3) If there's something wrong ping myself or someone else from l10n-drivers team
4) if everything is there, work on translation.
5) Once done, check if there's a commit in svn (http://svn.mozilla.org/projects/l10n-misc/trunk/myproject/locales/ab-CD/), and if there's a commit in between you updating the Verbatim translation and the moment you wanted to commit, then update from SVN again. This may cause you to lose some translations, or having some fuzzy strings, but that's better then messing up stuff on SVN. If that happens, and you update from SVN, revise your translation and go repeat steps 2-5.

I know this may be time consuming, but it's just the way that SVN works, and there's not really much we can do about it.

> Ref:
> *
> http://groups.google.com/group/mozilla.dev.l10n.web/browse_thread/thread/
> 10932c95d5f59ba2#

What's described in there certainly couldn't provoke any issues with strings on SVN, as you didn't commit after updating Verbatim. So, you either did something you forgot to mention, or your initial commit(from your working copy) messed up something.

Anyhow, given that this bug is about using django in the same way for every project, I'm marking it as invalid. If you do find some project that does not follow the rules I described, please reopen and let me know.

Thanks, and sorry for the long post :)
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → INVALID
(In reply to Milos Dinic [:Milos] from comment #6)
> Let me explain a few things, mainly repeating what I said to Pavel on IRC:
> 
> (In reply to Pavel Cvrcek (Mozilla.cz) [:JasnaPaka] from comment #0)
> > First I wanted to write it as a comment for Bug 703095 but maybe new bug is
> > better because it is different problem. At this moment it is real problem
> > that every project use Verbatim in different way and it is source of
> > problems.
> 
> As I said on IRC, there's no much chance for developers to update
> translation in `their own way`, 'cause there's really only one way. I'll
> explain further down this post.
> 
> > Two examples:
> > * SUMO updates strings in Verbatim automatically, for Add-ons you must
> > update from SVN manually.
> 
> Not sure what you mean by automatically/manually, but if you're referring to
> a script that Wil uses to add new strings, then it's pretty much the same
> for sumo, it's just that SUMO team runs the script manually when needed, and
> AMO team has a cronjob, IIRC.
> 
> > * Some projects use primary SVN (or different VCS) and from this source are
> > strings updated in Verbatim. Typical project which do it this way: Mozilla
> > Add-ons. 
> >
> > But some project do it in the different way. They use Verbatim as a primary
> > place for new strings and from Verbatim are strings commited to the VCS by
> > translators. Typical project which do it this way: Mozillians.
> 
> This is not true. So, Verbatim works with translations that are on the same
> virtual box as Verbatim itself(or Pootle installation, if you will), and
> only a few people, none of which are developers of any projects on Verbatim,
> have ssh access to it, which is needed for them to be able to avoid updating
> SVN before updating Verbatim.
> 
> So, the process is following: Web developer uses a script to extract new
> strings into locale PO files, which merges with the current PO files, and
> then they commit the changes to SVN. After that, web developer pings someone
> from l10n-drivers team to update locales on Verbatim for given project, or
> they do it themselves, if they have admin privileges on Verbatim.
> 
> Now, when one updates a language for a project from SVN, Pootle not just
> updates the local SVN checkout of the project, but also merges new strings
> with existing files(the one on Verbatim box). So, the only thing that can go
> wrong is that Verbatim messes up something and marks some of the strings as
> `fuzzy`, but those strings will still be visible to a localizers, and they
> will be able to make them regular translations by just un-checking `fuzzy`
> box by the string translation. 
> 
> > Real problem is that you dont know if you should update strings from VCS for
> > some project before you start with translation. Overwrite such operation new
> > strings in Verbatim or not? Or try to imagine situation when you work with
> > SVN directly (my situation for Mozillians project), you think all is
> > translated but it isn't because someone updated strings in Verbatim and you
> > don't know about it.
> 
> As I said before, you should always, no matter what, update strings from VCS
> in Verbatim before you start working. Updating strings from VCS, in a worst
> scenario can only mess up things in Verbatim, which we can possibly resolve,
> but it won't, under any circumstances, do anything to a remote repo if you
> don't specifically press the Commit to VCS button.
> 
> So, I'd say that you should always, when you come to Verbatim to work on
> translating a project, do the following:
> 
> 1) Update from VCS
> 2) Check if your language has proper number of strings(check against
> templates - e.g.
> https://localize.mozilla.org/templates/affiliates/LC_MESSAGES/ )
Can you give an example here of checking against the template? Is th template updated when the user updates from VCS? Wouldn't the template only include the total number of strings instead of the total number of untranslated strings displaying in the locale's repo (which would be different because it would have already been compared against previously translated strings)? If so, how would this indicate to the locale that the proper number of strings has been imported?
> 3) If there's something wrong ping myself or someone else from l10n-drivers
> team
Is there anything other than a mismatching number of new strings that could be considered "wrong"?
> 4) if everything is there, work on translation.
> 5) Once done, check if there's a commit in svn
> (http://svn.mozilla.org/projects/l10n-misc/trunk/myproject/locales/ab-CD/),
> and if there's a commit in between you updating the Verbatim translation and
> the moment you wanted to commit, then update from SVN again. This may cause
> you to lose some translations, or having some fuzzy strings, but that's
> better then messing up stuff on SVN. If that happens, and you update from
> SVN, revise your translation and go repeat steps 2-5.
> 
> I know this may be time consuming, but it's just the way that SVN works, and
> there's not really much we can do about it.
> 
> > Ref:
> > *
> > http://groups.google.com/group/mozilla.dev.l10n.web/browse_thread/thread/
> > 10932c95d5f59ba2#
> 
> What's described in there certainly couldn't provoke any issues with strings
> on SVN, as you didn't commit after updating Verbatim. So, you either did
> something you forgot to mention, or your initial commit(from your working
> copy) messed up something.
> 
> Anyhow, given that this bug is about using django in the same way for every
> project, I'm marking it as invalid. If you do find some project that does
> not follow the rules I described, please reopen and let me know.
> 
> Thanks, and sorry for the long post :)
(Assignee)

Updated

2 years ago
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.