1289385 - Sign-offs should be done by l10n-drivers instead of localizers

Status: RESOLVED FIXED

(Localization Infrastructure and Tools :: General, defect)

Description

[Axel Hecht [:Pike]]

In London, l10n-drivers decided to change the semantics of l10n sign-offs, see the following two blog posts:

https://blog.mozilla.org/l10n/2016/06/17/mozlondon-localization-sessions/
https://blog.mozilla.org/l10n/2016/07/11/change-in-sign-offs-process-for-firefox-and-firefox-for-android/

Let's document the background for this decision here.

The original intent of sign-offs was two-fold:

1) have a technically correct localization to ship to users of officially-branded builds
2) receive a signal from localizers when testing on unbranded channels got a the localization in hg into a good state that the team would like to ship to end-users.

The first point works good. We do find problems with the translation that would look like rather bad regressions to end-users, and we're having a tool to not ship those. Given that, we're not changing the role of sign-offs in how they're affecting what we put into our officially-branded builds.

The signal from localizers worked well for some communities.

For a pretty big chunk of localizers, sign-offs didn't exist in practice. Dwayne did sign-offs for teams on mozilla.locamotion.org, and he did so on technical metrics, not translation quality in terms of linguistic correctness.

And then there was another big chunk of localization teams, for which sign-offs were OK, but that forgot to sign-off, or had a sign-off, but for old revisions, etc.

As the person doing most sign-off reviews, I observed these caveats, and over the course of the past year, I had made a few gradual changes away from merely reviewing revisions that had been signed off by localizers or Dwayne.

This started with doing sign-offs on beta after merge days, when only the merge to beta triggered a new revision to have new data, nagging for sign-off on the team page. As those would be no-ops for, say Firefox if there was a newer Fennec commit, I would just sign-off for Firefox on beta. Also, merge commits would be an example where I would just move on.

On top of that, I started updating sign-offs by Dwayne to newer revisions, if they existed.

And then I started doing this for sign-offs done by localization teams, too.

The underlying idea always was "if this makes it better, let's take it".

Now, how did this look in practice? I went through the data of the last release cycles, and counted my sign-offs, for both fennec and firefox:

47 	218
46 	175
45 	77
44 	90
43 	134
42 	202
41 	2
40 	50
39 	3
38 	38
37 	3

As you can see, there are clear bumps for 38 and 42. Those were seasonal releases, and I took extra time to take as many translations as I could for those.

And then with 46 and 47, I just moved ahead and aggressively took things I could find, and updated the revisions of localizations just a cycle or two behind.

Fast forward to London: After the reorg of our team, flod and Delphine are put into the sign-off-review duty. We also realized that the signal we hoped to get through sign-offs as a sign that a particular team finished translating Firefox and/or Fennec that cycle lost its meaning, at least in a sense when comparing locales against each other.

Consequently, we decided to reshape the sign-off process to something that works consistently across all localization team and Firefox and Fennec.

Thus, we do sign-offs now completely inside the l10n-drivers team, and stop involving localizers in the technical sign-off process.

Using a needinfo on Jeff, Delphine, and flod to confirm this decision.

Comment 1

[Axel Hecht [:Pike]]

Forgot:

Consulted party over the course of this process:

We did talk with Dwayne on this.

Also, over the course of me changing my habits gradually, a few localizers noticed that I did that, and asked me about it. There were no objections after explaining what I did. I do have an email thread with Michal Stanke in my archives. Most of those conversations were on IRC, though, and thus are lost in the ways of electrons.

In that sense, we did get input on the changes as they gradually happened from some localizers.

We deemed a general concensus with the l10n community as not necessary for two reasons:
- what we published is only a small incremental change of what was happening in practice already
- it's less work and less process, who'd object?
- there's been no concerns with the few opportunities where I had the opportunity to spot check my habits.

Comment 2

[Francesco Lodolo [:flod]]

Thanks for filing the bug and digging up the stats from Elmo.

I can definitely confirm both the decision and rationale behind it. I would add that we made a similar decision for iOS a few months back (drop sign-offs completely), discussed about the change in following hackathons, and never received any complaints from localizers.

After one cycle of doing sign-offs on Aurora, I'm even more convinced that it was the right decision to make:
* We can start reviewing partial updates early in the cycle, spot errors, get in touch with localizers and even propose fixes in tools (Pontoon, Pootle). It happened countless times in the past weeks, and reaction from localizers for this kind of outreach has always been positive.
* There were plenty of locales that never got a sign-off, even if there were significant udpates in the pipes.
* It's easy to see real activity in locales once you rule out tools' automation, and spot dead locales as well. We also found out a few holes in shipping locales thanks to a cleaner dashboard, especially in Fennec territory.

Comment 3

[Delphine Lebédel [:delphine - Use needinfo]]

I agree with everything that has been detailed and explained here. Both flod and Axel have clearly described how and why l10n-drivers went ahead with this change, as well as the positive outcomes that have come out of this. Community health is now much more visible and we are shipping more localized work this way - while relieving localizers from a chore. This both simplifies their task, and gives more visibility to the most recent translations in our products. Definitely a win-win situation.

Comment 4

[Jeff Beatty [:gueroJeff]]

+1 to everything here.

This decision was made during MozLondon and based on conversations we'd had with localizers at l10n hackathons, All Hands meetings, the success of our eliminating sign-offs for Firefox for iOS, needs from release engineering, and the already gradual trajectory toward this in Axel and my sign-off review process.

Comment 5

[Axel Hecht [:Pike]]

Marking this as RESOLVED, and assigning to myself, as I've made mode of the decisions leading up to this one.