Closed Bug 790797 Opened 12 years ago Closed 12 years ago

KPI dashboard: taking levels into account on L10n coverage chart

Categories

(support.mozilla.org :: Knowledge Base Software, task, P2)

Tracking

(Not tracked)

RESOLVED FIXED
2012Q4

People

(Reporter: atopal, Assigned: rrosario)

References

Details

(Whiteboard: u=sumo-team c=kpidash p=2 s=2012.22)

As a SUMO KPI dashboard user I would like to get more accurate  information about the L10n coverage, so that the coverage is calculated  with the edit levels taken into account.

    https://sumo.etherpad.mozilla.org/99
Can we get a final algorithm/formula commented here?
Ricky, can you clarify what you mean by formula? We'll keep using the current one with these modifications to take levels into account:

* A level 2 review change drops the localization to 50% of the article
* A level 3 review change drops the localization to 0% of the article
* More than one level 2 review change drops the localization to 0%
Priority: -- → P2
Whiteboard: u=contributor c=wiki p= → u=sumo-team c=kpidash p= s=2012.22
We need to add some smarts to our algorithm and come up with more complicated test cases to verify. ~2pt
Whiteboard: u=sumo-team c=kpidash p= s=2012.22 → u=sumo-team c=kpidash p=2 s=2012.22
Assignee: nobody → rrosario
In a pull request:
https://github.com/mozilla/kitsune/pull/952

BTW, based on yesterday's database dump, coverage goes from 56% to 59% with the new forumula.
Ricky, that's without English though, right? The new KPI is a coverage percentage: What percentage of SUMO users is seeing content in their own language?

From the Etherpad: 

* Coverage is calculated from visitor numbers of a locale overall and the ranking of the article in particular. Example:
** German makes up 10% of our visitor base. 
** The home page article makes up 10% of our top 50 visits
** When the Home page article is the only localized article and it's up to date in German, then our overall coverage is 10% of 10%=1% overall (+ the English coverage)
** When German misses a level 2 change, the coverage drops to 50%, so 5% of 10%= 0.5% overall
(In reply to Kadir Topal [:atopal] from comment #5)
> Ricky, that's without English though, right? The new KPI is a coverage
> percentage: What percentage of SUMO users is seeing content in their own
> language?
Correct. I just adjusted for the 1 new MEDIUM SIGNFICANCE revision counting for 50%.
Ricky, please also add the English coverage to the total coverage number. Since this is a KPI it would do a better job of reflecting what percentage of our visitor base we are serving in their own language at any moment in time.
(In reply to Kadir Topal [:atopal] from comment #7)
> Ricky, please also add the English coverage to the total coverage number.
> Since this is a KPI it would do a better job of reflecting what percentage
> of our visitor base we are serving in their own language at any moment in
> time.

OK, Can you give me a new definition for the legend (currently, "Definition: L10n coverage is a measure of the amount of translations of the English top 50 articles that are up to date, weighted by the number of visits for each locale. "). Also, does the title still make sense?
Flags: needinfo?(a.topal)
If you include English, the coverage is 80% (based on yesterday's db dump).

We can probably get a better number once we have a few months google analytics data per article.
Definition: 
L10n coverage is a measure of the percentage of our visitor base that get served with KB articles in their own language. It's calculated by taking into account the English top 50 weighted by the localization status of an article and the the number of visits for each locale.

Edit levels are taken into account as follows:
* A level 2 review change drops the localization to 50% of the article
* A level 3 review change drops the localization to 0% of the article
* More than one level 2 review change drops the localization to 0%

Example:
* German makes up 10% of our visitor base. 
* The 'how to set your home page' article makes up 10% of our top 50 visits
* When the Home page article is the only localized article and it's up to date in German, then our overall coverage is 10% of 10%=1% overall (+ the English coverage)
* When German misses a level 2 change, the coverage drops to 50%, so 5% of 10%= 0.5% overall
Flags: needinfo?(a.topal)
Awesome. I updated the pull request:

https://github.com/mozilla/kitsune/pull/952
Landed and deployed:

https://github.com/mozilla/kitsune/commit/f3364ccd3aa64d7db379d14f34af9b3eb2409218
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.