1377969 - Replace Suggested state with Missing and introduce Unreviewed Suggestions

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, enhancement, P2)

Description

[Matjaz Horvat [:mathjazz]]

A string is in a Suggested state when it has suggestions, none of which is approved. We show Suggested counts in dashboards and progress charts, we have a Suggested filter and we mark Suggested strings with blue color in the string list.

The nice thing about the Suggested state is that we can display it in the progress charts, because Translated + Suggested + Fuzzy + Missing = Total string count. But that doesn't justify its existence.

Instead of strings in a Suggested state, localizers and reviewers are usually more interested in the total number of Unreviewed Suggestions, which can also be present in Translated and Fuzzy strings. To support this user demand, see bug 1288956 and some of its duplicates (bug 1229335, bug 1317139, bug 1362940).

As part of this bug, we should:

1. Get rid of the Suggested state and treat such strings as Missing, because that's what they are in the repository (and product).

2. In dashboards, display a number of Unreviewed Suggestions (or the number of strings with Unreviewed Suggestions), linked to the Unreviewed Suggestions filter where possible (blocked by bug 1288956). Note that we will no longer be able to display such strings graphically in charts.

Comment 1

[Matjaz Horvat [:mathjazz]]

Let's use this bug as an opportunity to unify the variable names and UI names for different stats. Currenty, what is Translated in the UI is being referred to as "approved" in the code, and Suggested is called "translated".

Comment 3

[Staś Małolepszy :stas]

I'm catching up with the discussion on translation status and perhaps this has been mentioned before: would it make sense to fold the fuzzy strings and strings with unreviewed suggestions into a single "Unreviewed Strings" statistic?

Comment 4

[Staś Małolepszy :stas]

Essentially, that would be the number of strings that must be acted upon in order to show up in the localized product.  Note that this is different from the "Unreviewed Suggestions" statistics which, as Matjaž notes, may apply to Translated, Fuzzy and Missing strings alike.

Comment 5

[Matjaz Horvat [:mathjazz]]

Would you fold them on dashboards only, or other places as well? We have filters for each of them, we mark them with different status color in the string list, and we keep their counts separate in the model.

Note that we have an "Untranslated filter", which combines Suggested, Fuzzy and Missing into "strings that must be acted upon in order to show up in the localized product", but we don't display this sum in dashboards.

Comment 6

[Staś Małolepszy :stas]

For non-gettext projects Fuzzy isn't useful. I imagine the same is true for Contributor's dashboards?

I'm tempted to suggest that we fold them everywhere. The following would always hold true:

    total = approved + unreviewed + missing

Looking at https://pontoon.mozilla.org/nb-NO/amo/all-resources/?status=fuzzy&string=153308, is it common that Fuzzy strings have their own suggestions too?

"Fuzzy" really is an artifact of gettext's merging algorithm. "Unreviewed" (fuzzy + strings with unreviewed suggestions) seems more user-facing to me.

I'm still trying to build the mental model for the translation lifecycle so it's possible that I'm off here. Please take my proposal with a grain of salt.

Comment 7

[Staś Małolepszy :stas]

Actually, it might be necessary to leave "fuzzy" in the model.  It's important for the VCS sync.

For the dashboards, perhaps the following set of statistics would be useful?

    Total Strings
    Approved Strings
    Unreviewed Strings (fuzzy + strings with any number of unreviewed suggestions)
    Unreviewed Suggestions
    Missing Strings

I could see the API exposing those as well. If we see demand for it, we could add "Fuzzy Strings" and "Strings with Suggestions" too, but I wouldn't start with them right away.

Comment 8

[Michal Stanke (Mozilla.cz) [:mstanke][:MikkCZ]]

What about if the dashboard just behaves like quick filters (note that in this case "All" = "Translated" + "Untranslated", and the other categories do not sum to the total count)?

    All strings
    Translated strings (strings with an approved translation synced or waiting for sync to VCS, regardless if there are any suggestions pending review)
    Unreviewed suggestions (the same as "Translated", but only those with unreviewed suggestions)
    Suggested strings (the same as "Untranslated", but only those with unreviewed suggestions)
    Untranslated strings or Missing strings (strings with no approved translation in the VCS, regardless if there are any suggestions pending review)

I am not sure where the fuzzy strings should go. They are in VCS (=> "Unreviewed suggestions"), but AFAIK they won't display in the product or website and behave like missing (=> "Suggested strings").

Comment 9

[Michal Stanke (Mozilla.cz) [:mstanke][:MikkCZ]]

For the API, I think it would be nice to have filters and their combination as we know from the translation UI.

Comment 10

[Matjaz Horvat [:mathjazz]]

I agree we should remove Fuzzy where it doesn't make sense to keep it (like non-Gettext project dashboards or even Contributors dashboards), but we need to keep it in the model to be able to sync properly.

Let's file a separate bug for the "Fuzzy" discussion and use this bug to talk about about dropping "Suggested" and introducing "Suggestions".

> total = approved + unreviewed + missing

This is not true, because we might have unreviewed suggestions in approved, fuzzy or missing strings.

This is currently true:
    total = approved + suggested + fuzzy + missing

This will be true once we fix this bug:
    total = approved + fuzzy + missing

Also shown on dashboards (but not included) will be the "Unreviewed Suggestions", that will link to the translate UI filter with the same name. In bug 1403861 we decided to call this count "stringsWithSuggestions".

Comment 11

[Staś Małolepszy :stas]

The reason why this naming exercise is hard is that it conflates different types of statuses:
 
    1. Status from the product POV.

This is the binary "will it show up in the product?" status. Currently that's "approved" vs. everything else.

    2. Status from the VCS POV.

This answers the question: "will it be serialized to the VCS?". Currently that's: "approved", "fuzzy" for "yes" vs. everything else for "no".

    3. Status from the localizer's POV.

This answers the question: "how much work is needed to complete the task?":

  - approved translations without unreviewed suggestions are 0% work,
  - unreviewed suggestions for translations in any state are X amount of work,
  - fuzzy strings without unreviewed suggestions are Y amount of work, and
  - missing strings without unreviewed suggestions are Z amount of work,

where 0 < X < Y < Z.  Crucially, the suggestions might be attached to strings in any of the states from different categories (#1 and #2). It also looks like different kinds of strings "without unreviewed suggestions" might be a helpful category.

I think it would be helpful to understand which category of status we'd like to display in the dashboard. If the primary audience is localizers then going for #3 will make sense. For PMs, I imagine #1 be also useful. If that's true, the API will need to cater to both of the use-cases, but it could start by supporting just one.

Comment 12

[Michal Stanke (Mozilla.cz) [:mstanke][:MikkCZ]]

> This answers the question: "how much work is needed to complete the task?":
> 
>   - approved translations without unreviewed suggestions are 0% work,
>   - unreviewed suggestions for translations in any state are X amount of
> work,
>   - fuzzy strings without unreviewed suggestions are Y amount of work, and
>   - missing strings without unreviewed suggestions are Z amount of work,
> 
I am not sure if fuzzy for the localizers are different than suggestions. Basically it's a state, where the strings does not have any approved translation, but has some known option (either suggestion from a localizer or fuzzy import "from the sky"). I differs only on the level of VCS, if it's in the localization file or not.

The way how Pontoon represents the strings in translation view is different if the suggestion is for an existing translation or for a missing string, the content of the input textarea will be in different state, the first position in the string history will be in a different state. Mixing these strings together by design may be confusing. I have to agree it's necessary to make the cut if you want to keep all the number summing to the total count (i.e. having each string in a single category only), but more I am thinking about it, I doubt it's necessary to obey any other rule in the UI than "all = translated + missing".

Comment 13

[Staś Małolepszy :stas]

(In reply to Michal Stanke (Mozilla.cz) [:MikkCZ][:mstanke] (use needinfo) from comment #12)

> I am not sure if fuzzy for the localizers are different than suggestions.
> Basically it's a state, where the strings does not have any approved
> translation, but has some known option (either suggestion from a localizer
> or fuzzy import "from the sky"). I differs only on the level of VCS, if it's
> in the localization file or not.

Aren't fuzzy strings in general in need of more work? Especially if they come from similar strings, some tiny details will require adjusting. This is different from suggestions (which might not be good translations) but at least were made with this one string in mind.

>I have to agree it's necessary
> to make the cut if you want to keep all the number summing to the total
> count (i.e. having each string in a single category only), but more I am
> thinking about it, I doubt it's necessary to obey any other rule in the UI
> than "all = translated + missing".

Yeah, I agree. I didn't make it clear in my previous comment but I meant to say that the status from category #3 are more like tags and can apply to strings in any of the categories #1 and #2. As such they don't sum up to 100%.

Comment 14

[Matjaz Horvat [:mathjazz]]

The primary audience is localizers, which is why we're doing #3.

That means we're doing #1 and #2, too. Implicitly. I'm in favour of API doing that more explicitly.

Comment 15

[Staś Małolepszy :stas]

Yeah, I agree. Being explicit in the API will help make it future-proof.

What if we went for the following naming scheme for string statistics

    <VCS category> "Strings" ["WithSuggestions"]

This would give us the following statistics:

    totalStrings
    totalStringsWithSuggestions
    approvedStrings
    approvedStringsWithSuggestions
    fuzzyStrings
    fuzzyStringsWithSuggestions
    missingStrings
    missingStringsWithSuggestions

and for suggestions across all kinds of strings:

    unreviewedSuggestions

I think this is very close to what you proposed in comment 0 and in comment 10. Sorry it took me a bit longer to get to the same understanding :)

In bug 1403861 comment 16 I also suggested "WithPendingSuggestions" as an alternative to "WithSuggestions".

Comment 16

[Staś Małolepszy :stas]

Comment 15 fits better into bug 1403861. Collapsing it as obsolete.

Comment 17

[Matjaz Horvat [:mathjazz]]

I'd like to avoid expanding the scope of this bug beyond what's presented in Comment #0, so that we can resolve it soon. It's a pity that many suggestions currently fall through the cracks, because users don't see them on dashboards. Since we want to fix this in the current quarter, I'm shifting priority to P2.

Stas, do you think we need to further clarify the details here?

Comment 18

[Staś Małolepszy :stas]

This makes perfect sense. Sorry for hijacking this bug for the discussion about the translation status in general. It all started with an innocent question in comment 3 :)

Comment 22

[Eduardo Trápani]

I not sure about all those bugs being marked as duplicate. I mean, they might depend on this bug, but they cover different aspects of the tool and workflow.

Maybe we can keep them opened, depending on this one, so that we can test things (UI, import, ...) work as expected when this bug is solved. They might even help find a correct resolution (by checking it actually solves the curently other bugs).

Comment 23

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/pontoon/pull/926

Comment 24

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/46ffcb913bf945492b61787949f3199582ac1c23
Fix bug 1377969: Introduce unreviewed suggestions (#926)

This commit removes the Suggested translation status and introduces
Unreviewed suggestions. The benefit is that Unreviewed suggestions also
include suggestions submitted to Translated and Fuzzy strings, which
were previously inaccessible from dashboards and thus easy to miss.

Translations that were previously in a Suggested state are now treated
as Missing, which is what they are in the repository (and product), too.

Unreviewed suggestions are displayed on all dashboards and also
available as a filter. Since Unreviewed suggestions cannot be painted
on progress charts, we're adding a new sortable Unreviewed suggestions
column to dashboards, represented by a lightbulb, painted blue if
unreviewed suggestions are present.

We're also introducing a new calculate_stats management command, which
re-calculates statistics for all translated resources and corresponding
objects.

As part of this commit, we made a few other changes:
* Renamed translated_strings to unreviewed_strings.
* Exposed unreviewed_strings in the API.
* Upgraded to Font-Awesome 5.