1506167 - Avoid deleted suggestion to be displayed as Translation Memory

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, defect, P2)

Description

[Peter Pin-Guang Chen [:petercpg] (MozTW.org)]

STR:
1. Open https://pontoon.mozilla.org/zh-TW/common-voice/all-resources/?string=192540
2. Switch to Machineary Translation Tab
3. Check suggestion labeled as Translation Memory

Actual:
Found '匈牙利語' (Hungarian) in suggestion

Correct:
'匈牙利語' should not be listed, as it is sourced from a rejected suggestion in https://pontoon.mozilla.org/zh-TW/mdn/all-resources/string=162090

Comment 1

[Matjaz Horvat [:mathjazz]]

That match actually doesn't come from this rejected string:
https://pontoon.mozilla.org/zh-TW/mdn/all-resources/string=162090

Its translation has been deleted (back when we had that option). As part of this bug we need to clean up our TM table of all entries with deleted Translation instances *and* existing Entity instances (if Entity is also deleted, it means the source file has been removed - bug 1468840):

> TranslationMemoryEntry.objects.filter(
>   translation__isnull=True,
>   entity__isnull=False,
> ).delete()

Some numbers to better illustrate the impact of this change:

> >>> TranslationMemoryEntry.objects.count()
> 5400648
> >>> TranslationMemoryEntry.objects.filter(translation__isnull=True).count()
> 903866
> >>> TranslationMemoryEntry.objects.filter(translation__isnull=True, entity__isnull=False).count()
> 109243

Note that we delete entries from TM when translations they originate from get *explicitly* rejected via Reject or Mass Reject options.

And while we keep entries of *implicitly* rejected translations in the DB (for why, see https://bugzilla.mozilla.org/show_bug.cgi?id=1390047#c1), we never expose them - neither through Machinery nor via Download TM option.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/pontoon/pull/1129

Comment 3

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/8d71348cd55f436920e19965e53faa0c08d7af3c
Fix bug 1506167: Remove TM entries of deleted Translations (#1129)