1455191 - [FTL] Machinery presents strings as source

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, defect, P2)

Description

[Djfe]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Build ID: 20180415100032

Steps to reproduce:

Used Machinery.


Actual results:

The percentage wasn't correct.


Expected results:

Machinery should compare text to the "non-FTL" variant (the actual string)
two examples:
92% instead of 100%:
https://pontoon.mozilla.org/de/common-voice/all-resources/?string=178203

79% instead of 100%:
https://pontoon.mozilla.org/de/common-voice/all-resources/?string=178208

about the second string:
clicking gives you the exact same string
The referenced string:
https://pontoon.mozilla.org/de/common-voice/all-resources/?search=privacy-info&string=176569
None of them use html entities at all.

Comment 1

[Djfe]

weird, for other strings it doesn't happen:
https://pontoon.mozilla.org/de/common-voice/all-resources/?time=201803250706-201804190125&string=177865

Comment 3

[Matjaz Horvat [:mathjazz]]

To clarify: we store TranslationMemoryEntries of FTL Translation objects as source FTL, but don't convert them to simple text in Machinery.

That why:
1. Source strings and suggestions from TM appear as source FTL.
2. When you click on the suggestion it gets inserted to the textarea as source (bug 1455189).

We also need to prevent escaping of HTML from the source string to propagate to the source string in TM entries in Machinery, e.g. here:
https://pontoon.mozilla.org/de/common-voice/all-resources/?string=178208

Comment 4

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/pontoon/pull/959

Comment 5

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/620353442d2f76df46e83fc916e9b638fa04e7a9
Fix bug 1455191: [FTL] Convert TM strings to simple strings (#959)

Also:
* Use raw source string when querying TM
* Add parameter to distinguish between custom machinery queries and
  source string queries.
* Turn getSimplePreview into a as_simple_translation clone. It
  simplifies the code and reduces the number of arguments.
* Rename isPluralized function to reflect what it actually does.

Comment 6

[Djfe]

Thanks for working on a fix!
When will the change be live, so I can confirm that it's fixed? :)

Comment 7

[Matjaz Horvat [:mathjazz]]

It's already on prod!

Note the your first example (https://pontoon.mozilla.org/de/common-voice/all-resources/?string=178203) no longer returns any matches from TM, because with the changes introduced in the patch (Use raw source string when querying TM), the 255 character limitation of Postgres levenshtein implementation is exceeded.

The percentage mismatch in the second example (https://pontoon.mozilla.org/de/common-voice/all-resources/?string=178208) will be resolved as part of bug 1453341.

The other problems are fixed:
- key is no longer part of the TM strings
- HTML entities are presented unescaped

Comment 8

[Djfe]

ah, very good!
Those were exactly the two things I was wondering about and would've asked about now :)