1014672 - & character translated to &

Status: VERIFIED FIXED

(Firefox :: Translations, defect)

Description

[Majken Connor [:Kensie]]

http://www.mercotte.fr/recettes/macarons/

Translating from fr to en, & character translated to &

See attachments for images.

Comment 1

[Majken Connor [:Kensie]]

Attachment: ampersandtranslation-fr.png

Original french.

Comment 2

[Majken Connor [:Kensie]]

Attachment: ampersandtranslation-en.png

Translated to english

Comment 3

[Mike de Boer [:mikedeboer]]

I believe Felipe actually mentioned this yesterday on IRC!

Comment 4

[:Felipe Gomes (needinfo for replies!)]

Yep, that's the one. This is a bug in Bing's service, that happens only for the plain-text strings that we send to it which contains &. It doesn't happen for the text/html strings sent.

I tried Mike's suggestion to replace & with & instead of & but that didn't work :(  However, it revealed some very interesting tidbit. Even if I send using the hex version, like:

"Gâteaux & Entremets"

what we get back from the service is...:

"Cakes & desserts"


So it's clearly an internal bug in their output encoding. We will need to have a workaround. Some options: 

1 - Do not send as plain-text strings that we've checked that contain &
2 - For plain-text strings, do not encode & and carefully verify that it won't cause invalid XML to be sent
3 - Experiment sending the string wrapped in a CDATA section
4 - Assume that the bug will happen, and unescape & to & on the string received

All options are kinda awful, but I'm leaning towards #4 which is certainly the simpler and safer of them.

Comment 5

[Mike de Boer [:mikedeboer]]

This looks like a blocker bug for the Bing devs.

Meanwhile, there *is* a fifth option to work around this:

5. Un-escape escaped entities with a (simple) regex; so `&` would become `&` and `>gt;` --> >

`.replace(/&(#[0-9]{1,3}|[a-zA-Z]+);(:?#[0-9]{1,3}|[a-zA-Z]+);/, "&$1;");`

Comment 6

[:Felipe Gomes (needinfo for replies!)]

Attachment: trAmpersand

Yeah, though I've checked that this only happens with the & char, and it doesn't happen with < and > etc.. So I think we can go that way but with a simpler solution instead of a more comprehensive regexp.

Comment 7

[Mike de Boer [:mikedeboer]]

Comment on attachment 8434689 [details] [diff] [review]
trAmpersand

This looks 'good' to me ;)

Do we have a way to file this bug for the Bing devs? Or did we already do that?

Comment 8

[:Felipe Gomes (needinfo for replies!)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a010a557a1b5

They have support forums, I'll try posting it there

Comment 9

[:Felipe Gomes (needinfo for replies!)]

Marco, can you add this bug to the current iteration?

Comment 10

[Marco Mucci [:MarcoM]]

Added to Iteration 32.3

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/a010a557a1b5