944858 - Double quot and single quot error in Persian texts

Status: RESOLVED INVALID

(Webtools Graveyard :: Verbatim, defect)

Description

[Amir Farsi]

Attachment: DubleQuots.jpg

User Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36

Steps to reproduce:

I wrote a translated text in Persian language, but in places which has a XML code or URL, i must use English double quotes and also in Persian texts, English single quotes is not usable and applicable. But verbatim shows double quotes and single quotes error. 


Actual results:

verbatim shows double quotes and single quotes error.


Expected results:

Must not show these errors.

Comment 1

[Dwayne Bailey]

@Amir a link to the unit is often more helpful then the screenshot, you can see the unit number at the bottom of the screen "String 1387755"

I'm not sure I 100% understand the issue.

Please be aware that these tests try as best they can to identify errors, but sometimes they are wrong.  To get things rights 100% of the time would make the test expensive.

In this case:
Single quote - these are used in - user's and someone's - so this can be ignored.  The no entry sign on the test cna be used to mark this as a false positive i.e. something where you feel you are correct and the test is wrong.  Be very sure that you are correct.

Double quote - not sure why that appears, I suspect because you use a double quote character in the target text.

We have some adjustments for Persian in our tests, specifically for these tests.  If you could have a look at them and give any feedback that would be useful:
https://github.com/translate/translate/blob/master/translate/lang/test_fa.py#L14-L28

Comment 2

[Amir Farsi]

Let me more describe this problem.

Single quote - For Persian texts the system(Verbatim) must be ignore that single quotes which are part of texts such as someone's and users's, because the we don't use the single quot in Persian like English for sentences and words abbreviation.

Double quote -The English double quotes aren't usable in Persian texts, but English double quotes are required for XML codes processing in Persian texts.

But i'm also suspect as well as you, because if you attend to attached picture, you can see the xml code in that source text(Original English text) haven't double quote(This row haven't double quote), and i doesn't use double quotes as well as source(original English) text, but verbatim(pootle) shows double qoute error. I think this is the third problem also!

Comment 3

[Amir Farsi]

Also about that python code on github, I'm not sure(because i don't know python but i know programming) ,but that code only works on converting Persian styles of single quote and double quotes from Persian style to English style or it make the system be able for understand what the mean of Persian style of single quot and double quotes. It's useful but i think can't processes and solve that problem which i reported.

Comment 4

[(no longer active)]

Sorry I don't know what that Verbatim check is checking for...

Comment 5

[Amir Farsi]

I think Mozilla must be change translation software from pootle to another one. Because pootle have many problems with bidi languages such as Persian(Farsi). Sara mansouri informed about a new translation software, named glotpress in here https://medium.com/my-experience-with-opw-and-mozilla/localisation-experience-9a67945ba035
Is anyone in mozilla to fix this problem? I have this problem in locomotion also.

Comment 6

[Amir Farsi]

Attachment: Pootle-FaProblem.jpg

Comment 7

[Amir Farsi]

Attachment: Pootle-FaProblem2.jpg

Comment 8

[Amir Farsi]

This problem is one of long-term bugs in Mozilla. From 2013-11-29 to today not solved yet!

Comment 9

[Dwayne Bailey]

Hi Amir, just wanted to flag some issues here:

1. The checks have actually changed quite a bit on Pootle vs Verbatim in that we distinguish serious errors from stylistic warnings.  So you at least know it is stylistic.
2. These checks can always be marked as false positives, that is always an option as this test is really designed for you to double check if anything was missed.  We can't always make checks that are 100% correct and thus we need to be able to mark false positives from time to time.
3. The checks sometimes aren't ideal for certain locale and scripts and we can switch them off in that case.
4. We decided it wasn't ideal to try and guess if the source text was using x's or xs' type syntax in English before applying the test.  So in these cases it is always a false positive.

With pictures I'm afraid I can't help much.  If you can link to actual units or provide the actual text of a failing test then we can actually replicate it and determine if their is a code issue.

But lets take this onto an issue in Pootle's issue on github.
https://github.com/translate/pootle/issues/4201

I managed to find a unit on locamotion that exhibits the concern.  Lets discuss this there, it would be helpful if you could assist in narrowing down the test PO file.

Comment 10

[Amir Farsi]

Hi Dwayne.
You can visit here as a sample: 
http://mozilla.locamotion.org/fa/translate/#filter=checks&checks=doublequoting&unit=4092240

Then you can click on next and previews (Control+Down and Control+Up) for more texts which have double quote error.

Comment 11

[Dwayne Bailey]

Hi Amir, I think you might have misunderstood me.  Can we try solve this over on the upstream bug?

I can pick up the ones that are in English and showing this issue. But since I can't read Persian I was hoping you could point to a specific and preferably small instance that we can use to debug the issue.  Thanks for the help.

Comment 12

[Amir Farsi]

Sara, Ehsan, And arash. Dwayne is True, he can't read Persian.
Can you help me to describe problem for Dwayne better?

Comment 13

[(no longer active)]

Hi Amir,

Honestly I'm a bit confused about the issue.  Since I'm not very familiar with pootle, I may be missing what's wrong, even though it may be obvious!  Do you mind describing the problem you're seeing please?  (Feel free to describe in Persian if that's easier, I can translate back for Dwayne!)

Comment 14

[Dwayne Bailey]

Hi Amir,  I think I understand the problem.  But I'd like to replicate it with really really simple examples.  Ideally one string that is very short in Persian that shows the problem. So description isn't an issue, a very simple example is.  If anyone is able to use pofilter and craft a very simple PO file that replicates the issue that work work. Or pop into #pootle on irc.freenode.net and I can try pair with someone to craft a very very simple failure.  Once we have a failure narrowed down replication should be easy.

Hope that's a bit clearer.

Comment 15

[Amir Farsi]

Dwayne. I have poedit software. I worked with it.
Do you want a po file with only one simple Persian string?
What i can do with poedit software for you.
Please describe step by step...

Comment 16

[Dwayne Bailey]

I'm not sure if poedit can do it.

But I need a simple example with:

msgid "English"
msgstr "Persian"

You can construct that in a text editor.

This should have the expected bad quote check.  If you are on linux and able to install the Translate Toolkit then running:

pofilter -t singlequote --language=fa test.po

Should be able to tell if this is making an error.  Hope that helps.

But if all you are able to do is give me that simple failure above that would be fine.  A short example from Pootle would also be fine.

Comment 17

[Arash Mousavi [:Arash-M]]

First you have misspelled my email address.

Then I have to say that the bug is invalid, and what Pootle does is actually correct. In Persian we don't have double/single quote and they should be replace with »« characters. Form the Pootle tests[1] I can see this properly handled. Amir I fixed the same string you mentioned above[2] and you can see Pootle is no longer showing the error.

In other cases where a single quote is used and Pootle is giving wrong alarm, as Dwayne mentioned above, you should just press the red button near the alarm and turn it off. They're just false positive and should be ignored.

So there's no bug here.

[2] https://github.com/translate/translate/blob/master/translate/lang/test_fa.py
[1] http://mozilla.locamotion.org/fa/translate/#filter=checks&checks=doublequoting&unit=4092240

Comment 18

[Dwayne Bailey]

I spent some time testing this with Amir on IRC and came to the same conclusion as Arash in comment 17.

It might help to understand how this test works.  For the sake of the test we do the following:
1. Convert the source text to have the expected quotes.  In our case both ' and " are converted to »«
2. Then test to see that the counts of »« are the same between source and target.

This might help explain why we get both single and double quote errors.  We could try refine that but I'd rather be reducing those errors once Persian has eliminated the ones the first class of valid errors.

I think we can mark this as invalid.

Comment 19

[Amir Farsi]

Thanks Arash for your description.
Thanks Dwayne for your very good support.