1433886 - Unicode directionality marks (LRM/RLM/ALM) do not interrupt cursive joining of adjacent characters, e.g. before Persian word suffix

Status: RESOLVED INVALID

(Core :: Layout: Text and Fonts, defect)

Description

[zahra]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20170303022339

Steps to reproduce:

1- opens my attachment which some of there words have two parts and the parts are separated by right to left mark in firefox.
2- open my attachment with ie and compare it with firefox.
3- in comparrison between firefox and ie, pay attention to the words:
كرده‏اى،
فرموده‏اى.
دهنده‏اى.



Actual results:

firefox skips right to left mark, combines two parts of word together and it makes reading web pages and pronunciation of farsi sentences unpleasant and incorrect at least for blind who uses screen reader like me!


Expected results:

firefox should consider right to left mark and does not ignore it, as it desplays all other characters and symbols.
i want behaviour of ie towards right to left mark in my favorite browser: firefox!

Comment 1

[zahra]

about:config has thousands of options and i sincerely appreciate if someone tell me which option should i change to firefox does not ignore right to left mark!
i said my problem to many iranians and they could not resolved my issue.
because i am not very technical in firefox about:config (specially complex items), and because this problem is very critical for me, i decided to report in bugzilla.
may someone can help me in this regard.
i am waiting for your help sincerely, i realy dont like other browsers and dont want to use any program except firefox.
but this bug makes reading my books very difficult!
i have hundreds of books which have hundreds of pages and i realy wish that read them with firefox without problem!
i tested many versions of firefox since version 3.6 with different versions of nvda screen reader and the result is the same!
God bless you all!

Comment 2

[Francesco Lodolo [:flod]]

Hi zahra, there are currently no attachments in this bug. Can you attach your examples?

As far as I can tell, Firefox doesn't have issues dealing with RTL marks.

Comment 3

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #2)
> Hi zahra, there are currently no attachments in this bug. Can you attach
> your examples?
> 
> As far as I can tell, Firefox doesn't have issues dealing with RTL marks.

hi.
unfortunately firefox ignores right to left mark (at least with the default settings of about:config).
i am sure that ie does not have this problem and even microsoft edge, when one time i tested in a computer of someone remotely.
i attached my file yesterday, but dont know whats the reason its removed from my report or even its not attached!

Comment 4

[zahra]

Attachment: doa.txt its one txt file which contains right to left mark and firefox ignores it.

Comment 5

[Francesco Lodolo [:flod]]

Attachment: Unicode example 1 (TXT)

Note: I'm afraid I can't help, but as someone who can't read Persian, it would really help to reduce the scope of the example.

This is Unicode's Example 1, with a rtl-mark before the exclamation point
https://www.w3.org/TR/WCAG20-TECHS/H34.html

I don't have IE available, but Chrome's behavior is different from Firefox.

Comment 6

[Francesco Lodolo [:flod]]

Attachment: Unicode example 1 (HTML)

While the behavior seems correct as HTML, with lang attribute defined.

Comment 7

[Francesco Lodolo [:flod]]

Not sure if this is the right component, in case feel free to move.

Comment 8

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #5)
> Created attachment 8946561 [details]
> Unicode example 1 (TXT)
> 
> Note: I'm afraid I can't help, but as someone who can't read Persian, it
> would really help to reduce the scope of the example.
> 
> This is Unicode's Example 1, with a rtl-mark before the exclamation point
> https://www.w3.org/TR/WCAG20-TECHS/H34.html
> 
> I don't have IE available, but Chrome's behavior is different from Firefox.

does chrome consider right to left mark as other characters and symbols?
is there any setting that i should change in about:config for firefox does not ignore it too?
i sincerely appreciate if someone finds a solution for me in this regard.

Comment 9

[zahra]

(In reply to zahra from comment #8)
> (In reply to Francesco Lodolo [:flod] from comment #5)
> > Created attachment 8946561 [details]
> > Unicode example 1 (TXT)
> > 
> > Note: I'm afraid I can't help, but as someone who can't read Persian, it
> > would really help to reduce the scope of the example.
> > 
> > This is Unicode's Example 1, with a rtl-mark before the exclamation point
> > https://www.w3.org/TR/WCAG20-TECHS/H34.html
> > 
> > I don't have IE available, but Chrome's behavior is different from Firefox.
> 
> does chrome consider right to left mark as other characters and symbols?
> is there any setting that i should change in about:config for firefox does
> not ignore it too?
> i sincerely appreciate if someone finds a solution for me in this regard.

the txt file was just one example.
it afects all htm and html files unfortunately too.
i did not see this problem in ie, can someone test with ie for me?
ie behaviour is desire behaviour and i am waiting for someone to send me one helpful option in about:config to change and get expected behaviour from firefox.

Comment 10

[Francesco Lodolo [:flod]]

If the use case is confirmed, it's a bug in Firefox, not something you can fix by changing settings in about:config

Does the HTML example attached here work as expected? Are you using Firefox in Persian or English?

P.S. try to avoid multiple comments in a row unless they help to figure out the issue, each one sends an email to several people ;-)

Comment 11

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #10)
> If the use case is confirmed, it's a bug in Firefox, not something you can
> fix by changing settings in about:config
> 
> Does the HTML example attached here work as expected? Are you using Firefox
> in Persian or English?
> 
> P.S. try to avoid multiple comments in a row unless they help to figure out
> the issue, each one sends an email to several people ;-)

i am sure its a bug in firefox, because at least ie and edge have not this issue.
but maybe its default behaviour is not good and maybe i change correct it by changing about:config
can you please search in about:config the words that maybe helpful for me in this regard to test?

i use firefox en-US, but i tested firefox version 22 persian and the result was the same.

Comment 12

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #10)
> If the use case is confirmed, it's a bug in Firefox, not something you can
> fix by changing settings in about:config
> 
> Does the HTML example attached here work as expected? Are you using Firefox
> in Persian or English?
> 
> P.S. try to avoid multiple comments in a row unless they help to figure out
> the issue, each one sends an email to several people ;-)


i forgot to say: your html attachment is in arabic, not persian and firefox did not show right to left mark for me.

Comment 13

[Francesco Lodolo [:flod]]

1) There are no changes in about:config if it's a bug. I hope that's clear at this point.

2) While the example is in Arabic, I set the lang attribute to Persian in the HTML example. As far as I can tell, the behavior is correct, and I haven't heard complaints about how Firefox treats HTML files. What do you mean that it does not show rtl marks?

3) In case, please always test with updated version of Firefox (22 is not).

Comment 14

[Anas Husseini [:linostar]]

Attachment: Screenshot of mixed text without RLM

Comment 15

[Anas Husseini [:linostar]]

Attachment: screenshot of the same mixed text with RLM

Comment 16

[Anas Husseini [:linostar]]

It was hard to see with the example that the reporter provided, since it didn't include two different languages with different directions, so Itested against the following URI:

data:text/html, hello %D8%A7%D8%AD%D9%85%D8%AF! bye

I added two snapshots taken from Firefox 58 (64-bit build for Windows 10), showing what that URI outputs without and with the RLM character after the exclamation point.

The issue claimed is not there, at least not anymore, unless the reporter was expecting a different behavior.

Comment 17

[Jonathan Kew [:jfkthame]]

If I am understanding the reporter correctly, the issue they're concerned about here is not about directionality (as shown by the mixed Latin/Arabic-script examples above); this is about joining behavior in relation to Persian suffixes like ‏-اى that are appended to a word with no intervening space, but should not be cursively joined.

Thus, reporter is mentioning words like كرده‏اى, where the suffix ‏-اى should not be joined to the root word:

  Bad: كردهاى
  OK:  كرده‌اى

The text file (attachment 8946553 [details]) has used U+200F RIGHT-TO-LEFT MARK before the suffix to try and achieve this.

However, this is not the correct way to write this in Unicode, and the fact that it happens to give the desired result in some environments (e.g. in IE, apparently) is actually a bug there. The RIGHT-TO-LEFT MARK character is categorized as "default-ignorable" in Unicode, and should by default have no effect on text display except for its specifically-defined effect on the bidirectional algorithm.

The proper control code for this situation is U+200C ZERO WIDTH NON-JOINER, because what needs to be controlled is joining behavior, not directionality.

See for example https://www.unicode.org/versions/Unicode10.0.0/ch09.pdf, page 374, where the example of Persian is specifically mentioned; see also recommendations like http://sartre2.byu.edu/persian/persianword/zwnj.htm and https://en.wikipedia.org/wiki/Persian_alphabet#Word_boundaries.

Comment 18

[Francesco Lodolo [:flod]]

Tomer (from our Hebrew team) also pointed me to bug 849404 regarding TXT files and RTL (fixed in 53).

Comment 19

[zahra]

you can test my file with notepad.
notepad behaves about right to left mark as expected.
i realy wish that find a solution for firefox.
i saw this problem in many web pages and many different files!
this file just one example.
i wish the behaviour of ie and notepad about right to left mark in firefox.

Comment 20

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #13)
> 1) There are no changes in about:config if it's a bug. I hope that's clear
> at this point.
> 
> 2) While the example is in Arabic, I set the lang attribute to Persian in
> the HTML example. As far as I can tell, the behavior is correct, and I
> haven't heard complaints about how Firefox treats HTML files. What do you
> mean that it does not show rtl marks?
> 
> 3) In case, please always test with updated version of Firefox (22 is not).

i realy cant use any windows versions of windows newer than xp, so the last version that i can have is only 52!
i tested with firefox 52 and any previous versions since 3.6 en-us and also version 22 of firefox persian interface.

Comment 21

[Anas Husseini [:linostar]]

I think I understand the reporter's issue now and confirmed it. Apparently, lots of Persian webpages use Right-to-Left Mark instead of the Zero-Width Joiner character. Microsoft IE, Edge, and other Microsoft non-web products deal with RLM when used between 2 persian letters as if it is a Zero-Width Joiner character. Firefox on the other hand, just ignores RLM if it isn't placed between letters from two reversely directed languages (like English and Persian). Firefox's behavior, although reasonable, is breaking existing Persian web content, especially for people with disabilities, who rely on a screen reader that will mix things up if RLM isn't behaving like the authors are usually using it for.

Comment 22

[Anas Husseini [:linostar]]

(In reply to Francesco Lodolo [:flod] from comment #18)
> Tomer (from our Hebrew team) also pointed me to bug 849404 regarding TXT
> files and RTL (fixed in 53).

This is a different issue. The RTL behavior itself is not the problem here, but the expected behavior of RLM character in persian/arabic/hebrew text alone is.

Comment 23

[Jonathan Kew [:jfkthame]]

After checking the Unicode standard further, and consulting some Unicode specialists, it's clear that the directional mark characters (e.g. RLM) are NOT supposed to interrupt joining of adjacent letters. These codes have General Category = Cf (format controls), and https://www.unicode.org/Public/10.0.0/ucd/ArabicShaping.txt states:

# Note: Code points that are not explicitly listed in this file are
# either of joining type T or U:
#
# - Those that are not explicitly listed and that are of General Category Mn, Me, or Cf
#   have joining type T.
# - All others not explicitly listed have joining type U.

So these characters, being GC=Cf, have Joining Type = T (transparent). This can be confirmed in https://www.unicode.org/Public/10.0.0/ucd/extracted/DerivedJoiningType.txt, where they are specifically listed.

And so in conclusion, the reporter's examples such as كرده‏اى, with RLM before the suffix, are mis-spelled; they should be using ZWNJ there. And software such as Notepad or IE that interrupts the joining is buggy.

Finally, according to my testing, Chrome shows the same behavior as Firefox here. So any pages/documents with this kind of encoding error will look wrong for a very large number of internet users; it's not just a Firefox issue.

Comment 24

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #17)
> If I am understanding the reporter correctly, the issue they're concerned
> about here is not about directionality (as shown by the mixed
> Latin/Arabic-script examples above); this is about joining behavior in
> relation to Persian suffixes like ‏-اى that are appended to a word with no
> intervening space, but should not be cursively joined.
> 
> Thus, reporter is mentioning words like كرده‏اى, where the suffix ‏-اى
> should not be joined to the root word:
> 
>   Bad: كردهاى
>   OK:  كرده‌اى
> 
> The text file (attachment 8946553 [details]) has used U+200F RIGHT-TO-LEFT
> MARK before the suffix to try and achieve this.
> 
> However, this is not the correct way to write this in Unicode, and the fact
> that it happens to give the desired result in some environments (e.g. in IE,
> apparently) is actually a bug there. The RIGHT-TO-LEFT MARK character is
> categorized as "default-ignorable" in Unicode, and should by default have no
> effect on text display except for its specifically-defined effect on the
> bidirectional algorithm.
> 
> The proper control code for this situation is U+200C ZERO WIDTH NON-JOINER,
> because what needs to be controlled is joining behavior, not directionality.
> 
> See for example https://www.unicode.org/versions/Unicode10.0.0/ch09.pdf,
> page 374, where the example of Persian is specifically mentioned; see also
> recommendations like http://sartre2.byu.edu/persian/persianword/zwnj.htm and
> https://en.wikipedia.org/wiki/Persian_alphabet#Word_boundaries.

H(In reply to Jonathan Kew (:jfkthame) from comment #17)
> If I am understanding the reporter correctly, the issue they're concerned
> about here is not about directionality (as shown by the mixed
> Latin/Arabic-script examples above); this is about joining behavior in
> relation to Persian suffixes like ‏-اى that are appended to a word with no
> intervening space, but should not be cursively joined.
> 
> Thus, reporter is mentioning words like كرده‏اى, where the suffix ‏-اى
> should not be joined to the root word:
> 
>   Bad: كردهاى
>   OK:  كرده‌اى
> 
> The text file (attachment 8946553 [details]) has used U+200F RIGHT-TO-LEFT
> MARK before the suffix to try and achieve this.
> 
> However, this is not the correct way to write this in Unicode, and the fact
> that it happens to give the desired result in some environments (e.g. in IE,
> apparently) is actually a bug there. The RIGHT-TO-LEFT MARK character is
> categorized as "default-ignorable" in Unicode, and should by default have no
> effect on text display except for its specifically-defined effect on the
> bidirectional algorithm.
> 
> The proper control code for this situation is U+200C ZERO WIDTH NON-JOINER,
> because what needs to be controlled is joining behavior, not directionality.
> 
> See for example https://www.unicode.org/versions/Unicode10.0.0/ch09.pdf,
> page 374, where the example of Persian is specifically mentioned; see also
> recommendations like http://sartre2.byu.edu/persian/persianword/zwnj.htm and
> https://en.wikipedia.org/wiki/Persian_alphabet#Word_boundaries.

(In reply to Jonathan Kew (:jfkthame) from comment #17)
> If I am understanding the reporter correctly, the issue they're concerned
> about here is not about directionality (as shown by the mixed
> Latin/Arabic-script examples above); this is about joining behavior in
> relation to Persian suffixes like ‏-اى that are appended to a word with no
> intervening space, but should not be cursively joined.
> 
> Thus, reporter is mentioning words like كرده‏اى, where the suffix ‏-اى
> should not be joined to the root word:
> 
>   Bad: كردهاى
>   OK:  كرده‌اى
> 
> The text file (attachment 8946553 [details]) has used U+200F RIGHT-TO-LEFT
> MARK before the suffix to try and achieve this.
> 
> However, this is not the correct way to write this in Unicode, and the fact
> that it happens to give the desired result in some environments (e.g. in IE,
> apparently) is actually a bug there. The RIGHT-TO-LEFT MARK character is
> categorized as "default-ignorable" in Unicode, and should by default have no
> effect on text display except for its specifically-defined effect on the
> bidirectional algorithm.
> 
> The proper control code for this situation is U+200C ZERO WIDTH NON-JOINER,
> because what needs to be controlled is joining behavior, not directionality.
> 
> See for example https://www.unicode.org/versions/Unicode10.0.0/ch09.pdf,
> page 374, where the example of Persian is specifically mentioned; see also
> recommendations like http://sartre2.byu.edu/persian/persianword/zwnj.htm and
> https://en.wikipedia.org/wiki/Persian_alphabet#Word_boundaries.

(In reply to Jonathan Kew (:jfkthame) from comment #17)
> If I am understanding the reporter correctly, the issue they're concerned
> about here is not about directionality (as shown by the mixed
> Latin/Arabic-script examples above); this is about joining behavior in
> relation to Persian suffixes like ‏-اى that are appended to a word with no
> intervening space, but should not be cursively joined.
> 
> Thus, reporter is mentioning words like كرده‏اى, where the suffix ‏-اى
> should not be joined to the root word:
> 
>   Bad: كردهاى
>   OK:  كرده‌اى
> 
> The text file (attachment 8946553 [details]) has used U+200F RIGHT-TO-LEFT
> MARK before the suffix to try and achieve this.
> 
> However, this is not the correct way to write this in Unicode, and the fact
> that it happens to give the desired result in some environments (e.g. in IE,
> apparently) is actually a bug there. The RIGHT-TO-LEFT MARK character is
> categorized as "default-ignorable" in Unicode, and should by default have no
> effect on text display except for its specifically-defined effect on the
> bidirectional algorithm.
> 
> The proper control code for this situation is U+200C ZERO WIDTH NON-JOINER,
> because what needs to be controlled is joining behavior, not directionality.
> 
> See for example https://www.unicode.org/versions/Unicode10.0.0/ch09.pdf,
> page 374, where the example of Persian is specifically mentioned; see also
> recommendations like http://sartre2.byu.edu/persian/persianword/zwnj.htm and
> https://en.wikipedia.org/wiki/Persian_alphabet#Word_boundaries.

hello.
i understand that its not correct way for separating such words using right to left mark, instead of ‌
but, i am blind user who use screen reader.
espeak and other tts have trouble in reading combined word, with right to left mark when firefox ignores it.
as you know, firefox has one address about:config
and we even can add more options to it.
i am sincerely appreciate you if you teach me which preference should i add to it to change the behaviour of firefox and make it like ie and notepad.
i have hundreds of books, many of them have this problem and i even see this behaviour in many web pages.
i sincerely appreciate you if you help me in this regard.
God bless you all!

Comment 25

[Khaled Hosny]

May be the issue with screen readers is something entirely different than what is reported here? I fail to see how Firefox’s visual rendering of the text affects how screen readers work since the screen reader would still be accessing the unchanged underlying text regardless of how Firefox is rendering it.

Comment 26

[Jonathan Kew [:jfkthame]]

(In reply to Khaled Hosny from comment #25)
> May be the issue with screen readers is something entirely different than
> what is reported here? I fail to see how Firefox’s visual rendering of the
> text affects how screen readers work since the screen reader would still be
> accessing the unchanged underlying text regardless of how Firefox is
> rendering it.

Yes, I suspect the screen reader would be unaffected by how the font shaping behaves, as it will be working with the underlying text. So what the user really needs is for the underlying text -- not just the visual rendering -- to be corrected.

Comment 27

[Khaled Hosny]

I just tested espeak from command line, it pronounces the words with ZWJ and RLM the same (as two words), so may be Firefox is not sending the RLM at all to the screen-reader (because it is default ignorable?).

Comment 28

[zahra]

(In reply to Khaled Hosny from comment #25)
> May be the issue with screen readers is something entirely different than
> what is reported here? I fail to see how Firefox’s visual rendering of the
> text affects how screen readers work since the screen reader would still be
> accessing the unchanged underlying text regardless of how Firefox is
> rendering it.


when one program, ignores right to left mark, text to speech programs, have trouble reading them.
for example nvda reads with wrong pronunciation.
it makes reading for screen readers difficult, unpleasant and with many wrong pronunciation.
there are many combined words that authors of books use right to left mark for separating them, so, its not only visual problem, it affects text to speech users very greatly!

i have hundreds of books both are in doc, docx format and also html books.
when i want to read my word books, since libreoffice does not support sayAll for screen readers, i should use preview in web browser and read them with my only favorite browser firefox!
when i read them with firefox, it ignores right to left mark and reading my books difficult!
fortunately, libreoffice does not ignore right to left mark and i did not see any program which ignores it until now excepf firefox!
what should i do realy?
can you please at least help me that which preference should i create in about:config of firefox to not ignore right to left mark?
i appreciate any help from all of you in this regard and sincerely pray for you always!
i am lover and advocator of opensource programs and wish that use them without such bugs!

Comment 29

[Jonathan Kew [:jfkthame]]

(In reply to zahra from comment #24)
> hello.
> i understand that its not correct way for separating such words using right
> to left mark, instead of ‌
> but, i am blind user who use screen reader.
> espeak and other tts have trouble in reading combined word, with right to
> left mark when firefox ignores it.
> as you know, firefox has one address about:config
> and we even can add more options to it.
> i am sincerely appreciate you if you teach me which preference should i add
> to it to change the behaviour of firefox and make it like ie and notepad.
> i have hundreds of books, many of them have this problem and i even see this
> behaviour in many web pages.
> i sincerely appreciate you if you help me in this regard.
> God bless you all!

Sorry, but there is no about:config setting that will make any difference here, because the Firefox code does not include an option to treat RLM differently. It simply conforms to the Unicode standard in this regard. The problem you are facing is that the text in question is spelled incorrectly, and because it is spelled incorrectly it displays and reads incorrectly as well.

The proper solution is for the authors of the text or pages to fix the spelling of their content. Wherever possible, I would recommend reporting the faulty text to the author or publisher, to encourage them to make corrections.

However, I realize that does not help you right now; some text may never be corrected, and even helpful authors may take some time to produce new versions. So as a workaround, I have created a Firefox add-on that tries to "fix" the text by adding a Zero-Width Non-Joiner character after any Right-to-Left Mark it finds in the content. This should correct the display (and perhaps also the text-to-speech result?) for pages that have this problem.

You can find this add-on at https://addons.mozilla.org/en-US/firefox/addon/rlm-add-zwnj/. I hope this may be helpful.

Note that the RLM-add-ZWNJ add-on may require a fairly up-to-date version of Firefox; it won't work in versions that are several years old. (I am not sure of exactly the earliest Firefox version that supports it.)

Comment 30

[zahra]

(In reply to zahra from comment #28)
> (In reply to Khaled Hosny from comment #25)
> > May be the issue with screen readers is something entirely different than
> > what is reported here? I fail to see how Firefox’s visual rendering of the
> > text affects how screen readers work since the screen reader would still be
> > accessing the unchanged underlying text regardless of how Firefox is
> > rendering it.
> 
> 
> when one program, ignores right to left mark, text to speech programs, have
> trouble reading them.
> for example nvda reads with wrong pronunciation.
> it makes reading for screen readers difficult, unpleasant and with many
> wrong pronunciation.
> there are many combined words that authors of books use right to left mark
> for separating them, so, its not only visual problem, it affects text to
> speech users very greatly!
> 
> i have hundreds of books both are in doc, docx format and also html books.
> when i want to read my word books, since libreoffice does not support sayAll
> for screen readers, i should use preview in web browser and read them with
> my only favorite browser firefox!
> when i read them with firefox, it ignores right to left mark and reading my
> books difficult!
> fortunately, libreoffice does not ignore right to left mark and i did not
> see any program which ignores it until now excepf firefox!
> what should i do realy?
> can you please at least help me that which preference should i create in
> about:config of firefox to not ignore right to left mark?
> i appreciate any help from all of you in this regard and sincerely pray for
> you always!
> i am lover and advocator of opensource programs and wish that use them
> without such bugs!

i sincerely appreciate your testing.
i did not see any screen reader or even tts program made iran which works with firefox in this regard.
i dont understand, whats the problem.
maybe firefox does not send this mark for screen readers, or firefox ignores it completely!
but i did not see this issue in any program except firefox until now!
please someone help me in this regard for using firefox for reading my books.
i can only use windows xp and so, the only solution for me is firefox 52, or changing one option in about:config (if its exists), or making new one for example new boolean or any types of options.

Comment 31

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #29)
> (In reply to zahra from comment #24)
> > hello.
> > i understand that its not correct way for separating such words using right
> > to left mark, instead of ‌
> > but, i am blind user who use screen reader.
> > espeak and other tts have trouble in reading combined word, with right to
> > left mark when firefox ignores it.
> > as you know, firefox has one address about:config
> > and we even can add more options to it.
> > i am sincerely appreciate you if you teach me which preference should i add
> > to it to change the behaviour of firefox and make it like ie and notepad.
> > i have hundreds of books, many of them have this problem and i even see this
> > behaviour in many web pages.
> > i sincerely appreciate you if you help me in this regard.
> > God bless you all!
> 
> Sorry, but there is no about:config setting that will make any difference
> here, because the Firefox code does not include an option to treat RLM
> differently. It simply conforms to the Unicode standard in this regard. The
> problem you are facing is that the text in question is spelled incorrectly,
> and because it is spelled incorrectly it displays and reads incorrectly as
> well.
> 
> The proper solution is for the authors of the text or pages to fix the
> spelling of their content. Wherever possible, I would recommend reporting
> the faulty text to the author or publisher, to encourage them to make
> corrections.
> 
> However, I realize that does not help you right now; some text may never be
> corrected, and even helpful authors may take some time to produce new
> versions. So as a workaround, I have created a Firefox add-on that tries to
> "fix" the text by adding a Zero-Width Non-Joiner character after any
> Right-to-Left Mark it finds in the content. This should correct the display
> (and perhaps also the text-to-speech result?) for pages that have this
> problem.
> 
> You can find this add-on at
> https://addons.mozilla.org/en-US/firefox/addon/rlm-add-zwnj/. I hope this
> may be helpful.
> 
> Note that the RLM-add-ZWNJ add-on may require a fairly up-to-date version of
> Firefox; it won't work in versions that are several years old. (I am not
> sure of exactly the earliest Firefox version that supports it.)

i sincerely appreciate your solution, i visited many different websites and have hundreds of books.
they dont accept my request and dont correct the misspelling words.
(maybe hundreds of such words) be in one books.
i told someone one time and he said: try different screen reader!
but whats the reason that all programs work properly in this regard except firefox?
can you please tell me does firefox completely ignore these marks or just does not send them for screen readers?

Comment 32

[Jonathan Kew [:jfkthame]]

(In reply to zahra from comment #31)
> can you please tell me does firefox completely ignore these marks or just
> does not send them for screen readers?

It certainly doesn't ignore them: it respects the directional-control behavior of RLM, and it respects the joining-control behavior of ZWNJ. However, it is possible that something different happens when interacting with screen readers. That would be an issue for the Accessibility component, it's a separate issue from how the text displays in Firefox.

If it is the case that both "كرده‏اى" (contains RLM) and "كرده‌اى" (contains ZWNJ) get pronounced as a single word by the screen reader, but get pronounced as two words when tested from the command line with the same screen reader software, then I would recommend filing a new bug in the "Core: Disability Access APIs" component, and hope that someone from the Accessibility team will be able to look into it further.

Comment 33

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #29)
> (In reply to zahra from comment #24)
> > hello.
> > i understand that its not correct way for separating such words using right
> > to left mark, instead of ‌
> > but, i am blind user who use screen reader.
> > espeak and other tts have trouble in reading combined word, with right to
> > left mark when firefox ignores it.
> > as you know, firefox has one address about:config
> > and we even can add more options to it.
> > i am sincerely appreciate you if you teach me which preference should i add
> > to it to change the behaviour of firefox and make it like ie and notepad.
> > i have hundreds of books, many of them have this problem and i even see this
> > behaviour in many web pages.
> > i sincerely appreciate you if you help me in this regard.
> > God bless you all!
> 
> Sorry, but there is no about:config setting that will make any difference
> here, because the Firefox code does not include an option to treat RLM
> differently. It simply conforms to the Unicode standard in this regard. The
> problem you are facing is that the text in question is spelled incorrectly,
> and because it is spelled incorrectly it displays and reads incorrectly as
> well.
> 
> The proper solution is for the authors of the text or pages to fix the
> spelling of their content. Wherever possible, I would recommend reporting
> the faulty text to the author or publisher, to encourage them to make
> corrections.
> 
> However, I realize that does not help you right now; some text may never be
> corrected, and even helpful authors may take some time to produce new
> versions. So as a workaround, I have created a Firefox add-on that tries to
> "fix" the text by adding a Zero-Width Non-Joiner character after any
> Right-to-Left Mark it finds in the content. This should correct the display
> (and perhaps also the text-to-speech result?) for pages that have this
> problem.
> 
> You can find this add-on at
> https://addons.mozilla.org/en-US/firefox/addon/rlm-add-zwnj/. I hope this
> may be helpful.
> 
> Note that the RLM-add-ZWNJ add-on may require a fairly up-to-date version of
> Firefox; it won't work in versions that are several years old. (I am not
> sure of exactly the earliest Firefox version that supports it.)

i sincerely appreciate your understanding my issue and your addon for me.
but i went to the page, pressed enter on add to firefox and recieved an error like this:
the file cant be download, because it maybe is corrupt.
whats the problem?

Comment 34

[Jonathan Kew [:jfkthame]]

(In reply to zahra from comment #33)
> but i went to the page, pressed enter on add to firefox and recieved an
> error like this:
> the file cant be download, because it maybe is corrupt.
> whats the problem?

What version of Firefox did you try? It installs successfully for me with release 57.

Comment 35

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #34)
> (In reply to zahra from comment #33)
> > but i went to the page, pressed enter on add to firefox and recieved an
> > error like this:
> > the file cant be download, because it maybe is corrupt.
> > whats the problem?
> 
> What version of Firefox did you try? It installs successfully for me with
> release 57.

i tested with firefox 51
as i said, i realy cant use other newer versions of windows and i promissed that at least use firefox 52 forever!

Comment 36

[Jonathan Kew [:jfkthame]]

I just tried with Firefox 52 (ESR) here, and it installed OK for me.

Comment 37

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #36)
> I just tried with Firefox 52 (ESR) here, and it installed OK for me.

can you please send me the file directly?
i tested with javascript both enabled and dissabled and the result was error!

Comment 38

[Jonathan Kew [:jfkthame]]

I don't actually know how to do that.... if I try to install it locally from the .zip file I created, it fails for me also; but installing from the addons.mozilla.org page works fine. I don't know why. Sorry! :(

Comment 39

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #38)
> I don't actually know how to do that.... if I try to install it locally from
> the .zip file I created, it fails for me also; but installing from the
> addons.mozilla.org page works fine. I don't know why. Sorry! :(

which settings should i change to allow installation of addons?
i changed many options and also about:config, but dont know which one may affect installing of addons.
i realy wish to use it and send you feedback.

Comment 40

[zahra]

(In reply to Jonathan Kew (:jfkthame) from comment #38)
> I don't actually know how to do that.... if I try to install it locally from
> the .zip file I created, it fails for me also; but installing from the
> addons.mozilla.org page works fine. I don't know why. Sorry! :(

now, i installed firefox 52.0esr and tried your link.
but the error was the same that cant install addons because the file may corrupt.

Comment 41

[Francesco Lodolo [:flod]]

Attachment: rlm_add_zwnj-1.0-an+fx.xpi

Try this: 

1) Download the file on your computer
2) Open about:addons
3) Drag the file on the window and confirm install

If this still doesn't work, I'm afraid there is some problem in your installation of Firefox, preventing add-ons from installing.

Comment 42

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #41)
> Created attachment 8947435 [details]
> rlm_add_zwnj-1.0-an+fx.xpi
> 
> Try this: 
> 
> 1) Download the file on your computer
> 2) Open about:addons
> 3) Drag the file on the window and confirm install
> 
> If this still doesn't work, I'm afraid there is some problem in your
> installation of Firefox, preventing add-ons from installing.

i made various changes in firefox about:config and options.
so, temporarily deleted my profile from app data to have fresh firefox.
your first method again said the corrupted file cant be install and your attachment, nvda said do you want to install addon?
i press enter on allow button many times, but did not work!
can you please test for me remotely?

Comment 43

[zahra]

whats your idea about making new preference in about:config of firefox esr and newer versions to make it configurable?
about:config has thousands of options and adding new one is very easy for you!
or help me which new option should i create to resolve the issue?
i also wish to test your addon and if it works as i want, just simply add it in the firefox installer file to all people can benefit of it.

Comment 44

[Francesco Lodolo [:flod]]

Point 1 should read "Right click and download the file on your computer".

Having said that:
* The problem cannot be solved by adding preferences to about:config
* Firefox 52 (ESR) is not going to get new functionalities, including preferences, since it's going to become unsupported in a few months.
* From Jonathan's explanation, Firefox is doing the right thing when it comes to display the text. If it's only a problem with screen readers (comment 32 has explanation to test that), it should be a new bug in a different component.

As for the inability to install the add-on, at this point I would suggest to try with SUMO, hopefully someone will be able to help you solve the issue
https://support.mozilla.org/en-US/questions/new/desktop/customize

Comment 45

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #44)
> Point 1 should read "Right click and download the file on your computer".
> 
> Having said that:
> * The problem cannot be solved by adding preferences to about:config
> * Firefox 52 (ESR) is not going to get new functionalities, including
> preferences, since it's going to become unsupported in a few months.
> * From Jonathan's explanation, Firefox is doing the right thing when it
> comes to display the text. If it's only a problem with screen readers
> (comment 32 has explanation to test that), it should be a new bug in a
> different component.
> 
> As for the inability to install the add-on, at this point I would suggest to
> try with SUMO, hopefully someone will be able to help you solve the issue
> https://support.mozilla.org/en-US/questions/new/desktop/customize

i am afraid that firefox esr is ended in recent month, so, i reported it hear, i did not understand whats the reason that adding new preference in about:config is not possible by myself on my system or just you add it in firefox core.
because if we press application key in about:config list, we have new which is submenu and it has boolean, integer, and String.
if Jonathan believes that its a correct behaviour, you can keep this preference as previous firefox until now, and let any user who needs it, have free choice to change it!

Comment 46

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #44)
> Point 1 should read "Right click and download the file on your computer".
> 
> Having said that:
> * The problem cannot be solved by adding preferences to about:config
> * Firefox 52 (ESR) is not going to get new functionalities, including
> preferences, since it's going to become unsupported in a few months.
> * From Jonathan's explanation, Firefox is doing the right thing when it
> comes to display the text. If it's only a problem with screen readers
> (comment 32 has explanation to test that), it should be a new bug in a
> different component.
> 
> As for the inability to install the add-on, at this point I would suggest to
> try with SUMO, hopefully someone will be able to help you solve the issue
> https://support.mozilla.org/en-US/questions/new/desktop/customize

i realy did not understand whats the reason that even i could not download your file.
i send it for someone and ask that send me the file.
about the link which you provided, i realy dont know how should i report my issue there, i dont want to make duplicate program.
just reported hear to get help from you and other developers.

Comment 47

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #44)
> Point 1 should read "Right click and download the file on your computer".
> 
> Having said that:
> * The problem cannot be solved by adding preferences to about:config
> * Firefox 52 (ESR) is not going to get new functionalities, including
> preferences, since it's going to become unsupported in a few months.
> * From Jonathan's explanation, Firefox is doing the right thing when it
> comes to display the text. If it's only a problem with screen readers
> (comment 32 has explanation to test that), it should be a new bug in a
> different component.
> 
> As for the inability to install the add-on, at this point I would suggest to
> try with SUMO, hopefully someone will be able to help you solve the issue
> https://support.mozilla.org/en-US/questions/new/desktop/customize

i asked someone to download and send your addon for me.
but when installing, firefox again says: corrup file and cant be installed!
whats the problem?

its title is:
rlm_add_zwnj-1.0-an+fx.xpi

Comment 48

[zahra]

i sent the link for someone to test.
it works as expected for him.
i sincerely request to put this in core of new firefox installer file.
maybe firefox 52.7 or something like this which supports it!
i realy need it and sincerely appreciate you in this regard!
God bless you and his infinite mercy and graces i pray for you for your great work!

Comment 49

[zahra]

firefox is very popular in iran.
also, many blind people use it, i share the link with others and encourage them to download and use your great work!
we are all appreciate your work,
i never forget your kindness, your helpful work and quick response and finding a solution for me.
just please put it in firefox installer exe file to use it forever and noone never face such big problem!

Comment 50

[Francesco Lodolo [:flod]]

@zahra
While I understand your struggle, and I'm really sympathetic to your problem, please understand that adding more comments in this bug is not going go change the situation. 

Firefox is currently behaving following the standards, Jonathan's add-on tries to work around errors in the source material, and that's your best chance. My suggestion is to try to solve the problem that prevents you from installing the add-on

Trying to figure out if the Persian community has a support channel in the meantime.

Comment 51

[zahra]

i just created account on mozilla website, (just for appreciation to you and supporting your great work!
your work is extremely excellent,
it was the first time that my issue is resolved in just two days!
i am so glad that dont know how should appreciate you and show my gratitude and sincere praying towards your great work.

Comment 52

[zahra]

i just created account on mozilla website, (just for appreciation to you and supporting your great work!
your work is extremely excellent,
it was the first time that my issue is resolved in just two days!
i am so glad that dont know how should appreciate you and show my gratitude and sincere praying towards your great work.

Comment 53

[zahra]

(In reply to Francesco Lodolo [:flod] from comment #50)
> @zahra
> While I understand your struggle, and I'm really sympathetic to your
> problem, please understand that adding more comments in this bug is not
> going go change the situation. 
> 
> Firefox is currently behaving following the standards, Jonathan's add-on
> tries to work around errors in the source material, and that's your best
> chance. My suggestion is to try to solve the problem that prevents you from
> installing the add-on
> 
> Trying to figure out if the Persian community has a support channel in the
> meantime.

i did not have the problem for installing addons before, while i dont use any addons, but did not install anything that preventing installation of addons.
i also dont trust and dont have anti virus programs and did not change anything about addons except deactivating automatic update of them and also pocket.

Comment 54

[Reza Habibi (rehb)]

(In reply to zahra from comment #53)
> (In reply to Francesco Lodolo [:flod] from comment #50)
> > @zahra
> > While I understand your struggle, and I'm really sympathetic to your
> > problem, please understand that adding more comments in this bug is not
> > going go change the situation. 
> > 
> > Firefox is currently behaving following the standards, Jonathan's add-on
> > tries to work around errors in the source material, and that's your best
> > chance. My suggestion is to try to solve the problem that prevents you from
> > installing the add-on
> > 
> > Trying to figure out if the Persian community has a support channel in the
> > meantime.
> 
> i did not have the problem for installing addons before, while i dont use
> any addons, but did not install anything that preventing installation of
> addons.
> i also dont trust and dont have anti virus programs and did not change
> anything about addons except deactivating automatic update of them and also
> pocket.


(In reply to zahra from comment #52)
> i just created account on mozilla website, (just for appreciation to you and
> supporting your great work!
> your work is extremely excellent,
> it was the first time that my issue is resolved in just two days!
> i am so glad that dont know how should appreciate you and show my gratitude
> and sincere praying towards your great work.


Hi @Zahra,

My name is Reza from Firefox Persian team.
I read all of the comments and It seems your problem solved. 
I totally understand your problem. However, Jonathan add-on can help you.
If you need any furthermore help regarding your problems, Please send your questions to the Persian mailing list:

[e-mail]

@flod, It seems the problem solved. However, we should look closer to this problem and maybe file a new bug. 

Bests,
Reza

Comment 55

[zahra]

(In reply to Reza Habibi (rehb) from comment #54)
> (In reply to zahra from comment #53)
> > (In reply to Francesco Lodolo [:flod] from comment #50)
> > > @zahra
> > > While I understand your struggle, and I'm really sympathetic to your
> > > problem, please understand that adding more comments in this bug is not
> > > going go change the situation. 
> > > 
> > > Firefox is currently behaving following the standards, Jonathan's add-on
> > > tries to work around errors in the source material, and that's your best
> > > chance. My suggestion is to try to solve the problem that prevents you from
> > > installing the add-on
> > > 
> > > Trying to figure out if the Persian community has a support channel in the
> > > meantime.
> > 
> > i did not have the problem for installing addons before, while i dont use
> > any addons, but did not install anything that preventing installation of
> > addons.
> > i also dont trust and dont have anti virus programs and did not change
> > anything about addons except deactivating automatic update of them and also
> > pocket.
> 
> 
> (In reply to zahra from comment #52)
> > i just created account on mozilla website, (just for appreciation to you and
> > supporting your great work!
> > your work is extremely excellent,
> > it was the first time that my issue is resolved in just two days!
> > i am so glad that dont know how should appreciate you and show my gratitude
> > and sincere praying towards your great work.
> 
> 
> Hi @Zahra,
> 
> My name is Reza from Firefox Persian team.
> I read all of the comments and It seems your problem solved. 
> I totally understand your problem. However, Jonathan add-on can help you.
> If you need any furthermore help regarding your problems, Please send your
> questions to the Persian mailing list:
> 
> [e-mail]
> 
> @flod, It seems the problem solved. However, we should look closer to this
> problem and maybe file a new bug. 
> 
> Bests,
> Reza

hi.
yes, the addon from jonathan can help me.
but i dont understand whats the reason that i cant install any addon in my firefox.
i wish that the change be included in new esr version of firefox, or at least, developers make new preference in about:config for users like me.
i sent you an email and will contact you for my questions and issues about about:config or other functionality of firefox.
thanks for the help from all of you and God bless you all!

Comment 56

[zahra]

hi again.
i tested with google chrome in the system of someone remotely,
the behaviour of chrome was like ie and other programs.
nvda reads the word correctly, does not ignore right to left mark and announces it when navigating letter by letter.
so, please include your addon in your next firefox please, to resolve the issue of many blind users that may incounter this problem, and also correct none standard books when displaying in firefox!

Comment 57

[zahra]

hello again.
since i use windows xp,
i cant test with newer versions of firefox.
but i asked one of my friends to test for me.
she tested with firefox 76 and the issue still persists.
can someone please help?
while i know that adding right to left mark is not standard for separating words,
but unfortunately many websites do this.
and firefox ignored right to left mark,
and it causes problems when nvda screen reader reads persian documents for me!
thanks so much for any of your help and God bless you!

Comment 58

[Reza Habibi (rehb)]

(In reply to zahra from comment #57)

hello again.
since i use windows xp,
i cant test with newer versions of firefox.
but i asked one of my friends to test for me.
she tested with firefox 76 and the issue still persists.
can someone please help?
while i know that adding right to left mark is not standard for separating words,
but unfortunately many websites do this.
and firefox ignored right to left mark,
and it causes problems when nvda screen reader reads persian documents for me!
thanks so much for any of your help and God bless you!

Hello Zahra,

Thank you for reporting.
Could you please send me a link to an example related to your problem? e.g. website with the same issue.

Thanks,
Reza

Comment 59

[zahra]

(In reply to Reza Habibi (rehb) from comment #58)

(In reply to zahra from comment #57)

hello again.
since i use windows xp,
i cant test with newer versions of firefox.
but i asked one of my friends to test for me.
she tested with firefox 76 and the issue still persists.
can someone please help?
while i know that adding right to left mark is not standard for separating words,
but unfortunately many websites do this.
and firefox ignored right to left mark,
and it causes problems when nvda screen reader reads persian documents for me!
thanks so much for any of your help and God bless you!

Hello Zahra,

Thank you for reporting.
Could you please send me a link to an example related to your problem? e.g. website with the same issue.

Thanks,
Reza

hello and thanks so much for your time and your help.
i sent you a book which demonstrates the issue as an example.
i explained problem details for you via email, (in persian),
i really hope that very soon, we see fix for this bug!
thanks for your help, God bless you and all developers and supporters of the best browser in the universe, (firefox)!