Support writing-mode (lr/rl) text attribute

ASSIGNED
Unassigned

Status

()

Core
Disability Access APIs
ASSIGNED
9 years ago
3 years ago

People

(Reporter: Aaron Leventhal, Unassigned)

Tracking

(Blocks: 1 bug, {access})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

14.73 KB, patch
MarcoZ
: review+
davidb
: review+
Details | Diff | Splinter Review
(Reporter)

Description

9 years ago
We need to expose the current text direction through this attribute:
http://wiki.mozilla.org/Accessibility/TextAttributes

And add to docs when done:
http://developer.mozilla.org/en/Accessibility/AT-APIs#Supported_Text_Attributes

Comment 1

8 years ago
I'm not sure it's worth to expose these until our text interface work right with "rl". In the meantime all is "lr" - we navigate DOM and accessibles trees from left to right always.

Comment 2

8 years ago
(In reply to comment #1)
> I'm not sure it's worth to expose these until our text interface work right
> with "rl". In the meantime all is "lr" - we navigate DOM and accessibles trees
> from left to right always.

Actually it's ok, we should navigate trees from left to right until a11y tree is built based on frame tree I think. Generally we create a11y tree based on frame tree for few cases only (for native anonymous content in HTML). We could handle them separately if they'll bring any problems.

Comment 3

8 years ago
Created attachment 381725 [details] [diff] [review]
patch
Assignee: nobody → surkov.alexander
Status: NEW → ASSIGNED
Attachment #381725 - Flags: review?(marco.zehe)
Attachment #381725 - Flags: review?(david.bolter)

Comment 4

8 years ago
Created attachment 381728 [details] [diff] [review]
patch2
Attachment #381725 - Attachment is obsolete: true
Attachment #381728 - Flags: review?(marco.zehe)
Attachment #381728 - Flags: review?(david.bolter)
Attachment #381725 - Flags: review?(marco.zehe)
Attachment #381725 - Flags: review?(david.bolter)

Updated

8 years ago
Attachment #381728 - Flags: superreview?(roc)
Comment on attachment 381728 [details] [diff] [review]
patch2

I'm still not sure we need to be creating new classes for each text attribute (thinking performance), but this patch fits the current style.

>+
>+////////////////////////////////////////////////////////////////////////////////
>+// nsDirTextAttr
>+
>+nsDirTextAttr::nsDirTextAttr(nsIFrame *aRootFrame, nsIFrame *aFrame) :
>+  nsTextAttr<PRUint32>(aFrame == nsnull), mRootFrame(aRootFrame)
>+{
>+  mIsRootDefined = GetDirection(mRootFrame, &mRootNativeValue);

You probably meant to use the passed in root frame right?


>+void
>+nsDirTextAttr::Format(const PRUint32& aValue, nsAString& aFormattedValue)
>+{
>+  // LTR and RTL directions are supported in mozilla only.

This isn't true out of context, but I'm not sure exactly what you want to say here.
(see http://www.quirksmode.org/dom/w3c_html.html for dir support)
Oops just saw you used the passed in root frame to initialize mRootFrame... please disregard.

Comment 7

8 years ago
(In reply to comment #5)

> >+void
> >+nsDirTextAttr::Format(const PRUint32& aValue, nsAString& aFormattedValue)
> >+{
> >+  // LTR and RTL directions are supported in mozilla only.
> 
> This isn't true out of context, but I'm not sure exactly what you want to say
> here.
> (see http://www.quirksmode.org/dom/w3c_html.html for dir support)

I didn't realized @dir doesn't support "top to bottom" direction. I just oriented to constants http://mxr.mozilla.org/mozilla-central/source/layout/base/nsStyleConsts.h#365

Comment 8

8 years ago
(In reply to comment #5)
> (From update of attachment 381728 [details] [diff] [review])
> I'm still not sure we need to be creating new classes for each text attribute
> (thinking performance), but this patch fits the current style.

Yes, it is. Would be nice to know some details how are class approach less performant comparing to non classes approach. What do you keep in mind?

Updated

8 years ago
Attachment #381728 - Flags: review?(bolterbugz) → review+
Comment on attachment 381728 [details] [diff] [review]
patch2

r=me

Let's make perf testing a goal for Q2 or Q3. We can stress test for text attributes and try using static util methods to compare with the class creation.
erm, Q3, or Q4 ;)
Comment on attachment 381728 [details] [diff] [review]
patch2

r=me, but I'd also like to see a test with a native right-to-left language such as Arabic or Hebrew.
Attachment #381728 - Flags: review?(marco.zehe) → review+

Comment 12

8 years ago
(In reply to comment #11)
> (From update of attachment 381728 [details] [diff] [review])
> r=me, but I'd also like to see a test with a native right-to-left language such
> as Arabic or Hebrew.

Probably it would complete the test from some point of view but it's not very helpful for us I think.
Alex, why do you think this test would not be helpful? Because if the right-to-left rendering of such languages is broken, other tests inside content or layout would already catch that?

Comment 14

8 years ago
(In reply to comment #13)
> Alex, why do you think this test would not be helpful? Because if the
> right-to-left rendering of such languages is broken, other tests inside content
> or layout would already catch that?

Marco, my point the Arabic or any other rl language text written from right to left looks the same as left to right text from point of view who doesn't know these languages :) I think neither of us can read it. Our test shows that DOM is rendered from right to left and we expose proper writing-mode text attribute. So I think if screen reader supports right to left mode then it should announce "text" as "txet" if author pointed dir="rlt" for example.
+  // LTR and RTL directions are supported in mozilla only.

I think you mean "only LTR and RTL directions are supported in Mozilla".

> So I think if screen reader supports right to left mode then it
> should announce "text" as "txet" if author pointed dir="rlt" for example.

No, that would make no sense. You would always read text in the logical order, the order of the characters in the DOM.

I'm not sure how screen readers should use this attribute. One thing that makes it difficult is that you're only reporting style information here, but the text itself has its own directionality. E.g. you might have direction:ltr, but if the text is all Arabic it will be rendered right-to-left anyway.

Comment 16

8 years ago
(In reply to comment #15)
> +  // LTR and RTL directions are supported in mozilla only.
> 
> I think you mean "only LTR and RTL directions are supported in Mozilla".

ok, thanks.

> > So I think if screen reader supports right to left mode then it
> > should announce "text" as "txet" if author pointed dir="rlt" for example.
> 
> No, that would make no sense. You would always read text in the logical order,
> the order of the characters in the DOM.
> 
> I'm not sure how screen readers should use this attribute. One thing that makes
> it difficult is that you're only reporting style information here, but the text
> itself has its own directionality. E.g. you might have direction:ltr, but if
> the text is all Arabic it will be rendered right-to-left anyway.

I think I don't mind. If Arabic text is placed from right to left in DOM already then it's great and nothing to do for us.

Comment 17

8 years ago
Robert, any other comments?
I don't understand what screen readers are expected to do with this attribute. If there's no way to use it reliably, we shouldn't provide it.

Comment 19

8 years ago
(In reply to comment #18)
> I don't understand what screen readers are expected to do with this attribute.
> If there's no way to use it reliably, we shouldn't provide it.

I think this attribute is a hint for screen readers. I would assume if screen readers get the text
<span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it should read "normal text" from left to right, "arabic text" from right to left since DOM order doesn't reflect text direction. I'm not sure about the case when arabic text is used but direction is ltr. We could try to manipulate with encoding but I'm not sure.

Any way I don't know if this text attribute is implemented by screen readers yet. So I think we need to expose something reasonable before we'll get some feedback.

> No, that would make no sense. You would always read text in the logical order,
> the order of the characters in the DOM.

I'm not sure I get you. I think when they write something then their caret position in the right of the screen and text is moved to left when they type symbols. So if would write "bla" in arabic editor then I should get "alb" if I would read from left to right. Therefore "alb" should be stored in DOM as well. If author put dir="rtl" on "alb" then we expose "alb" to screen reader with "rtl" text attribute and screen reader should read "bla" (from right to left). Do I understand something wrong?
(In reply to comment #19)
> I think this attribute is a hint for screen readers. I would assume if screen
> readers get the text
> <span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it
> should read "normal text" from left to right, "arabic text" from right to left
> since DOM order doesn't reflect text direction. I'm not sure about the case
> when arabic text is used but direction is ltr. We could try to manipulate with
> encoding but I'm not sure.

Normally you would use 'direction' on a block. I don't think <div style="direction:rtl"> should affect the reading of the text at all.

> > No, that would make no sense. You would always read text in the logical
> > order, the order of the characters in the DOM.
> 
> I'm not sure I get you. I think when they write something then their caret
> position in the right of the screen and text is moved to left when they type
> symbols. So if would write "bla" in arabic editor then I should get "alb" if I
> would read from left to right. Therefore "alb" should be stored in DOM as
> well.

No. If you type H E B then it would be displayed as "BEH" but the DOM would contain HEB. Normally the DOM contains characters in the order you type them and read them.

Comment 21

8 years ago
(In reply to comment #20)
> (In reply to comment #19)
> > I think this attribute is a hint for screen readers. I would assume if screen
> > readers get the text
> > <span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it
> > should read "normal text" from left to right, "arabic text" from right to left
> > since DOM order doesn't reflect text direction. I'm not sure about the case
> > when arabic text is used but direction is ltr. We could try to manipulate with
> > encoding but I'm not sure.
> 
> Normally you would use 'direction' on a block. I don't think <div
> style="direction:rtl"> should affect the reading of the text at all.

I just keep in mind language learning course or dictionary, for example 
<ul>
  <li>
    <span>cat</span> - <span dir="rtl">cat in arabic</span>
  </li>
</ul>

> 
> > > No, that would make no sense. You would always read text in the logical
> > > order, the order of the characters in the DOM.
> > 
> > I'm not sure I get you. I think when they write something then their caret
> > position in the right of the screen and text is moved to left when they type
> > symbols. So if would write "bla" in arabic editor then I should get "alb" if I
> > would read from left to right. Therefore "alb" should be stored in DOM as
> > well.
> 
> No. If you type H E B then it would be displayed as "BEH" but the DOM would
> contain HEB. Normally the DOM contains characters in the order you type them
> and read them.

If I have <p dir="rtl">HEB</p> then I see "HEB" in the right of screen when I read from left to right.
(In reply to comment #21)
> (In reply to comment #20)
> > No. If you type H E B then it would be displayed as "BEH" but the DOM would
> > contain HEB. Normally the DOM contains characters in the order you type them
> > and read them.
> 
> If I have <p dir="rtl">HEB</p> then I see "HEB" in the right of screen when I
> read from left to right.

The two of you are at cross-purposes. Roc means that if you type HEB in a right-to-left language, it is displayed BEH. For example, if I type ש, then ל, then ו, and then ם, it appears in right-to-left order as שלום, even in this left-to-right paragraph.

This does not mean that it should be stored in reverse order in DOM - that would break all sorts of things, charAt for example.

Comment 23

8 years ago
Ok, if I have

<p dir="rtl"> שלום - <span dir="ltr">example of right to left word</p>

then I should read שלום  (from right to left) and "example of right to left word" from left to right. Do I understand correct? If so then screen reader should do the same. But actually I think it's screen reader task to decide how to read this. We just should say where ltr word and where rtl word.
The point is that the what determines the reading order is not the dir attribute but the characters themselves. Even if you have

<p dir="ltr"> שלום - <span dir="rtl">example of right to left word</p>

you should also read שלום from right to left and "example of right to left word" from left to right.

Comment 25

8 years ago
Ok, please correct me if I'm wrong. @dir attribute is used to specify how the content should be oriented. Therefore we shouldn't rely on @dir attribute because we need to expose direction of text. So how can we know this word is rtl but this one is ltr?
(In reply to comment #25)
> Ok, please correct me if I'm wrong. @dir attribute is used to specify how the
> content should be oriented.

Correct.

> Therefore we shouldn't rely on @dir attribute
> because we need to expose direction of text.

Correct, but I still don't understand why you even need to expose direction of text. I have zero experience with screen readers but I would expect that if they work with right-to-left languages at all they would work with the text in its stored logical order, just as spell checkers do.

Comment 27

8 years ago
(In reply to comment #26)

> Correct, but I still don't understand why you even need to expose direction of
> text. I have zero experience with screen readers but I would expect that if
> they work with right-to-left languages at all they would work with the text in
> its stored logical order, just as spell checkers do.

Unfortunately I don't know how spell checkers do. All screen readers can do is to get portion of text. I don't know how they can know if this text is rtl or lrt. Probably they can analyze the characters or encoding. But they might prefer if we will do this.

Comment 28

8 years ago
Pete, what is your opinion?

Comment 29

8 years ago
In most cases, screen readers don't care about ltr/rtl, assuming that the text is programmatically exposed in reading/logical order. Ltr/rtl is just how the text is visually rendered, but when it is spoken, it is still read in logical order. Normally, the caret is provided by the application, so if I understand correctly, for ltr text, right arrow will move forward in the logical order, but for rtl, left arrow moves forward in the logical order.

Unfortunately, on the web, things are not quite as simple. This is because most screen readers provide a "virtual caret" themselves. (Caret navigation in Firefox is insufficient, so we roll our own.) In this case, a problem does arise, as we need to know which way our caret should move. In our code, for example, right arrow always moves forward in the logical order, which is obviously incorrect for an rtl language. In this case, we would need some way of knowing when text is rtl so we can adjust our caret movement accordingly.

As pointed out in comment 15, the dir attribute doesn't necessarily indicate the text directionality, which would cause problems. Unfortunately, this is the best we have. Question: how does Firefox figure out what text directionality to use? Is this somehow specified by the character set?

I have no idea how we're going to handle caret movement mixed ltr/rtl text, but I think that is outside the scope of this bug.
We work out the directionality of each character by running the Unicode bidi algorithm. That's hard, and you probably don't want to deal with it.

What about caret navigation in Firefox is insufficient? Anyway, if you must have your own caret, then you need a different API from this one, and that is outside the scope of this bug.

So I think we still have no clear evidence this writing-mode attribute is useful.

Comment 31

8 years ago
(In reply to comment #30)
> What about caret navigation in Firefox is insufficient?
This is a rather lengthy discussion which probably falls outside the scope of this bug.

> Anyway, if you must
> have your own caret, then you need a different API from this one
No. Under Windows, screen readers render their own flat representation of the document into a buffer by using the a11y tree. Our virtual caret navigation is then done within this buffer. I'm not sure how Orca handles this; it does use the a11y tree, but it does not render its own flat representation.

> So I think we still have no clear evidence this writing-mode attribute is
> useful.
When we render the document into our own representation, we would access this attribute to determine which parts of the text are ltr and which are rtl. As you point out in comment 15, this could be incorrect, but it really is the best we have.

Comment 32

8 years ago
(In reply to comment #31)

> When we render the document into our own representation, we would access this
> attribute to determine which parts of the text are ltr and which are rtl. As
> you point out in comment 15, this could be incorrect, but it really is the best
> we have.

But could we expose writing-mode attribute based on Unicode bidi algorithm instead?
Yes we could. It would be per-character, though.

James: when you "render your own flat representation", is that a visual representation or a "speaking" representation? Because for the latter, you don't care about ltr/rtl.

Comment 34

8 years ago
(In reply to comment #33)
> James: when you "render your own flat representation", is that a visual
> representation or a "speaking" representation?
It's a logical representation, so I guess you'd say it's a speaking representation.

> Because for the latter, you
> don't care about ltr/rtl.
We provide a virtual caret within that representation. Therefore, when we hit rtl text, users might expect that the left and right arrows should be reversed. However, this creates a bit of a problem with mixed ltr/rtl. I have no experience with handling this. What does a word processor do with regard to the left and right arrow keys when it encounters mixed ltr/rtl? For that matter, what does Firefox caret navigation do?
In Firefox, the left arrow key always moves visually left. However, the backspace key always deletes the logically previous character.

Is this virtual caret shown on the screen at all? I guess that's irrelevant for fully blind users but perhaps you show it for partially sighted users?

I don't know how this should work either. It seems kind of weird to have a "speaking representation" where the left arrow key behaves visually. That means it changes behaviour based on visual cues, which your users probably can't even see.
I think we need to locate an RTL accessibility expert ... I'm not sure how many of those there are :-).

Comment 37

8 years ago
(In reply to comment #35)
> In Firefox, the left arrow key always moves visually left. However, the
> backspace key always deletes the logically previous character.
So if you're on some ltr text and then arrow into some rtl text, I assume hitting right arrow will just bounce you back and forth between the last ltr char and the first rtl char? :)

> Is this virtual caret shown on the screen at all?
No.

> It seems kind of weird to have a
> "speaking representation" where the left arrow key behaves visually. That means
> it changes behaviour based on visual cues, which your users probably can't even
> see.
No, but users would be used to arrowing left to move forward through rtl text, I guess.

I definitely agree we need someone who has more of a clue than I on rtl accessibility. :)
(In reply to comment #37)
> (In reply to comment #35)
> > In Firefox, the left arrow key always moves visually left. However, the
> > backspace key always deletes the logically previous character.
> So if you're on some ltr text and then arrow into some rtl text, I assume
> hitting right arrow will just bounce you back and forth between the last ltr
> char and the first rtl char? :)

No. Pressing right-arrow will move you to the logical end of the RTL text.
(In reply to comment #35)
> In Firefox, the left arrow key always moves visually left.

This is true, but not the whole story: left arrow alone always moves visually left, but shift+left arrow moves logically left, i.e. backwards in a left-to-right block and forwards in a right-to-left block. This means of course that the visual direction changes according to the text directionality. 

(In reply to comment #31)
> When we render the document into our own representation, we would access this
> attribute to determine which parts of the text are ltr and which are rtl.

Firefox caret code uses frame properties for this, and you could possibly use something similar: see the #defines of NS_GET_BASE_LEVEL and NS_GET_EMBEDDING_LEVEL in layout/generic/nsIFrame.h. "Base level" is the direction of the block, and "embedding level" is the direction of the text in the frame as resolved by the Unicode Bidi Algorithm. In both cases even levels are left-to-right and odd levels are right-to-left.
To be clear, that information is *per text frame*, not *per text node*, because you can have many runs of text in different directions in a single text node.
(In reply to comment #39)
> left arrow alone always moves visually
> left, but shift+left arrow moves logically left, i.e. backwards in a
> left-to-right block and forwards in a right-to-left block.

I should have mentioned that this is controlled by a pref, bidi.edit.caret_movement_style. The default value is 2, which gives the behaviour I described, but it can also be set to 1, giving always visual caret movement; or 0, giving always logical caret movement.
Comment on attachment 381728 [details] [diff] [review]
patch2

minusing because I still don't understand how this property would be used by screenreaders
Attachment #381728 - Flags: superreview?(roc) → superreview-

Updated

5 years ago
Assignee: surkov.alexander → nobody
You need to log in before you can comment on or make changes to this bug.