Last Comment Bug 454805 - Support writing-mode (lr/rl) text attribute
: Support writing-mode (lr/rl) text attribute
Status: ASSIGNED
: access
Product: Core
Classification: Components
Component: Disability Access APIs (show other bugs)
: unspecified
: x86 Windows XP
: -- normal with 1 vote (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
: alexander :surkov
Mentors:
Depends on:
Blocks: textattra11y
  Show dependency treegraph
 
Reported: 2008-09-11 07:40 PDT by Aaron Leventhal
Modified: 2014-11-05 00:49 PST (History)
10 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
patch (14.73 KB, patch)
2009-06-05 00:14 PDT, alexander :surkov
no flags Details | Diff | Splinter Review
patch2 (14.73 KB, patch)
2009-06-05 00:19 PDT, alexander :surkov
mzehe: review+
dbolter: review+
roc: superreview-
Details | Diff | Splinter Review

Description Aaron Leventhal 2008-09-11 07:40:28 PDT
We need to expose the current text direction through this attribute:
http://wiki.mozilla.org/Accessibility/TextAttributes

And add to docs when done:
http://developer.mozilla.org/en/Accessibility/AT-APIs#Supported_Text_Attributes
Comment 1 alexander :surkov 2009-03-04 05:27:12 PST
I'm not sure it's worth to expose these until our text interface work right with "rl". In the meantime all is "lr" - we navigate DOM and accessibles trees from left to right always.
Comment 2 alexander :surkov 2009-06-04 23:03:23 PDT
(In reply to comment #1)
> I'm not sure it's worth to expose these until our text interface work right
> with "rl". In the meantime all is "lr" - we navigate DOM and accessibles trees
> from left to right always.

Actually it's ok, we should navigate trees from left to right until a11y tree is built based on frame tree I think. Generally we create a11y tree based on frame tree for few cases only (for native anonymous content in HTML). We could handle them separately if they'll bring any problems.
Comment 3 alexander :surkov 2009-06-05 00:14:54 PDT
Created attachment 381725 [details] [diff] [review]
patch
Comment 4 alexander :surkov 2009-06-05 00:19:07 PDT
Created attachment 381728 [details] [diff] [review]
patch2
Comment 5 David Bolter [:davidb] 2009-06-05 05:54:45 PDT
Comment on attachment 381728 [details] [diff] [review]
patch2

I'm still not sure we need to be creating new classes for each text attribute (thinking performance), but this patch fits the current style.

>+
>+////////////////////////////////////////////////////////////////////////////////
>+// nsDirTextAttr
>+
>+nsDirTextAttr::nsDirTextAttr(nsIFrame *aRootFrame, nsIFrame *aFrame) :
>+  nsTextAttr<PRUint32>(aFrame == nsnull), mRootFrame(aRootFrame)
>+{
>+  mIsRootDefined = GetDirection(mRootFrame, &mRootNativeValue);

You probably meant to use the passed in root frame right?


>+void
>+nsDirTextAttr::Format(const PRUint32& aValue, nsAString& aFormattedValue)
>+{
>+  // LTR and RTL directions are supported in mozilla only.

This isn't true out of context, but I'm not sure exactly what you want to say here.
(see http://www.quirksmode.org/dom/w3c_html.html for dir support)
Comment 6 David Bolter [:davidb] 2009-06-05 06:02:29 PDT
Oops just saw you used the passed in root frame to initialize mRootFrame... please disregard.
Comment 7 alexander :surkov 2009-06-05 07:09:51 PDT
(In reply to comment #5)

> >+void
> >+nsDirTextAttr::Format(const PRUint32& aValue, nsAString& aFormattedValue)
> >+{
> >+  // LTR and RTL directions are supported in mozilla only.
> 
> This isn't true out of context, but I'm not sure exactly what you want to say
> here.
> (see http://www.quirksmode.org/dom/w3c_html.html for dir support)

I didn't realized @dir doesn't support "top to bottom" direction. I just oriented to constants http://mxr.mozilla.org/mozilla-central/source/layout/base/nsStyleConsts.h#365
Comment 8 alexander :surkov 2009-06-05 07:13:22 PDT
(In reply to comment #5)
> (From update of attachment 381728 [details] [diff] [review])
> I'm still not sure we need to be creating new classes for each text attribute
> (thinking performance), but this patch fits the current style.

Yes, it is. Would be nice to know some details how are class approach less performant comparing to non classes approach. What do you keep in mind?
Comment 9 David Bolter [:davidb] 2009-06-05 16:45:50 PDT
Comment on attachment 381728 [details] [diff] [review]
patch2

r=me

Let's make perf testing a goal for Q2 or Q3. We can stress test for text attributes and try using static util methods to compare with the class creation.
Comment 10 David Bolter [:davidb] 2009-06-05 16:46:22 PDT
erm, Q3, or Q4 ;)
Comment 11 Marco Zehe (:MarcoZ) 2009-06-06 00:04:59 PDT
Comment on attachment 381728 [details] [diff] [review]
patch2

r=me, but I'd also like to see a test with a native right-to-left language such as Arabic or Hebrew.
Comment 12 alexander :surkov 2009-06-06 21:41:43 PDT
(In reply to comment #11)
> (From update of attachment 381728 [details] [diff] [review])
> r=me, but I'd also like to see a test with a native right-to-left language such
> as Arabic or Hebrew.

Probably it would complete the test from some point of view but it's not very helpful for us I think.
Comment 13 Marco Zehe (:MarcoZ) 2009-06-06 21:58:45 PDT
Alex, why do you think this test would not be helpful? Because if the right-to-left rendering of such languages is broken, other tests inside content or layout would already catch that?
Comment 14 alexander :surkov 2009-06-06 22:19:42 PDT
(In reply to comment #13)
> Alex, why do you think this test would not be helpful? Because if the
> right-to-left rendering of such languages is broken, other tests inside content
> or layout would already catch that?

Marco, my point the Arabic or any other rl language text written from right to left looks the same as left to right text from point of view who doesn't know these languages :) I think neither of us can read it. Our test shows that DOM is rendered from right to left and we expose proper writing-mode text attribute. So I think if screen reader supports right to left mode then it should announce "text" as "txet" if author pointed dir="rlt" for example.
Comment 15 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-07 15:20:21 PDT
+  // LTR and RTL directions are supported in mozilla only.

I think you mean "only LTR and RTL directions are supported in Mozilla".

> So I think if screen reader supports right to left mode then it
> should announce "text" as "txet" if author pointed dir="rlt" for example.

No, that would make no sense. You would always read text in the logical order, the order of the characters in the DOM.

I'm not sure how screen readers should use this attribute. One thing that makes it difficult is that you're only reporting style information here, but the text itself has its own directionality. E.g. you might have direction:ltr, but if the text is all Arabic it will be rendered right-to-left anyway.
Comment 16 alexander :surkov 2009-06-15 02:08:41 PDT
(In reply to comment #15)
> +  // LTR and RTL directions are supported in mozilla only.
> 
> I think you mean "only LTR and RTL directions are supported in Mozilla".

ok, thanks.

> > So I think if screen reader supports right to left mode then it
> > should announce "text" as "txet" if author pointed dir="rlt" for example.
> 
> No, that would make no sense. You would always read text in the logical order,
> the order of the characters in the DOM.
> 
> I'm not sure how screen readers should use this attribute. One thing that makes
> it difficult is that you're only reporting style information here, but the text
> itself has its own directionality. E.g. you might have direction:ltr, but if
> the text is all Arabic it will be rendered right-to-left anyway.

I think I don't mind. If Arabic text is placed from right to left in DOM already then it's great and nothing to do for us.
Comment 17 alexander :surkov 2009-06-17 20:57:35 PDT
Robert, any other comments?
Comment 18 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-17 21:08:07 PDT
I don't understand what screen readers are expected to do with this attribute. If there's no way to use it reliably, we shouldn't provide it.
Comment 19 alexander :surkov 2009-06-17 21:52:46 PDT
(In reply to comment #18)
> I don't understand what screen readers are expected to do with this attribute.
> If there's no way to use it reliably, we shouldn't provide it.

I think this attribute is a hint for screen readers. I would assume if screen readers get the text
<span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it should read "normal text" from left to right, "arabic text" from right to left since DOM order doesn't reflect text direction. I'm not sure about the case when arabic text is used but direction is ltr. We could try to manipulate with encoding but I'm not sure.

Any way I don't know if this text attribute is implemented by screen readers yet. So I think we need to expose something reasonable before we'll get some feedback.

> No, that would make no sense. You would always read text in the logical order,
> the order of the characters in the DOM.

I'm not sure I get you. I think when they write something then their caret position in the right of the screen and text is moved to left when they type symbols. So if would write "bla" in arabic editor then I should get "alb" if I would read from left to right. Therefore "alb" should be stored in DOM as well. If author put dir="rtl" on "alb" then we expose "alb" to screen reader with "rtl" text attribute and screen reader should read "bla" (from right to left). Do I understand something wrong?
Comment 20 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-17 21:59:49 PDT
(In reply to comment #19)
> I think this attribute is a hint for screen readers. I would assume if screen
> readers get the text
> <span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it
> should read "normal text" from left to right, "arabic text" from right to left
> since DOM order doesn't reflect text direction. I'm not sure about the case
> when arabic text is used but direction is ltr. We could try to manipulate with
> encoding but I'm not sure.

Normally you would use 'direction' on a block. I don't think <div style="direction:rtl"> should affect the reading of the text at all.

> > No, that would make no sense. You would always read text in the logical
> > order, the order of the characters in the DOM.
> 
> I'm not sure I get you. I think when they write something then their caret
> position in the right of the screen and text is moved to left when they type
> symbols. So if would write "bla" in arabic editor then I should get "alb" if I
> would read from left to right. Therefore "alb" should be stored in DOM as
> well.

No. If you type H E B then it would be displayed as "BEH" but the DOM would contain HEB. Normally the DOM contains characters in the order you type them and read them.
Comment 21 alexander :surkov 2009-06-17 23:50:17 PDT
(In reply to comment #20)
> (In reply to comment #19)
> > I think this attribute is a hint for screen readers. I would assume if screen
> > readers get the text
> > <span dir="ltr">normal text</span><span dir="rtl">arabic text</span> then it
> > should read "normal text" from left to right, "arabic text" from right to left
> > since DOM order doesn't reflect text direction. I'm not sure about the case
> > when arabic text is used but direction is ltr. We could try to manipulate with
> > encoding but I'm not sure.
> 
> Normally you would use 'direction' on a block. I don't think <div
> style="direction:rtl"> should affect the reading of the text at all.

I just keep in mind language learning course or dictionary, for example 
<ul>
  <li>
    <span>cat</span> - <span dir="rtl">cat in arabic</span>
  </li>
</ul>

> 
> > > No, that would make no sense. You would always read text in the logical
> > > order, the order of the characters in the DOM.
> > 
> > I'm not sure I get you. I think when they write something then their caret
> > position in the right of the screen and text is moved to left when they type
> > symbols. So if would write "bla" in arabic editor then I should get "alb" if I
> > would read from left to right. Therefore "alb" should be stored in DOM as
> > well.
> 
> No. If you type H E B then it would be displayed as "BEH" but the DOM would
> contain HEB. Normally the DOM contains characters in the order you type them
> and read them.

If I have <p dir="rtl">HEB</p> then I see "HEB" in the right of screen when I read from left to right.
Comment 22 Simon Montagu :smontagu 2009-06-18 00:07:53 PDT
(In reply to comment #21)
> (In reply to comment #20)
> > No. If you type H E B then it would be displayed as "BEH" but the DOM would
> > contain HEB. Normally the DOM contains characters in the order you type them
> > and read them.
> 
> If I have <p dir="rtl">HEB</p> then I see "HEB" in the right of screen when I
> read from left to right.

The two of you are at cross-purposes. Roc means that if you type HEB in a right-to-left language, it is displayed BEH. For example, if I type ש, then ל, then ו, and then ם, it appears in right-to-left order as שלום, even in this left-to-right paragraph.

This does not mean that it should be stored in reverse order in DOM - that would break all sorts of things, charAt for example.
Comment 23 alexander :surkov 2009-06-18 00:48:52 PDT
Ok, if I have

<p dir="rtl"> שלום - <span dir="ltr">example of right to left word</p>

then I should read שלום  (from right to left) and "example of right to left word" from left to right. Do I understand correct? If so then screen reader should do the same. But actually I think it's screen reader task to decide how to read this. We just should say where ltr word and where rtl word.
Comment 24 Simon Montagu :smontagu 2009-06-18 01:09:17 PDT
The point is that the what determines the reading order is not the dir attribute but the characters themselves. Even if you have

<p dir="ltr"> שלום - <span dir="rtl">example of right to left word</p>

you should also read שלום from right to left and "example of right to left word" from left to right.
Comment 25 alexander :surkov 2009-06-18 01:14:02 PDT
Ok, please correct me if I'm wrong. @dir attribute is used to specify how the content should be oriented. Therefore we shouldn't rely on @dir attribute because we need to expose direction of text. So how can we know this word is rtl but this one is ltr?
Comment 26 Simon Montagu :smontagu 2009-06-18 01:42:45 PDT
(In reply to comment #25)
> Ok, please correct me if I'm wrong. @dir attribute is used to specify how the
> content should be oriented.

Correct.

> Therefore we shouldn't rely on @dir attribute
> because we need to expose direction of text.

Correct, but I still don't understand why you even need to expose direction of text. I have zero experience with screen readers but I would expect that if they work with right-to-left languages at all they would work with the text in its stored logical order, just as spell checkers do.
Comment 27 alexander :surkov 2009-06-24 00:46:37 PDT
(In reply to comment #26)

> Correct, but I still don't understand why you even need to expose direction of
> text. I have zero experience with screen readers but I would expect that if
> they work with right-to-left languages at all they would work with the text in
> its stored logical order, just as spell checkers do.

Unfortunately I don't know how spell checkers do. All screen readers can do is to get portion of text. I don't know how they can know if this text is rtl or lrt. Probably they can analyze the characters or encoding. But they might prefer if we will do this.
Comment 28 alexander :surkov 2009-06-24 00:50:38 PDT
Pete, what is your opinion?
Comment 29 James Teh [:Jamie] 2009-06-24 16:29:40 PDT
In most cases, screen readers don't care about ltr/rtl, assuming that the text is programmatically exposed in reading/logical order. Ltr/rtl is just how the text is visually rendered, but when it is spoken, it is still read in logical order. Normally, the caret is provided by the application, so if I understand correctly, for ltr text, right arrow will move forward in the logical order, but for rtl, left arrow moves forward in the logical order.

Unfortunately, on the web, things are not quite as simple. This is because most screen readers provide a "virtual caret" themselves. (Caret navigation in Firefox is insufficient, so we roll our own.) In this case, a problem does arise, as we need to know which way our caret should move. In our code, for example, right arrow always moves forward in the logical order, which is obviously incorrect for an rtl language. In this case, we would need some way of knowing when text is rtl so we can adjust our caret movement accordingly.

As pointed out in comment 15, the dir attribute doesn't necessarily indicate the text directionality, which would cause problems. Unfortunately, this is the best we have. Question: how does Firefox figure out what text directionality to use? Is this somehow specified by the character set?

I have no idea how we're going to handle caret movement mixed ltr/rtl text, but I think that is outside the scope of this bug.
Comment 30 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-24 17:23:19 PDT
We work out the directionality of each character by running the Unicode bidi algorithm. That's hard, and you probably don't want to deal with it.

What about caret navigation in Firefox is insufficient? Anyway, if you must have your own caret, then you need a different API from this one, and that is outside the scope of this bug.

So I think we still have no clear evidence this writing-mode attribute is useful.
Comment 31 James Teh [:Jamie] 2009-06-24 17:31:01 PDT
(In reply to comment #30)
> What about caret navigation in Firefox is insufficient?
This is a rather lengthy discussion which probably falls outside the scope of this bug.

> Anyway, if you must
> have your own caret, then you need a different API from this one
No. Under Windows, screen readers render their own flat representation of the document into a buffer by using the a11y tree. Our virtual caret navigation is then done within this buffer. I'm not sure how Orca handles this; it does use the a11y tree, but it does not render its own flat representation.

> So I think we still have no clear evidence this writing-mode attribute is
> useful.
When we render the document into our own representation, we would access this attribute to determine which parts of the text are ltr and which are rtl. As you point out in comment 15, this could be incorrect, but it really is the best we have.
Comment 32 alexander :surkov 2009-06-24 17:56:28 PDT
(In reply to comment #31)

> When we render the document into our own representation, we would access this
> attribute to determine which parts of the text are ltr and which are rtl. As
> you point out in comment 15, this could be incorrect, but it really is the best
> we have.

But could we expose writing-mode attribute based on Unicode bidi algorithm instead?
Comment 33 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-24 20:36:37 PDT
Yes we could. It would be per-character, though.

James: when you "render your own flat representation", is that a visual representation or a "speaking" representation? Because for the latter, you don't care about ltr/rtl.
Comment 34 James Teh [:Jamie] 2009-06-24 21:04:44 PDT
(In reply to comment #33)
> James: when you "render your own flat representation", is that a visual
> representation or a "speaking" representation?
It's a logical representation, so I guess you'd say it's a speaking representation.

> Because for the latter, you
> don't care about ltr/rtl.
We provide a virtual caret within that representation. Therefore, when we hit rtl text, users might expect that the left and right arrows should be reversed. However, this creates a bit of a problem with mixed ltr/rtl. I have no experience with handling this. What does a word processor do with regard to the left and right arrow keys when it encounters mixed ltr/rtl? For that matter, what does Firefox caret navigation do?
Comment 35 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-24 21:18:07 PDT
In Firefox, the left arrow key always moves visually left. However, the backspace key always deletes the logically previous character.

Is this virtual caret shown on the screen at all? I guess that's irrelevant for fully blind users but perhaps you show it for partially sighted users?

I don't know how this should work either. It seems kind of weird to have a "speaking representation" where the left arrow key behaves visually. That means it changes behaviour based on visual cues, which your users probably can't even see.
Comment 36 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-24 21:19:07 PDT
I think we need to locate an RTL accessibility expert ... I'm not sure how many of those there are :-).
Comment 37 James Teh [:Jamie] 2009-06-24 21:24:14 PDT
(In reply to comment #35)
> In Firefox, the left arrow key always moves visually left. However, the
> backspace key always deletes the logically previous character.
So if you're on some ltr text and then arrow into some rtl text, I assume hitting right arrow will just bounce you back and forth between the last ltr char and the first rtl char? :)

> Is this virtual caret shown on the screen at all?
No.

> It seems kind of weird to have a
> "speaking representation" where the left arrow key behaves visually. That means
> it changes behaviour based on visual cues, which your users probably can't even
> see.
No, but users would be used to arrowing left to move forward through rtl text, I guess.

I definitely agree we need someone who has more of a clue than I on rtl accessibility. :)
Comment 38 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-24 21:53:40 PDT
(In reply to comment #37)
> (In reply to comment #35)
> > In Firefox, the left arrow key always moves visually left. However, the
> > backspace key always deletes the logically previous character.
> So if you're on some ltr text and then arrow into some rtl text, I assume
> hitting right arrow will just bounce you back and forth between the last ltr
> char and the first rtl char? :)

No. Pressing right-arrow will move you to the logical end of the RTL text.
Comment 39 Simon Montagu :smontagu 2009-06-25 00:38:07 PDT
(In reply to comment #35)
> In Firefox, the left arrow key always moves visually left.

This is true, but not the whole story: left arrow alone always moves visually left, but shift+left arrow moves logically left, i.e. backwards in a left-to-right block and forwards in a right-to-left block. This means of course that the visual direction changes according to the text directionality. 

(In reply to comment #31)
> When we render the document into our own representation, we would access this
> attribute to determine which parts of the text are ltr and which are rtl.

Firefox caret code uses frame properties for this, and you could possibly use something similar: see the #defines of NS_GET_BASE_LEVEL and NS_GET_EMBEDDING_LEVEL in layout/generic/nsIFrame.h. "Base level" is the direction of the block, and "embedding level" is the direction of the text in the frame as resolved by the Unicode Bidi Algorithm. In both cases even levels are left-to-right and odd levels are right-to-left.
Comment 40 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-06-25 00:47:23 PDT
To be clear, that information is *per text frame*, not *per text node*, because you can have many runs of text in different directions in a single text node.
Comment 41 Simon Montagu :smontagu 2009-06-25 02:15:18 PDT
(In reply to comment #39)
> left arrow alone always moves visually
> left, but shift+left arrow moves logically left, i.e. backwards in a
> left-to-right block and forwards in a right-to-left block.

I should have mentioned that this is controlled by a pref, bidi.edit.caret_movement_style. The default value is 2, which gives the behaviour I described, but it can also be set to 1, giving always visual caret movement; or 0, giving always logical caret movement.
Comment 42 Robert O'Callahan (:roc) (email my personal email if necessary) 2010-03-01 16:54:58 PST
Comment on attachment 381728 [details] [diff] [review]
patch2

minusing because I still don't understand how this property would be used by screenreaders

Note You need to log in before you can comment on or make changes to this bug.