287737 - Expose language changes in document

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Aaron Leventhal]

The only way we currently expose language changes via our accessibility APIs is
via ISimpleDOMNode, which is a Windows only API. It is a bad API to use because
the AT would have to check it for every node in the document.

I'm not sure how ATK exposes language changes. Bill - do you know?

Comment 1

[Bill Haneman]

ATK exposes language changes as TextAttributes - for instance, see 
ATK_TEXT_ATTR_LANGUAGE, atk_text_get_default_attributes(),
atk_text_get_run_attributes (), and the text-attributes-changed signal.

Comment 2

[Aaron Leventhal]

(In reply to comment #1)
> ATK exposes language changes as TextAttributes - for instance, see 
> ATK_TEXT_ATTR_LANGUAGE, atk_text_get_default_attributes(),
> atk_text_get_run_attributes (), and the text-attributes-changed signal.

That works for me. Bill, forgive the naive ATK questions :/
One more - how does ATK expose what language an image's alternative text is in?
Actually now that I look at the AtkImage interface, I'm not clear if/how it
exposes its alternative text. In Firefox I don't see that we support AtkImage at
all.

Comment 3

[Bill Haneman]

Hi Aaron:

As for AtkImage, bear in mind that these are queryable optional interfaces... so
if an object implements AtkImage, then it by definition also implements
AtkObject, and thus allows atk_object_get_name, atk_object_get_description, etc.
 The results of these interfaces are assumed to be localized in the target
locale of the application, i.e. they are usually passed through gettext().
 
This means that in a document, the AtkImage implementing object is a child of
the document, i.e. usually a sibling of various block text objects.  Captions
for text would use ATK_RELATION_LABEL_FOR/ATK_RELATION_LABELLED_BY (but alt text
would not, it would just use atk_object_get_name(imageObject);

If you have a more complex document, in which for instance an image is imbedded
"inside" a text flow region, you can use the
ATK_RELATION_EMBEDS/ATK_RELATION_EMBEDDED_BY relations to indicate this (same
goes for tables, etc. embedded inside complex documents).

Comment 4

[Aaron Leventhal]

This has been fixed on the Windows side, via ISimpleDOMNode::get_language

Comment 5

[Bill Haneman]

(In reply to comment #0)
> The only way we currently expose language changes via our accessibility APIs is
> via ISimpleDOMNode, which is a Windows only API. It is a bad API to use because
> the AT would have to check it for every node in the document.
> 
> I'm not sure how ATK exposes language changes. Bill - do you know?

It does so via AtkText attributes, which tag 'attribute spans' within the text
content, and also we are currently reviewing a proposal which would add locale
info to AtkImage.

In your Windows solution, I take it that foreign-language text within a text,
object, <locale lang=fr>comme çi</locale>, would require that the out-of-locale
text have its own node.  That seems potentially wrong to me, but maybe there's
no other way to express this in HTML.

Bill

Comment 6

[Aaron Leventhal]

(In reply to comment #5)
> In your Windows solution, I take it that foreign-language text within a text,
> object, <locale lang=fr>comme çi</locale>, would require that the out-of-locale
> text have its own node.  That seems potentially wrong to me, but maybe there's
> no other way to express this in HTML.

There is no way to change any text attributes in HTML without using a separate
DOM node.

Comment 7

[Aaron Leventhal]

We'll use object/text attributes for this.

Comment 8

[Mike Gao]

Aaron, We can use GetAttributes() to set "lang" attribute according to the "lang" attribute of the element, right?

Comment 9

[Aaron Leventhal]

(In reply to comment #8)
> Aaron, We can use GetAttributes() to set "lang" attribute according to the
> "lang" attribute of the element, right?

It's a little bit more complicated than that because the HTTP header can set the language.

Look at the code here:
http://lxr.mozilla.org/seamonkey/source/accessible/src/msaa/nsAccessNodeWrap.cpp#483

Some of that should be refactored so that the nsAccessNodeWrap gets that info from something cross-platform.

Comment 10

[Mike Gao]

Attachment: patch draft v１

Draft. Has not compile on windows yet. Compiled on Linux and "lang" can be exposed when "Content-Language" for page or "lang" attribute for parent DOM node is set. But it's only expose "parent"'s lang, is that intended to? Or we should change walkUp = walkUp->GetParent()?

Comment 11

[Håkan Waara]

Comment on attachment 238197 [details] [diff] [review]
patch draft v１

>+NS_IMETHODIMP
>+nsAccessNode::GetLanguage(nsAutoString *aLanguage)
>+{

Use nsAString &aLanguage for the method instead. This maps to the AString type in the IDL.

Then you can simply pass in your nsAutoString "as it is". nsAutoString is stack-based, and will automatically be freed when it goes out of scope. It shouldn't be in the method declaration though, where it is preferred to use the abstract "mother" string type of nsAString.

Comment 12

[Mike Gao]

Attachment: patch draft v2

- Addressing Håkan's comments. 
- Move code getting lang attribute from nsHyperTextAccessible to nsAccessible.
- If the lang attribute is empty, it will not be exposed.

Compile passed on both Linux and Windows.

Comment 13

[Aaron Leventhal]

In accessible/src/atk/nsMaiInterfaceDocument.cpp you need to support atk_document_get_locale. It can use GetLanguage() from the doc accessible to expose that.

Instead of declaring NS_IMETHOD GetLanguage() in nsAccessNode.h, what you need to do is add it to nsIAccessNode.idl as readonly attribute DOMString language; That will automatically add it to nsAccessNode.h via NS_DECL_NSIACCESSNODE. Then you'll get nsAString& instead of nsAString* which is how we usually pass out string parameters. You won't need your null pointer checks on language anymore.

> If the lang attribute is empty, it will not be exposed.
Not really true, because the parent could have a lang attribute.
What we need to do is only expose the lang attribute in GetAttributes() when it's different from the parent.
So, GetParent(getter_AddRefs(parentAccessible)) and then check GetLanguage() for it. If it's the same, don't expose the attribute via GetAttributes()

Comment 14

[Aaron Leventhal]

Let's call the attribute "locale" so that we're consistent with the naming used by ATK/AT-SPI's document interface.

When this gets checked in we need to change 
http://developer.mozilla.org/en/docs/Accessibility/ATSPI_Support#Supported_AT-SPI_Object_Attributes
and the text attributes section below that

Comment 15

[Bill Haneman]

Aaron, all:

Note that ATK expects several different ways that language can change within a document.  "locale" is used in ATK only to refer to the locale of an entire document (http://developer.gnome.org/doc/API/2.0/atk/AtkDocument.html#atk-document-get-locale), or an image (http://developer.gnome.org/doc/API/2.0/atk/AtkImage.html#atk-image-get-image-locale).  Within a text object, language changes are delineated by AtkTextAttributes.
(http://developer.gnome.org/doc/API/2.0/atk/AtkText.html#atk-text-get-run-attributes)  

I realize that in mozilla, if a piece of text has a different lang from its surroundings, it's probably in its own DOM node.  It may be that these nodes get flattened somewhat before being presented via ATK (for instance, within a single <p> element).  In such a case, the attribute exposed via ATK is (IIRC) "lang".  Note also that the enum which AtkAttributeSet uses for language attribute is
ATK_TEXT_ATTR_LANGUAGE (http://developer.gnome.org/doc/API/2.0/atk/AtkText.html#AtkTextAttribute)

Lastly, while one could make ATK_TEXT_ATTR_LANGUAGE one of the default attributes of an AtkText node (see atk_text_get_default_attributes), 
in the case where the language of a node doesn't match that of its host document or surrounding text, I would recommend exposing it as an explicit text attribute instead, i.e. via atk_text_get_attribute_run, even though it might be invariant over that AtkText object.  This is to prevent assistive techs from having to call atk_text_get_default_attributes() just to check the language attribute.

regards,

Bill

Comment 16

[Aaron Leventhal]

Mike, since this is different than what we planned and I have questions about it, I'm going to resolve my issues with it offline with Bill before asking you to go further with this bug.

Until then you can also look at bug 352150.

Comment 17

[Aaron Leventhal]

Taking -- I want to be careful with this one. Mike, please work on bug 352522 next.

Comment 18

[Nian Liu(n/a in a long time)]

image locale should also be fixed(bug 357430)

Comment 19

[Aaron Leventhal]

So, we'll do it as Bill suggests
* Use the "lang" text attribute on a run
* We still need to support locale on AtkImage. 
* If an image or any other embedded object in text is in a different locale then the text that it's in we also use the lang attribute for a text run of a single character (it's embedded object character can have a text attribute run)

> This is to prevent assistive techs from having to call 
> atk_text_get_default_attributes() just to check the language attribute.
When should default text attributes be used?

Comment 20

[alexander :surkov (:asurkov)]

What's a difference between GetLanguage and language text attribute? If language is changed over hypertext (for example, "Guten Tag, Привет, Hello") then we should reflect this, right? Or language means something different?

Comment 21

[Aaron Leventhal]

Right, since it can change over a hyper text we need to use text attributes to expose it.

Comment 22

[alexander :surkov (:asurkov)]

(In reply to comment #21)
> Right, since it can change over a hyper text we need to use text attributes to
> expose it.
> 

Any idea how to get this info? :)

Comment 23

[Aaron Leventhal]

From the current element (or parent if it's a text node). Same way GetLanguage() does.

Comment 24

[alexander :surkov (:asurkov)]

fixed per bug 345759

Comment 25

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1527381