Closed Bug 1585622 Opened 4 years ago Closed 3 months ago

Add support for getComputedLabel and getComputedRole commands to WebDriver

Categories

(Remote Protocol :: Marionette, enhancement, P1)

Product:
Remote Protocol
Component:
Marionette
Version:
Version 3
Type:
enhancement
Points:
2

Tracking

(firefox113 fixed)

Status:
RESOLVED FIXED
Milestone:
113 Branch
Tracking Flags:
Tracking Status
firefox113 --- fixed

People

(Reporter: automatedtester, Assigned: jdescottes)

References

(Blocks 1 open bug,
URL
)

Details

(Whiteboard: [webdriver:m6] [webdriver:relnote])

Attachments

(2 files, 2 obsolete files)

Bug 1585622: Add support for getComputedLabel and getComputedRole commands to WebDriver
47 bytes, text/x-phabricator-request
Details | Review
Bug 1585622 - [marionette] Add support for getComputedRole and getComputedLabel
48 bytes, text/x-phabricator-request
Details | Review

at TPAC 2019, commands for helping with accessibility were requested.

see https://github.com/w3c/webdriver/issues/1439

Initial prose in https://github.com/w3c/webdriver/pull/1444

Priority: -- → P3
Assignee: nobody → dburns

What is the best way to get a label and a role for an DOMElement from the accessibility tree. I can see that I can get the Role from nsIAccessibilityService[1] but I can't see a way of getting the label. Could you point me in the right direction here? Thanks!

[1] https://searchfox.org/mozilla-central/source/accessible/interfaces/nsIAccessibilityService.idl

Flags: needinfo?(yzenevich)

You should do it like our test frameworks do, e. g. from the service, only use GetAccessibleFor, then use that instance variable's properties such as .name, .role, .description etc., to get the actual properties from the accessibility engine. See nsIAccessible.idl for methods you can use and types they return. Role, for example, is a standardized number, and the string roles are not being used much and should be avoided for the most part. Note that this needs to be privileged access to even get these accessible objects, but I assume webDriver is allowed that?

(In reply to Marco Zehe (:MarcoZ) from comment #2)

You should do it like our test frameworks do, e. g. from the service, only use GetAccessibleFor, then use that instance variable's properties such as .name, .role, .description etc., to get the actual properties from the accessibility engine.
See nsIAccessible.idl for methods you can use and types they return. Role, for example, is a standardized number, and the string roles are not being used much and should be avoided for the most part. Note that this needs to be privileged access to even get these accessible objects, but I assume webDriver is allowed that?

Yes, we have priviledged access. This is being added to marionette which has access to XPCOM APIs.

Looking at https://github.com/w3c/webdriver/issues/1439#issuecomment-533381779 I can't seem to see how I would get the label from the accessibility system. Sorry if I am overlooking it.

Flags: needinfo?(mzehe)

The label is equivalent to the name. So when you ask for the name of an accessible, you get the string the engine computed from the markup. This could be the text contents of a label element which is associated to an input via the for attribute, the inner text of a button element, the alt attribute's content of an img element, or the aria-label for any element that has that attribute. Label and Name are equivalent here. The role is what the accessibility engine made from the markup. It could be a native host language role such as entry for an input of type "text", checkbox for an input of type "checkbox", a paragraph accessible from a p element, etc.

Flags: needinfo?(mzehe)

One caveat, if you are using XPCOM accessibility API, you would want to shut down the accessibility service, otherwise, unless Firefox is restarted, it will not be shutdown (unless things get garbage collected).

Flags: needinfo?(yzenevich)
Assignee: dburns → nobody
Assignee: nobody → dburns
Status: NEW → ASSIGNED
Assignee: dburns → nobody
Status: ASSIGNED → NEW

David, are you planning to finish the work on this patch or shall we try to find someone else? Please let us know. Thanks.

Flags: needinfo?(david.burns)
No longer blocks: 1788867

I am not planning on finishing this work.

Flags: needinfo?(david.burns)
Severity: normal → S3
Product: Testing → Remote Protocol

FYI, the ARIA Working Group is planning to start leveraging these extensively for ARIA testing, as they are supported in both WebKit and Chromium. It will also likely be a core part of the Accessibility Investigation for InterOp 2023.

Whiteboard: [webdriver:triage]

We can work on this during M7 this year which is from April - June.

Priority: P3 → P2
Whiteboard: [webdriver:triage] → [webdriver:backlog]
Assignee: nobody → jdescottes
Status: NEW → ASSIGNED
Points: --- → 3
Priority: P2 → P1
Whiteboard: [webdriver:backlog] → [webdriver:m6]
Blocks: 1820782

(In reply to Simon Pieters [:zcorpan] from comment #13)

The tests in comment 12 are no longer part of scoring the interop 2023 accessibility testing investigation area, see https://github.com/web-platform-tests/interop-2023-accessibility-testing/issues/3 (It would be nice to pass them, but they're not blocking.)

That is totally fine given that those are basically unit tests for the different WebDriver implementations to actually make sure the implementation is correct.

A base getAccessible helper is extracted from the accessibility.Checks module to avoid issues related to the "strict" flag, which do not make sense here.

Comment on attachment 9322183 [details]
Bug 1585622 - [marionette-client] Sort marionette-client unit-tests.ini alphabetically

Revision D172156 was moved to bug 1818173. Setting attachment 9322183 [details] to obsolete.

Attachment #9322183 - Attachment is obsolete: true
Blocks: 1821677

Reducing to 2 points since the marionette client changes were split to another bug.

Points: 3 → 2

Comment on attachment 9322177 [details]
Bug 1585622 - [marionette-client] Add marionette-client tests for getComputedLabel and getComputedRole

Revision D172155 was moved to bug 1821677. Setting attachment 9322177 [details] to obsolete.

Attachment #9322177 - Attachment is obsolete: true
Blocks: 1822112
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c8f206c77ab7
[marionette] Add support for getComputedRole and getComputedLabel r=webdriver-reviewers,whimboo

https://hg.mozilla.org/mozilla-central/rev/c8f206c77ab7

Status: ASSIGNED → RESOLVED
Closed: 3 months ago
status-firefox113: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 113 Branch
Whiteboard: [webdriver:m6] → [webdriver:m6] [webdriver:relnote]
You need to log in before you can comment on or make changes to this bug.