465755 - Fix failing tests in a11y suite and re-enable in production.

Status: RESOLVED WORKSFORME

(Testing :: Mochitest, defect)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

While automating the a11y suite, we noticed some tests which *used* to pass, now start failing. Also, the "test_testboxes" suite was hanging. See details in https://bugzilla.mozilla.org/show_bug.cgi?id=424195#c60. There is some concern that these started failing around the same time as tracemonkey was turned on by default in mozilla-central, but this needs investigation.


Marco has commented out the test_textboxes suite for now, so at least the other a11y tests can pass.This bug is to track work on debugging and re-enabling these failing test_textboxes tests.

Comment 1

[Marco Zehe (:MarcoZ)]

Lukas, perhaps you could add the logs of these specific failures to this bug so any TM or other dev doesn't have to dig through that other bug to find the relevant info?

Comment 2

[Lukas Blakk [:lsblakk] use ?needinfo]

So for linux there are 6 consistently failing tests:

*** 755 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessible_selects.html | wrong name for combo2 at index 0! - got "Search in: Newsticker", expected "Search in: Newsticker Entire site"

*** 759 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessible_selects.html | wrong name for combo2 at index 2! - got "Search in: Newsticker", expected "Search in: Newsticker Entire site"

*** 763 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessible_selects.html | wrong name for combo2 at index 3! - got "Search in:", expected "Search in: Newsticker Entire site"

*** 793 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessible_selects.html | wrong name for list2 at index 0! - got "Version:", expected "Version: 2.0 3.0 3.1 trunk"

*** 797 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_nsIAccessible_selects.html | wrong name for list2 at index 2! - got "Version:", expected "Version: 2.0 3.0 3.1 trunk"

*** 1009 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_textattrs.html | Wrong count of 'text attribute changed' events for area8 - got 0, expected 2

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

The windows tests are still timing out before completely running, so the jury is still out on those.

Comment 4

[(not currently active) Ted Mielczarek]

Given that we've had mochitest-a11y in production for a while, I think this is WFM.