Open Bug 1665751 Opened 2 years ago Updated 2 years ago

not running textselection/* results in failing tests

Categories

(Core :: Disability Access APIs, defect, P3)

defect

Tracking

()

ASSIGNED

People

(Reporter: jmaher, Assigned: MarcoZ)

Details

Attachments

(3 files)

here is a try push:
https://treeherder.mozilla.org/#/jobs?repo=try&selectedTaskRun=MfpS9pQtQNOP-UqJT63BtQ.0&revision=40f8e9e2f1df6e46bd55bfc831ba59649a3e2f39

and these two failures occur when I skip the 2 tests in testselection:
TEST-UNEXPECTED-FAIL | accessible/tests/mochitest/tree/test_img.html | Test timed out.
TEST-UNEXPECTED-FAIL | accessible/tests/mochitest/treeupdate/test_imagemap.html | Test timed out.

there are two tests:
[test_general.html]
[test_userinput.html]

If I only run test_general.html, I get a failure.
If I only run test_userinput.html, I get no failures.

this indicates that test_userinput.html is causing the browser to be in a state which future tests depends on, or that test_general.html puts the browser in a state that test_userinput.html resets.

https://searchfox.org/mozilla-central/source/accessible/tests/mochitest/textselection/test_userinput.html

I was not able to see any issues running these tests locally on my windows 10 laptop, but on try this fails on all platforms.

I would recommend disabling one or both of these tests so we don't end up with odd test dependencies in tree and can schedule appropriate tests for a11y (we are trying to switch to manifest based scheduling which could exclude some tests based on historical data of failures and code changes)

We saw these tests fail on autoland; see bug 1520779 comment 16. Does that mean autoland doesn't run these text selection tests sometimes or does that suggest that this isn't related to the selection tests?

It seems that creation of image map accessibles is unreliable; see bug 799781. In those tests, we move the mouse over the image map to force its creation (bug 745788), which has been sufficient until now. Perhaps something changed recently which makes even this hack unreliable?

for context, we had switched to manifest based scheduling instead of task based scheduling. At the manifest level we select the most relevant manifests to run, not the most relevant tasks to run. A task would typically run 10-40 manifests, where at the manifest level we usually run 25-50% of the manifests in a given task.

In this specific case that we saw on autoland, we had removed 6 manifests from a11y and ran into this problem, that was narrowed down to removing test_userinput.html and causing the same problem.

I don't mind pushing a few try pushes, how far back in history do we want to go to test for recent breakage?

Joel, I am shortly going to upload a patch for that file which converts it from our old event queue based approach to async/await. Can you do a try push to see if that solves the problem with the manifests?

Flags: needinfo?(jmaher)
Assignee: nobody → mzehe
Status: NEW → ASSIGNED
Flags: needinfo?(jmaher)

unfortunately this change didn't fix the problem

Even though this particular doesn't fix the bug yet, it is still good to have it because it gets rid of old cruft.

Keywords: leave-open
Pushed by mzehe@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6917838bb773
Convert test_userinput.html to promisified events, r=yzen DONTBUILD
Pushed by mzehe@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ac97b9243102
Part 2: Factor out the Select All And Focus element sequence into a utility function for use by other test files, r=yzen DONTBUILD
https://hg.mozilla.org/integration/autoland/rev/9d74ce5f7e04
Part 3: Rewrite textSelection/test_general.html to use async await instead of event queue, r=yzen DONTBUILD
Severity: -- → S3
Priority: -- → P3

The leave-open keyword is there and there is no activity for 6 months.
:Jamie, maybe it's time to close this bug?

Flags: needinfo?(jteh)

The patches landed here clean things up (which is still useful), but unfortunately didn't fix the failure. So, this bug is still valid and should remain open.

Flags: needinfo?(jteh)
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.