Closed Bug 1337370 Opened 8 years ago Closed 8 years ago

Add test coverage for startup crash issue exposed by patch in bug 1326084

Categories

(Core :: Disability Access APIs, defect)

Product:
Core
Component:
Disability Access APIs
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1326084
Tracking Flags:
Tracking Status
firefox54 --- affected

People

(Reporter: jimm, Assigned: yzen)

Details

(Whiteboard: aes+)

Some work landed in bug 1326084 that resulted in a startup crash that pretty much hit everyone on Windows. This should have been caught by a11y test coverage. I turns out that we have no test coverage for e10s and a11y, which has a mochitest suite that runs within the oth test suite. survey of test coverage for a11y for mozilla-central and mozilla-inbound: * Windows 7 opt: non-e10s * Windows 7 pgo: no coverage * Windows 7 VM debug: no coverage * Windows 8 opt: no coverage * Windows 8 x64 opt: non-e10s * Windows 8 x64 pgo: non-e10s * Windows 8 x64 debug: non-e10s * Windows 10 x64 VM opt: no coverage * Windows 10 x64 VM debug: no coverage * Windows 2012 opt: no coverage * Windows 2012 pgo: no coverage * Windows 2012 debug: no coverage * Windows 2012 x64 opt: no coverage * Windows 2012 x64 pgo: no covergae * Windows 2012 x64 debug: no coverage So we have zero test coverage of accessibility with e10s enabled, and very little test coverage with non-e10s, which is probably ok since we're not far off from dropping official support for it in our main dev branches. We need to get e10s oth running asap. This is going to block out rollout of e10s + a11y.
Assignee: nobody → yzenevich
If capacity is an issue, we should drop non-e10s runs and replace them with e10s runs. We should also consider running other mochitests suites with a11y forced on. It might be a good opportunity to break a11y out from other into a dedicated suite, and add existing mochitest tests to it for general testing.
I asked multiple times about getting the mochitest-e10s-a11y suite enabled in production and was told it wouldn't be helpful (see bug 1305124). Also, it should be noted that we do in fact have a11y browser-chrome tests running under e10s mode: https://dxr.mozilla.org/mozilla-central/source/accessible/tests/browser/e10s/browser.ini Except they're still disabled on Windows (supposedly due to bug 1269369, though that was fixed nearly a month ago). Maybe we should prioritize getting those enabled? :)
Enabling tests is currently dependent on bug 1288839. Some tests can be enabled at this point, I suspect. In terms of coverage the tests do the following: * All tests are run in parent process for DOM and DOM changes in content (unlike mochitests) * Updates for accessible properties that happen on proxy accessibles * Accessible events from content are propagated to parent * Accessible tree mutations are correct for DOM in content process
Yura as per chat, let's enable whatever browser-chrome tests are ready. No sense waiting. We do also need to figure out a11y startup test(s) though. I'm not sure we've ever that that under automation and we should. We should use this bug for that.
(In reply to Jim Mathies [:jimm] from comment #1) > If capacity is an issue, we should drop non-e10s runs and replace them with > e10s runs. I'd argue that non-10s has to be kept running until e10s a11y is ready and the most of users switched to it. > We should also consider running other mochitests suites with a11y > forced on. > > It might be a good opportunity to break a11y out from other into a dedicated > suite, and add existing mochitest tests to it for general testing. I second that this is a good idea since it's a good stress test for the a11y engine. I'm not sure though what scope of this bug is. Is it either one of these or all of them: * enable a11y mochitests on e10s build * enable e10s a11y browser tests * run all other test suits with a11y forced on * new tests.
(In reply to alexander :surkov from comment #6) > (In reply to Jim Mathies [:jimm] from comment #1) > > If capacity is an issue, we should drop non-e10s runs and replace them with > > e10s runs. > > I'd argue that non-10s has to be kept running until e10s a11y is ready and > the most of users switched to it. > > > We should also consider running other mochitests suites with a11y > > forced on. > > > > It might be a good opportunity to break a11y out from other into a dedicated > > suite, and add existing mochitest tests to it for general testing. > > I second that this is a good idea since it's a good stress test for the a11y > engine. > > I'm not sure though what scope of this bug is. Is it either one of these or > all of them: > * enable a11y mochitests on e10s build > * enable e10s a11y browser tests > * run all other test suits with a11y forced on > * new tests. Locally import the patches that broke Windows builds and push to try for the a11y bc and oth tests turned on for Windows to see if they turn orange. If that's true, we have test coverage that would have caught this.
(In reply to Jim Mathies [:jimm] from comment #7) > > I'm not sure though what scope of this bug is. Is it either one of these or > > all of them: > > * enable a11y mochitests on e10s build > > * enable e10s a11y browser tests > > * run all other test suits with a11y forced on > > * new tests. > > Locally import the patches that broke Windows builds and push to try for the > a11y bc and oth tests turned on for Windows to see if they turn orange. If > that's true, we have test coverage that would have caught this. Yura, do you have more data on this? Is a problem that our mochitests are running too late and start-up crash cannot be caught by these? Would bc tests be helpful here?
Yura successfully wrote a test for this and posted it to bug 1326084.
Jim shall we close this bug? (see comment 9)
Flags: needinfo?(jmathies)
Thanks!
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: needinfo?(jmathies)
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.