1150755 - Make gaiatest not depend on SpecialPowers

Status: RESOLVED FIXED

Description

[Jonathan Griffin (:jgriffin)]

We're removing SpecialPowers from Marionette in bug 1149618.  Most of the uses of SpecialPowers in gaiatest are adaptable, however the accessibility tests present a special challenge.  They use chrome API's on content elements, and there's no way to make that work in Marionette without SpecialPowers.

Desktop tests which use that API are all chrome-based.

We will be able to initialize SpecialPowers in the runner even after it's removed from core Marionette, and that may be the best approach here.

Dave, are the gaiatest a11y tests run on real devices, or just b2gdesktop builds?

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚️UTC]

As far as I'm aware the a11y tests are only regularly run on b2g desktop builds. That said, there's nothing to stop them from being run on devices, and I have run them on devices in the past due to them being referenced in the master manifest file: https://github.com/mozilla-b2g/gaia/blob/17ae7477c2f0042c9857b0537611cf5b6f18933d/tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini#L55

Geo might know if there are any current or future plans for running a11y tests against device, however I strongly suspect not.

Comment 2

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

I don't--my team doesn't own those, so we really only intersect at the page objects. Best bet would be to chase the git logs to see who's working on them nowadays.

Comment 3

[GitHub Autolander Bot]

Created attachment 8600619 [details] [review]
[gaia] jonallengriffin:removeSpecialPowers > mozilla-b2g:master

Comment 4

[Jonathan Griffin (:jgriffin)]

Created attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

I was able to keep the a11y tests intact.  Here's a green try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=300a308db174

I changed the way we load accessibility.js so that we don't always have Components.classes in imported scripts, otherwise this patch mostly relies on moving things to chrome context if needed, or using the new sandbox='system' feature from bug 1149618.

Note that the test run automatically kicked off for this PR will fail because bug 1149618 hasn't landed yet; the try run linked above points to my gaia fork with these changes.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚️UTC]

Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

I've raised a couple of questions on the pull request. Have you tried running these tests on device? If not, I can do so either using your patches from the other bug or waiting for them to land on m-c.

Comment 6

[Jonathan Griffin (:jgriffin)]

(In reply to Dave Hunt (:davehunt) from comment #5)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> I've raised a couple of questions on the pull request. Have you tried
> running these tests on device? If not, I can do so either using your patches
> from the other bug or waiting for them to land on m-c.

I haven't run them on device.  The relevant gecko changes have landed on m-c, but I haven't bumped marionette-driver or marionette-client yet.

Comment 7

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

Review comments addressed, new try run:  https://treeherder.mozilla.org/#/jobs?repo=try&revision=3a3afaac56f0

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚️UTC]

Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

r=me with a couple of nits addressed. I've also successfully run a few functional and accessibility tests on device.

We should land this after new releases of the driver and client are out, and include a version bump in requirements.txt. I think it makes sense to pin marionette-driver to a specific version in addition to marionette-client to avoid incompatible new releases being automatically picked up.

Comment 9

[Jonathan Griffin (:jgriffin)]

(In reply to Dave Hunt (:davehunt) from comment #8)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> r=me with a couple of nits addressed. I've also successfully run a few
> functional and accessibility tests on device.
> 
> We should land this after new releases of the driver and client are out, and
> include a version bump in requirements.txt. I think it makes sense to pin
> marionette-driver to a specific version in addition to marionette-client to
> avoid incompatible new releases being automatically picked up.

I've addressed the nits, updated requirements.txt with the new pinned versions of both client and driver, and the resulting try run is green:  https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=bd99d75409ae69b9ffba4ec236b1affefc8ea334

Dave, let me know if this is safe to merge.

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚️UTC]

I've checked out your branch and successfully run both functional and accessibility tests on device, so I'd say this is safe to merge.

Comment 11

[GitHub Autolander Bot]

Pull request has landed in master: https://github.com/mozilla-b2g/gaia/commit/0a9f9b4939bdef6a1b34cac4803ee899d903c7c9