Make gaiatest not depend on SpecialPowers

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jgriffin, Assigned: jgriffin)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

3 years ago
We're removing SpecialPowers from Marionette in bug 1149618.  Most of the uses of SpecialPowers in gaiatest are adaptable, however the accessibility tests present a special challenge.  They use chrome API's on content elements, and there's no way to make that work in Marionette without SpecialPowers.

Desktop tests which use that API are all chrome-based.

We will be able to initialize SpecialPowers in the runner even after it's removed from core Marionette, and that may be the best approach here.

Dave, are the gaiatest a11y tests run on real devices, or just b2gdesktop builds?
(Assignee)

Updated

3 years ago
Blocks: 1149618
Flags: needinfo?(dave.hunt)
As far as I'm aware the a11y tests are only regularly run on b2g desktop builds. That said, there's nothing to stop them from being run on devices, and I have run them on devices in the past due to them being referenced in the master manifest file: https://github.com/mozilla-b2g/gaia/blob/17ae7477c2f0042c9857b0537611cf5b6f18933d/tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini#L55

Geo might know if there are any current or future plans for running a11y tests against device, however I strongly suspect not.
Flags: needinfo?(dave.hunt) → needinfo?(gmealer)
I don't--my team doesn't own those, so we really only intersect at the page objects. Best bet would be to chase the git logs to see who's working on them nowadays.
Flags: needinfo?(gmealer)
Created attachment 8600619 [details] [review]
[gaia] jonallengriffin:removeSpecialPowers > mozilla-b2g:master
(Assignee)

Comment 4

3 years ago
Created attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

I was able to keep the a11y tests intact.  Here's a green try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=300a308db174

I changed the way we load accessibility.js so that we don't always have Components.classes in imported scripts, otherwise this patch mostly relies on moving things to chrome context if needed, or using the new sandbox='system' feature from bug 1149618.

Note that the test run automatically kicked off for this PR will fail because bug 1149618 hasn't landed yet; the try run linked above points to my gaia fork with these changes.
Attachment #8600620 - Flags: review?(dave.hunt)
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

I've raised a couple of questions on the pull request. Have you tried running these tests on device? If not, I can do so either using your patches from the other bug or waiting for them to land on m-c.
Attachment #8600620 - Flags: review?(dave.hunt) → review-
(Assignee)

Updated

3 years ago
Blocks: 1161606
(Assignee)

Comment 6

3 years ago
(In reply to Dave Hunt (:davehunt) from comment #5)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> I've raised a couple of questions on the pull request. Have you tried
> running these tests on device? If not, I can do so either using your patches
> from the other bug or waiting for them to land on m-c.

I haven't run them on device.  The relevant gecko changes have landed on m-c, but I haven't bumped marionette-driver or marionette-client yet.
(Assignee)

Comment 7

3 years ago
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

Review comments addressed, new try run:  https://treeherder.mozilla.org/#/jobs?repo=try&revision=3a3afaac56f0
Attachment #8600620 - Flags: review- → review?(dave.hunt)
(Assignee)

Updated

3 years ago
Assignee: nobody → jgriffin
Status: NEW → ASSIGNED
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

r=me with a couple of nits addressed. I've also successfully run a few functional and accessibility tests on device.

We should land this after new releases of the driver and client are out, and include a version bump in requirements.txt. I think it makes sense to pin marionette-driver to a specific version in addition to marionette-client to avoid incompatible new releases being automatically picked up.
Attachment #8600620 - Flags: review?(dave.hunt) → review+
(Assignee)

Comment 9

3 years ago
(In reply to Dave Hunt (:davehunt) from comment #8)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> r=me with a couple of nits addressed. I've also successfully run a few
> functional and accessibility tests on device.
> 
> We should land this after new releases of the driver and client are out, and
> include a version bump in requirements.txt. I think it makes sense to pin
> marionette-driver to a specific version in addition to marionette-client to
> avoid incompatible new releases being automatically picked up.

I've addressed the nits, updated requirements.txt with the new pinned versions of both client and driver, and the resulting try run is green:  https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=bd99d75409ae69b9ffba4ec236b1affefc8ea334

Dave, let me know if this is safe to merge.
Flags: needinfo?(dave.hunt)
I've checked out your branch and successfully run both functional and accessibility tests on device, so I'd say this is safe to merge.
Flags: needinfo?(dave.hunt)
(Assignee)

Updated

3 years ago
Keywords: checkin-needed

Updated

3 years ago
Keywords: checkin-needed
Pull request has landed in master: https://github.com/mozilla-b2g/gaia/commit/0a9f9b4939bdef6a1b34cac4803ee899d903c7c9

Updated

3 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.