Closed Bug 1150755 Opened 6 years ago Closed 6 years ago

Make gaiatest not depend on SpecialPowers

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jgriffin, Assigned: jgriffin)

References

Details

Attachments

(2 files)

We're removing SpecialPowers from Marionette in bug 1149618.  Most of the uses of SpecialPowers in gaiatest are adaptable, however the accessibility tests present a special challenge.  They use chrome API's on content elements, and there's no way to make that work in Marionette without SpecialPowers.

Desktop tests which use that API are all chrome-based.

We will be able to initialize SpecialPowers in the runner even after it's removed from core Marionette, and that may be the best approach here.

Dave, are the gaiatest a11y tests run on real devices, or just b2gdesktop builds?
Blocks: 1149618
Flags: needinfo?(dave.hunt)
As far as I'm aware the a11y tests are only regularly run on b2g desktop builds. That said, there's nothing to stop them from being run on devices, and I have run them on devices in the past due to them being referenced in the master manifest file: https://github.com/mozilla-b2g/gaia/blob/17ae7477c2f0042c9857b0537611cf5b6f18933d/tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini#L55

Geo might know if there are any current or future plans for running a11y tests against device, however I strongly suspect not.
Flags: needinfo?(dave.hunt) → needinfo?(gmealer)
I don't--my team doesn't own those, so we really only intersect at the page objects. Best bet would be to chase the git logs to see who's working on them nowadays.
Flags: needinfo?(gmealer)
I was able to keep the a11y tests intact.  Here's a green try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=300a308db174

I changed the way we load accessibility.js so that we don't always have Components.classes in imported scripts, otherwise this patch mostly relies on moving things to chrome context if needed, or using the new sandbox='system' feature from bug 1149618.

Note that the test run automatically kicked off for this PR will fail because bug 1149618 hasn't landed yet; the try run linked above points to my gaia fork with these changes.
Attachment #8600620 - Flags: review?(dave.hunt)
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

I've raised a couple of questions on the pull request. Have you tried running these tests on device? If not, I can do so either using your patches from the other bug or waiting for them to land on m-c.
Attachment #8600620 - Flags: review?(dave.hunt) → review-
Blocks: 1161606
(In reply to Dave Hunt (:davehunt) from comment #5)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> I've raised a couple of questions on the pull request. Have you tried
> running these tests on device? If not, I can do so either using your patches
> from the other bug or waiting for them to land on m-c.

I haven't run them on device.  The relevant gecko changes have landed on m-c, but I haven't bumped marionette-driver or marionette-client yet.
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

Review comments addressed, new try run:  https://treeherder.mozilla.org/#/jobs?repo=try&revision=3a3afaac56f0
Attachment #8600620 - Flags: review- → review?(dave.hunt)
Assignee: nobody → jgriffin
Status: NEW → ASSIGNED
Comment on attachment 8600620 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840

r=me with a couple of nits addressed. I've also successfully run a few functional and accessibility tests on device.

We should land this after new releases of the driver and client are out, and include a version bump in requirements.txt. I think it makes sense to pin marionette-driver to a specific version in addition to marionette-client to avoid incompatible new releases being automatically picked up.
Attachment #8600620 - Flags: review?(dave.hunt) → review+
(In reply to Dave Hunt (:davehunt) from comment #8)
> Comment on attachment 8600620 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/29840
> 
> r=me with a couple of nits addressed. I've also successfully run a few
> functional and accessibility tests on device.
> 
> We should land this after new releases of the driver and client are out, and
> include a version bump in requirements.txt. I think it makes sense to pin
> marionette-driver to a specific version in addition to marionette-client to
> avoid incompatible new releases being automatically picked up.

I've addressed the nits, updated requirements.txt with the new pinned versions of both client and driver, and the resulting try run is green:  https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=bd99d75409ae69b9ffba4ec236b1affefc8ea334

Dave, let me know if this is safe to merge.
Flags: needinfo?(dave.hunt)
I've checked out your branch and successfully run both functional and accessibility tests on device, so I'd say this is safe to merge.
Flags: needinfo?(dave.hunt)
Keywords: checkin-needed
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.