Create Imagecompare RTL test for Settings:Device

RESOLVED FIXED

Status

RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: njpark, Assigned: njpark)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

3 years ago
Create Imagecompare script that covers the Device section of Settings app
(Assignee)

Updated

3 years ago
Assignee: nobody → npark
No longer depends on: 1195992, 1197372
Created attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master
(Assignee)

Updated

3 years ago
Attachment #8652997 - Flags: review?(martijn.martijn)
Comment on attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master

I'm still looking at the pull request, but I see one pattern that needs to be fixed in many places.
You do something like this:
        Wait(self.marionette).until(expected.element_displayed(
+            self.marionette.find_element(*self._notice_text_locator)))

Where you can actually do this:
        Wait(self.marionette).until(expected.element_displayed(*self._notice_text_locator))

You need to fix all of those instances.
Attachment #8652997 - Flags: review?(martijn.martijn) → review-
(Assignee)

Comment 3

3 years ago
Comment on attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master

Is there a way to look into the content of Iframe?  I can't seem to find basic elements within.  I also put a detailed question in the PR.
Attachment #8652997 - Flags: review- → review?(martijn.martijn)
Comment on attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master

Looks good to me, apart for some nits.
These are the mismatches, I got:
http://people.mozilla.org/~mwargers/imgcomparescreenshots/mismatches5/
One related to battery charging, one related to Downloads (the "No Downloads" text comes a bit later, you might want to adjust your screenshot taking time for that) and one related to crash reporting (which was my fault, because one crash interfered during the test).
Attachment #8652997 - Flags: review?(martijn.martijn) → review+
Oh, btw, I tried to run the test on the Aries device and that didn't work.
It started failing in tap_legal_info(), when I added scrollintoview before the element tapping there, it worked (apparently, the resolution difference is interfering with your scrolling).
After that, I got another failure in return_to_prev_menu with .tap(25, 25). I gave up on that. I guess that coordinate causes it to not work in Aries.
(Assignee)

Comment 6

3 years ago
Aha, so this doesn't work on Aries?  I guess I'll have to open a new bug to make the RTL tests to work with Aries device.  Thanks for letting me know.

I saw the mismatches too, I'll get them on new builds because it lists the gecko and gaia build version as well, I'm documenting the discrepancies so the testers would be aware of them.
(Assignee)

Comment 7

3 years ago
Comment on attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master

resubmitting for review after using the iframe check.
Attachment #8652997 - Flags: review+ → review?(martijn.martijn)
Comment on attachment 8652997 [details] [review]
[gaia] npark-mozilla:1197910 > mozilla-b2g:master

Looks great, weird that I didn't get this switch to iframe thing working when I tried it, but it's certainly working now. This time I also didn't get any imagecompare failures.

(In reply to No-Jun Park [:njpark] from comment #6)
> Aha, so this doesn't work on Aries?  I guess I'll have to open a new bug to
> make the RTL tests to work with Aries device.  Thanks for letting me know.

Yes, what I saw was that apparently, the scrolling you do is not scrolling far enough on the Aries device, which results at some point that the element can't be tapped upon, because it's out of view.
Another failure was  in return_to_prev_menu with .tap(25, 25), which looks also like a coordinate issue.
Attachment #8652997 - Flags: review?(martijn.martijn) → review+
(Assignee)

Comment 9

3 years ago
merged:
https://github.com/mozilla-b2g/gaia/commit/ab26e8748a63cbc6c7c7fa26574001ef15acca2d
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Blocks: 1200321
(Assignee)

Updated

3 years ago
No longer blocks: 1200321
You need to log in before you can comment on or make changes to this bug.