Write Gaia-ui accessibility tests for the first time use app.

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: yzen, Assigned: yzen)

Tracking

({access})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [b2ga11y p=1] )

Attachments

(1 attachment, 1 obsolete attachment)

47 bytes, text/x-github-pull-request
Details | Review | Splinter Review
(Assignee)

Description

4 years ago
Need to write some gaia-ui tests for the FTU a11y.
(Assignee)

Updated

4 years ago
Component: Gaia::First Time Experience → Gaia::UI Tests
(Assignee)

Comment 1

4 years ago
Created attachment 8433813 [details] [review]
Github PR
Attachment #8433813 - Flags: review?(zcampbell)
(Assignee)

Comment 2

4 years ago
Travis is happy, Zac, let me know if you have time to take a look at this or I should mark someone else for r? Thanks!
Flags: needinfo?(zcampbell)

Comment 3

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

r-, a concern about the naming structure on the test and location of `a11y_select`

Also the test fails on a device with a SIM because you get the 'cell data' panel. It is hidden with no sim.
Attachment #8433813 - Flags: review?(zcampbell) → review-
Flags: needinfo?(zcampbell)
(Assignee)

Comment 4

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

Updated the pull requests with concerns addressed.
Attachment #8433813 - Flags: review- → review?(zcampbell)

Comment 5

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

r-. Looking very good but for 2 nits:
- remove the SIM based condition, assume that always 1 sim is present and working
- add the test into tbpl-manifest.ini
Attachment #8433813 - Flags: review?(zcampbell) → review-
(Assignee)

Comment 6

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

Updated.. Thanks for the review.
Attachment #8433813 - Flags: review- → review?(zcampbell)
(Assignee)

Comment 7

4 years ago
So it looks like it's green on Travis, but fails on tbpl: https://tbpl.mozilla.org/?tree=Gaia-Try&rev=77332a8801f3 . Would you have any ideas? Can it be that the data screen is on there?
Flags: needinfo?(zcampbell)

Comment 8

4 years ago
I know why Yura.

The profiles on TBPL and Travis are different. TBPL has a mock carrier/SIM card in it and Travis does not. Thus here[1] you're hitting the cell data panel which you would not hit on Travis.
There's some back history on it here: https://bugzilla.mozilla.org/show_bug.cgi?id=1013201

Maybe we'll have to put a condition into the test (to skip this panel on Travis) after all!


[1] http://mozilla-releng-blobs.s3.amazonaws.com/blobs/gaia-try/sha512/9464e9bcfe34c1326c645bddb11e244544200d18970a3adeb15b88a263aef11dd77ecf11b8e5670b2247e98370bc929e83c60d738d1c46e489e23c949a0b6d36
^ get this by clicking output.html on the TBPL job status panel.
Flags: needinfo?(zcampbell)

Comment 9

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

r? again when you've sorted it as per comment #8!
Attachment #8433813 - Flags: review?(zcampbell)
(Assignee)

Comment 10

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

test_a11y_ftu_desktopb2g.py will only run on TBPL.
Attachment #8433813 - Flags: review?(zcampbell)
(Assignee)

Comment 11

4 years ago
TBPL looks good in terms of passing these now: https://tbpl.mozilla.org/?tree=Gaia-Try&rev=7531967fbfe5

Comment 12

4 years ago
Comment on attachment 8433813 [details] [review]
Github PR

good move!
Attachment #8433813 - Flags: review?(zcampbell) → review+

Comment 13

4 years ago
Merged:
https://github.com/mozilla-b2g/gaia/commit/17f9e468f258f45ebe0513f9734e37dd741822cf
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Unfortunately I've had to revert this out for intermittent test failures, eg:

TEST-UNEXPECTED-FAIL | test_a11y_ftu_desktopb2g.py test_a11y_ftu_desktopb2g.TestFtuAccessibility.test_a11y_ftu | ScriptTimeoutException: ScriptTimeoutException: timed out
https://tbpl.mozilla.org/php/getParsedLog.php?id=42099022&tree=Mozilla-Inbound

TEST-UNEXPECTED-FAIL | test_a11y_ftu_desktopb2g.py test_a11y_ftu_desktopb2g.TestFtuAccessibility.test_a11y_ftu | ScriptTimeoutException: ScriptTimeoutException: timed out
https://tbpl.mozilla.org/php/getParsedLog.php?id=42107639&tree=Mozilla-Central

https://github.com/mozilla-b2g/gaia/commit/7506b713672a5e802073993c14173215043f776d
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 16

4 years ago
Depends on some improvements to accessibility.js in bug 1018214
Depends on: 1018214
(Assignee)

Comment 17

4 years ago
Created attachment 8443690 [details] [review]
GIthub PR

Carrying over the r+ from Zac. Removed the flag from manifest.ini
Attachment #8433813 - Attachment is obsolete: true
Re-aligning priorities with 2.1 accessibility goals.
Whiteboard: [b2ga11y p=1]
(Assignee)

Comment 19

4 years ago
Hi Zac, do you think we should try putting this back in? I re-ran the tests on gaia-try after some changes to accessibility.js atom, I can do that again if you'd like.
Flags: needinfo?(zcampbell)

Comment 20

4 years ago
You can land it again (it's your code after all) but just be on the lookout for a backout or some intermittents reported by the Sheriffs.
Flags: needinfo?(zcampbell)
(Assignee)

Comment 22

4 years ago
(In reply to Yura Zenevich [:yzen] from comment #21)
> https://tbpl.mozilla.org/?tree=Try&rev=09ecef7dee97

Looks like there's no data screen again, new try:

https://tbpl.mozilla.org/?rev=689aaf8e593342ce5c8b028de91b85aec3d51120&tree=Gaia-Try
(Assignee)

Comment 25

4 years ago
Try was successful, will monitor in case intermittents arise.
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.