It would be nice to have (UI) tests for features relying on restricted profiles. We probably won't get much manual testing coverage. Since we are hiding and disabling features for restricted profiles this part of the code is great for introducing regressions. So let's investigate whether we can test these things somehow.
If this ends up being too difficult to automate we can create a set of manual tests and define some critical steps that must be tested every release.
Having UI tests for restricted profiles seems to be out of scope for 42. But I'd like to have at least a simple test that verifies that all restrictions are not enforced (isAllowed(*) == true) whenever using a normal profile.
Assignee: nobody → s.kaspari
Status: NEW → ASSIGNED
Simple test to verify that isAllowed() will return true for all restrictions using a normal profile.
Attachment #8640494 - Flags: review?(margaret.leibovic)
Comment on attachment 8640494 [details] [diff] [review] 1184094-restrictions-test.patch Review of attachment 8640494 [details] [diff] [review]: ----------------------------------------------------------------- Nice sanity check. Let's make sure it's green on try before landing, though, because automation can be flakey.
Attachment #8640494 - Flags: review?(margaret.leibovic) → review+
(In reply to :Margaret Leibovic from comment #4) > Nice sanity check. Let's make sure it's green on try before landing, though, > because automation can be flakey. Yeah, I guess I'll push all "land-able" kidfox patches at once to try today.
(In reply to Sebastian Kaspari (:sebastian) from comment #5) > Yeah, I guess I'll push all "land-able" kidfox patches at once to try today. https://treeherder.mozilla.org/#/jobs?repo=try&revision=f1c1a95bb1c2
url: https://hg.mozilla.org/integration/fx-team/rev/06ea5f2003a2484690df9a935050d23e1f32f4f7 changeset: 06ea5f2003a2484690df9a935050d23e1f32f4f7 user: Sebastian Kaspari <firstname.lastname@example.org> date: Fri Jul 31 09:49:04 2015 +0200 description: Bug 1184094 - testRestrictions: No restrictions should be enforced when using a normal profile. r=margaret
(In reply to Kevin Brosnan [:kbrosnan] from comment #1) > If this ends up being too difficult to automate we can create a set of > manual tests and define some critical steps that must be tested every > release. Kevin: What would be the process to create this set of manual tests? Most v1 bugs are about hiding features and UI elements. So I guess most manual tests would verify that these features are still accessible with a normal profile and hidden with a restricted profile. Restricted profiles can be configured by the device admin so another test could be to verify that with all restrictions set to OFF the restricted profile behaves almost like a normal profile.
After talking the gbrown, the conclusion is that automated tests cannot happen until the migration to autophone occurs. the tablet emulator is not running a new enough version and the actual hardware are not tablets.
To clarify..."Normal" automated unit tests run on emulated Android 2.3 tablets and emulated Android 4.3 phones, and neither configuration supports restricted profiles. Autophone runs a select group of tests against a variety of real devices. Theoretically, we could run new autophone tests for restricted profiles, limited to certain appropriate (4.3+) devices (perhaps manually pre-configured with a restricted profile?). This is not technically blocked by the project which is migrating Android Talos tests to autophone, but the people who are most familiar with autophone are currently focused on the Talos migration. autophone tests run today on most trees, but are hidden (they are "Tier-2" jobs). The obvious challenges for restricted profile autophone tests are simply: - writing the test(s) - finding appropriate devices - ensuring sufficient capacity for running the new test(s)
Closing this bug. The scope was investigating. That's what we did. We have a simple sanity check and /could/ write UI tests in the future.
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
You need to log in before you can comment on or make changes to this bug.