Closed Bug 1184046 Opened 6 years ago Closed 4 years ago

Make open new private tab tests more robust

Categories

(Firefox for Android Graveyard :: Testing, defect)

All
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: mhaigh, Unassigned)

References

Details

Attachments

(1 file)

Bug 1175970 introduced a UI element on the first time a user opens a new tab, also it introduced a change to the testPrivateBrowsing test which cancels the new UI when any tab is opened.  This is pretty fragile as if two private tabs are open in a test, this will cause it to fail.

Lets make these tests a bit more robust.
Have adjusted the tests as requested.  We now automatically track the state of if we've seen the TP UI before or not and handle it in PixelTest.  I've also added some verification in to testPrivateBrowing to test that we show UI on the first private tab, and also add a second private tab to ensure that this works as normal.

How's this looking to you?

Also - there's a bit of noise in there, my IDE decided to trim some spaces and everytime I save now it keeps trimming, no biggie but wanted to point it out.
Attachment #8634645 - Flags: feedback?(margaret.leibovic)
Comment on attachment 8634645 [details] [diff] [review]
Make open new private tab tests more robust

Review of attachment 8634645 [details] [diff] [review]:
-----------------------------------------------------------------

::: mobile/android/tests/browser/robocop/PixelTest.java
@@ +77,5 @@
> +    protected void dismissTrackingProtectionUI() {
> +        waitForText(mStringHelper.TRACKING_PROTECTION_PROMPT_TITLE);
> +        mSolo.clickOnText(mStringHelper.TRACKING_PROTECTION_PROMPT_BUTTON);
> +        waitForText(mStringHelper.PRIVATE_BROWSING_TITLE);
> +    }

Instead of adding methods in here to deal with the tracking protection UI, I was thinking we would find a way to set PREFS_TRACKING_PROTECTION_PROMPT_SHOWN to true before any tests run.

We already have a way to do this for gecko prefs:
http://mxr.mozilla.org/mozilla-central/source/testing/profiles/prefs_general.js

I don't think we currently have a way to do this for SharedPreferences, so maybe now would be a good time to figure that out. Or we could just set this pref in some central place, such as BaseTest.

It seems wrong to me for PixelTest to be dealing with this tracking protection UI.

::: mobile/android/tests/browser/robocop/testPrivateBrowsing.java
@@ +46,4 @@
>  
> +        // Add another private tab, this time the Tracking Protection UI shouldn't show.
> +        addTab(bigLinkUrl, true);
> +        verifyTabCount(2);

Does this actually verify that the tracking protection onboarding screen didn't show? Is there a way to check for the absence of some text on the screen?
Attachment #8634645 - Flags: feedback?(margaret.leibovic) → feedback-
gbrown, do you know if there's any way for us to modify SharedPreferences in our testing profile, similarly to how we can set gecko prefs in testing/profiles/prefs_general.js?

If not, do you have a way to recommend setting a SharedPreference for all test runs? We're looking to disable a piece on onboarding UI that appears when you first open a new private tab.

Actually, liuche, what did you end up doing to solve this problem for the first run UI? I remember there were some test struggles with that at one point.
Flags: needinfo?(liuche)
Flags: needinfo?(gbrown)
I am pretty sure we have no support for setting SharedPreferences in tests. You asked about "for all test runs" -- do you need this for all test suites, like mochitest and reftest? That seems difficult. If we can limit our aspirations to Robocop, I think some new code in BaseRobocopTest (hook into setUp()?) might be the way to go.
Flags: needinfo?(gbrown)
We do all the test disabling on the Java side, actually. The robocop BaseTest starts the test with an Intent extra that disables first run.

http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/BrowserApp.java#908
http://mxr.mozilla.org/mozilla-central/source/mobile/android/tests/browser/robocop/BaseRobocopTest.java#130
Flags: needinfo?(liuche)
Assignee: mhaigh → nobody
Mass closing Firefox for Android :: Testing bugs with no progress in 2017.

If this bug is important to you, please re-open.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.