Closed Bug 1144898 Opened 5 years ago Closed 4 years ago

Android 4.3 testSettingsMenuItems | Exception caught - junit.framework.AssertionFailedError: Text string: '^Mozilla$' is not found!

Categories

(Firefox for Android :: Testing, defect)

x86_64
Linux
defect
Not set

Tracking

()

RESOLVED WONTFIX

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(2 files, 1 obsolete file)

On the Android 4.3 emulator, testSettingsMenuItems frequently fails.

http://ftp.mozilla.org/pub/mozilla.org/mobile/try-builds/gbrown@mozilla.com-e80c0fdd1ef6/try-android-api-9/try_ubuntu64_vm_mobile_test-robocop-3-bm68-tests1-linux64-build35.txt.gz

10:38:05     INFO -  TEST-START | testSettingsMenuItems
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Robocop tests need the test device screen to be powered on. - 
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Given message occurred for registered event: {"type":"Gecko:Ready"} - Gecko:Ready should equal Gecko:Ready
10:38:05     INFO -  EventExpecter: no longer listening for Gecko:Ready
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | The Settings menu did not load - Settings
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | about:home did not load - Search or enter address
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Browser toolbar URL stayed the same - about:home should equal about:home
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | The Settings menu did not load - Settings
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for Sync option - Sync
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Paint flashing$ in section ^Developer tools$ - The ^Paint flashing$ option is present in section ^Developer tools$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Remote debugging$ in section ^Developer tools$ - The ^Remote debugging$ option is present in section ^Developer tools$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Learn more$ in section ^Developer tools$ - The ^Learn more$ option is present in section ^Developer tools$
10:38:05     INFO -  waitForCondition timeout after 10000 ms.
10:38:05     INFO -  waitForEnabledText timeout on ^Home$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Panels$ in section ^Home$ - The ^Panels$ option is present in section ^Home$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Automatic updates$ in section ^Home$ - The ^Automatic updates$ option is present in section ^Home$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item default ^Enabled$ in section ^Home$ - The ^Enabled$ default is present in section ^Home$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Enabled$ in section ^Home$ - The ^Enabled$ choice is present in section ^Home$
10:38:05     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Only over Wi-Fi$ in section ^Home$ - The ^Only over Wi-Fi$ choice is present in section ^Home$
10:38:05     INFO -  waitForCondition timeout after 10000 ms.
10:38:05     INFO -  waitForEnabledText timeout on ^Mozilla$
10:38:05     INFO -  1 ERROR Exception caught during test! - junit.framework.AssertionFailedError: Text string: '^Mozilla$' is not found!
10:38:05     INFO -  	at junit.framework.Assert.fail(Assert.java:50)
10:38:05     INFO -  	at junit.framework.Assert.assertTrue(Assert.java:20)
10:38:05     INFO -  	at com.jayway.android.robotium.solo.Clicker.clickOnText(Clicker.java:383)
10:38:05     INFO -  	at com.jayway.android.robotium.solo.Solo.clickOnText(Solo.java:1028)
10:38:05     INFO -  	at org.mozilla.gecko.tests.testSettingsMenuItems.checkMenuHierarchy(testSettingsMenuItems.java:221)
10:38:05     INFO -  	at org.mozilla.gecko.tests.testSettingsMenuItems.testSettingsMenuItems(testSettingsMenuItems.java:140)
10:38:05     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
10:38:05     INFO -  	at java.lang.reflect.Method.invoke(Method.java:525)
10:38:05     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
10:38:05     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
10:38:05     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
10:38:05     INFO -  	at org.mozilla.gecko.tests.BaseTest.runTest(BaseTest.java:132)
10:38:05     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:134)
10:38:05     INFO -  	at junit.framework.TestResult$1.protect(TestResult.java:115)
10:38:05     INFO -  	at junit.framework.TestResult.runProtected(TestResult.java:133)
10:38:05     INFO -  	at junit.framework.TestResult.run(TestResult.java:118)
10:38:05     INFO -  	at junit.framework.TestCase.run(TestCase.java:124)
10:38:05     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
10:38:05     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
10:38:05     INFO -  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
10:38:05     INFO -  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
10:38:05     INFO -  
10:38:05  WARNING -  TEST-UNEXPECTED-FAIL | testSettingsMenuItems | Exception caught - junit.framework.AssertionFailedError: Text string: '^Mozilla$' is not found!
10:38:05     INFO -  TEST-OK | testSettingsMenuItems | took 164764ms
10:38:05     INFO -  TEST-START | Shutdown
10:38:05     INFO -  Passed: 20
10:38:05     INFO -  Failed: 1
Blocks: 1136477
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Advanced$ in section ^Display$ - The ^Advanced$ option is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Character encoding$ in section ^Display$ - The ^Character encoding$ option is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item default ^Don't show menu$ in section ^Display$ - The ^Don't show menu$ default is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Show menu$ in section ^Display$ - The ^Show menu$ choice is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Don't show menu$ in section ^Display$ - The ^Don't show menu$ choice is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Plugins$ in section ^Display$ - The ^Plugins$ option is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item default ^Tap to play$ in section ^Display$ - The ^Tap to play$ default is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Enabled$ in section ^Display$ - The ^Enabled$ choice is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Tap to play$ in section ^Display$ - The ^Tap to play$ choice is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Disabled$ in section ^Display$ - The ^Disabled$ choice is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Text reflow$ in section ^Display$ - The ^Text reflow$ option is present in section ^Display$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Tracking protection$ in section ^Privacy$ - The ^Tracking protection$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Do not track$ in section ^Privacy$ - The ^Do not track$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Cookies$ in section ^Privacy$ - The ^Cookies$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item default ^Enabled$ in section ^Privacy$ - The ^Enabled$ default is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Enabled, excluding 3rd party$ in section ^Privacy$ - The ^Enabled, excluding 3rd party$ choice is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Disabled$ in section ^Privacy$ - The ^Disabled$ choice is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Remember passwords$ in section ^Privacy$ - The ^Remember passwords$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Manage logins$ in section ^Privacy$ - The ^Manage logins$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Use master password$ in section ^Privacy$ - The ^Use master password$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Section-item - Each item must at least have a title
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item ^Clear now$ in section ^Privacy$ - The ^Clear now$ option is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item default ^$ in section ^Privacy$ - The ^$ default is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Browsing history$ in section ^Privacy$ - The ^Browsing history$ choice is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Search history$ in section ^Privacy$ - The ^Search history$ choice is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Downloads$ in section ^Privacy$ - The ^Downloads$ choice is present in section ^Privacy$
13:30:51     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Form history$ in section ^Privacy$ - The ^Form history$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Cookies & active logins$ in section ^Privacy$ - The ^Cookies & active logins$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Saved passwords$ in section ^Privacy$ - The ^Saved passwords$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Cache$ in section ^Privacy$ - The ^Cache$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Offline website data$ in section ^Privacy$ - The ^Offline website data$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Site settings$ in section ^Privacy$ - The ^Site settings$ choice is present in section ^Privacy$
13:31:32     INFO -  TEST-PASS | testSettingsMenuItems | Waiting for settings item choice ^Clear data$ in section ^Privacy$ - The ^Clear data$ choice is present in section ^Privacy$
13:31:32     INFO -  waitForCondition timeout after 10000 ms.
13:31:32     INFO -  waitForEnabledText timeout on ^Customize$
13:31:32     INFO -  0 ERROR Exception caught during test! - junit.framework.AssertionFailedError: Text string: '^Customize$' is not found!
13:31:32     INFO -  	at junit.framework.Assert.fail(Assert.java:50)
13:31:32     INFO -  	at junit.framework.Assert.assertTrue(Assert.java:20)
13:31:32     INFO -  	at com.jayway.android.robotium.solo.Clicker.clickOnText(Clicker.java:383)
13:31:32     INFO -  	at com.jayway.android.robotium.solo.Solo.clickOnText(Solo.java:1028)
13:31:32     INFO -  	at org.mozilla.gecko.tests.testSettingsMenuItems.checkMenuHierarchy(testSettingsMenuItems.java:241)
13:31:32     INFO -  	at org.mozilla.gecko.tests.testSettingsMenuItems.testSettingsMenuItems(testSettingsMenuItems.java:160)
13:31:32     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
13:31:32     INFO -  	at java.lang.reflect.Method.invoke(Method.java:525)
13:31:32     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
13:31:32     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
13:31:32     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
13:31:32     INFO -  	at org.mozilla.gecko.tests.BaseTest.runTest(BaseTest.java:133)
13:31:32     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:134)
13:31:32     INFO -  	at junit.framework.TestResult$1.protect(TestResult.java:115)
13:31:32     INFO -  	at junit.framework.TestResult.runProtected(TestResult.java:133)
13:31:32     INFO -  	at junit.framework.TestResult.run(TestResult.java:118)
13:31:32     INFO -  	at junit.framework.TestCase.run(TestCase.java:124)
13:31:32     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
13:31:32     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
13:31:32     INFO -  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
13:31:32     INFO -  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
13:31:32  WARNING -  TEST-UNEXPECTED-FAIL | testSettingsMenuItems | Exception caught - junit.framework.AssertionFailedError: Text string: '^Customize$' is not found!
13:31:32     INFO -  TEST-OK | testSettingsMenuItems | took 349306ms
Attached file Test failure screenshot (obsolete) —
I made a try push re-enabling testSettingsMenuItems on 4.3, and it fails when trying to find "Customize".

It looks like we're not backing out of the Privacy menu correctly (even after finding all the Privacy settings items).
Attachment #8621868 - Attachment is obsolete: true
Bug 1144898 - Android 4.3 testSettingsMenuItems | Exception caught - junit.framework.AssertionFailedError: Text string: '^Mozilla$' is not found! r=mcomella
Attachment #8623418 - Flags: review?(michael.l.comella)
Attachment #8623418 - Flags: review?(michael.l.comella) → review+
Comment on attachment 8623418 [details]
MozReview Request: Bug 1144898 - Android 4.3 testSettingsMenuItems | Exception caught - junit.framework.AssertionFailedError: Text string: '^Mozilla$' is not found! r=mcomella

https://reviewboard.mozilla.org/r/11545/#review9921

Ship It!
I ran another try run and I'm going to re-enable for 2.3 - going to [leave open] for the 4.3 fix.
Whiteboard: [test disabled on android 4.3] → [test disabled on android 4.3][leave-open]
Assignee: liuche → nobody
testSettingsMenuItems was deleted in bug 1216257.
Assignee: nobody → gbrown
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
Whiteboard: [test disabled on android 4.3][leave-open]
You need to log in before you can comment on or make changes to this bug.