Closed Bug 1946883 Opened 2 months ago Closed 4 days ago

Perma [tier 2] org.mozilla.fenix.ui.SettingsAddonsTest#noCrashWithAddonInstalledTest | java.lang.AssertionError: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/add_ons_progress_bar] is not gone

Categories

(Firefox for Android :: UI Tests, defect, P5)

Product:
Firefox for Android
Component:
UI Tests
Platform:
All
Android
Type:
defect

Tracking

()

Status:
RESOLVED INCOMPLETE
Tracking Flags:
Tracking Status
firefox135 --- unaffected
firefox136 --- unaffected
firefox137 --- wontfix

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: intermittent-failure)

Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=493930350&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WU4cNx5_S32H9jk7h2sFyQ/runs/0/artifacts/public/logs/live_backing.log

[task 2025-02-07T21:15:39.192Z]   Shard 99: Expected: 0s, Actual: 104s, Diff: 104s
[task 2025-02-07T21:15:39.192Z] 
[task 2025-02-07T21:15:39.192Z] 
[task 2025-02-07T21:15:39.192Z] Command /usr/bin/java -jar /builds/worker/test-tools/flank.jar android run --config ./automation/taskcluster/androidTest/flank-arm64-v8a.yml --app /builds/worker/checkouts/gecko/mobile/android/fenix/app.apk --local-result-dir /builds/worker/artifacts/results --project moz-fenix --client-details matrixLabel=autoland,geckoRev=9eb272c39eea1a5c87d2788a6bc38e560a77d206 --test /builds/worker/checkouts/gecko/mobile/android/fenix/android-test.apk failed with exit code 10
[task 2025-02-07T21:15:39.248Z] TEST-UNEXPECTED-FAIL | org.mozilla.fenix.ui.SettingsAddonsTest#noCrashWithAddonInstalledTest | java.lang.AssertionError: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/add_ons_progress_bar] is not gone
[task 2025-02-07T21:15:39.248Z]  at org.junit.Assert.fail(Assert.java:89)
[task 2025-02-07T21:15:39.248Z]  at org.junit.Assert.assertTrue(Assert.java:42)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.helpers.MatcherHelper.assertUIObjectIsGone(MatcherHelper.kt:113)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.waitForAddonsListProgressBarToBeGone(SettingsSubMenuAddonsManagerRobot.kt:69)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:253)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:252)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.ThreeDotMenuMainRobot$Transition.openAddonsManagerMenu(ThreeDotMenuMainRobot.kt:669)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.installAddon(SettingsSubMenuAddonsManagerRobot.kt:252)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:114)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:113)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobotKt.addonsMenu(SettingsSubMenuAddonsManagerRobot.kt:394)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest.noCrashWithAddonInstalledTest(SettingsAddonsTest.kt:113)
[task 2025-02-07T21:15:39.248Z] 
[task 2025-02-07T21:15:39.248Z]     
[task 2025-02-07T21:15:39.248Z] TEST-UNEXPECTED-FAIL | org.mozilla.fenix.ui.SettingsAddonsTest#noCrashWithAddonInstalledTest | java.lang.AssertionError: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/addonProgressOverlay] is not gone
[task 2025-02-07T21:15:39.248Z]  at org.junit.Assert.fail(Assert.java:89)
[task 2025-02-07T21:15:39.248Z]  at org.junit.Assert.assertTrue(Assert.java:42)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.helpers.MatcherHelper.assertUIObjectIsGone(MatcherHelper.kt:113)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.waitForAddonsDownloadOverlayToBeGone(SettingsSubMenuAddonsManagerRobot.kt:71)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.verifyAddonPermissionPrompt(SettingsSubMenuAddonsManagerRobot.kt:80)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:255)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:252)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.ThreeDotMenuMainRobot$Transition.openAddonsManagerMenu(ThreeDotMenuMainRobot.kt:669)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.installAddon(SettingsSubMenuAddonsManagerRobot.kt:252)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:114)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:113)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobotKt.addonsMenu(SettingsSubMenuAddonsManagerRobot.kt:394)
[task 2025-02-07T21:15:39.248Z]  at org.mozilla.fenix.ui.SettingsAddonsTest.noCrashWithAddonInstalledTest(SettingsAddonsTest.kt:113)
[task 2025-02-07T21:15:39.248Z] 
[task 2025-02-07T21:15:39.248Z]     
[task 2025-02-07T21:15:39.248Z] ╔══════════════════════════════════════════════════════════════════════════════════╤═════════╤═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
[task 2025-02-07T21:15:39.248Z] ║ UI Test                                                                          │ Outcome │ Details                                                                                                                                                         ║
[task 2025-02-07T21:15:39.248Z] ╟──────────────────────────────────────────────────────────────────────────────────┼─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╢
[task 2025-02-07T21:15:39.248Z] ║ org.mozilla.fenix.ui.SettingsAddonsTest#noCrashWithAddonInstalledTest            │ Failure │ java.lang.AssertionError: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/add_ons_progress_bar] is not gone                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.junit.Assert.fail(Assert.java:89)                                                                                                                       ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.junit.Assert.assertTrue(Assert.java:42)                                                                                                                 ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.helpers.MatcherHelper.assertUIObjectIsGone(MatcherHelper.kt:113)                                                                          ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.waitForAddonsListProgressBarToBeGone(SettingsSubMenuAddonsManagerRobot.kt:69)                 ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:253)                               ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:252)                               ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.ThreeDotMenuMainRobot$Transition.openAddonsManagerMenu(ThreeDotMenuMainRobot.kt:669)                                            ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.installAddon(SettingsSubMenuAddonsManagerRobot.kt:252)                                        ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:114)                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:113)                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobotKt.addonsMenu(SettingsSubMenuAddonsManagerRobot.kt:394)                                        ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest.noCrashWithAddonInstalledTest(SettingsAddonsTest.kt:113)                                                            ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.248Z] ╟──────────────────────────────────────────────────────────────────────────────────┼─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╢
[task 2025-02-07T21:15:39.248Z] ║ org.mozilla.fenix.ui.SettingsAddonsTest#noCrashWithAddonInstalledTest            │ Failure │ java.lang.AssertionError: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/addonProgressOverlay] is not gone                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.junit.Assert.fail(Assert.java:89)                                                                                                                       ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.junit.Assert.assertTrue(Assert.java:42)                                                                                                                 ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.helpers.MatcherHelper.assertUIObjectIsGone(MatcherHelper.kt:113)                                                                          ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.waitForAddonsDownloadOverlayToBeGone(SettingsSubMenuAddonsManagerRobot.kt:71)                 ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.verifyAddonPermissionPrompt(SettingsSubMenuAddonsManagerRobot.kt:80)                          ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:255)                               ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot$installAddon$3.invoke(SettingsSubMenuAddonsManagerRobot.kt:252)                               ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.ThreeDotMenuMainRobot$Transition.openAddonsManagerMenu(ThreeDotMenuMainRobot.kt:669)                                            ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobot.installAddon(SettingsSubMenuAddonsManagerRobot.kt:252)                                        ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:114)                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest$noCrashWithAddonInstalledTest$1.invoke(SettingsAddonsTest.kt:113)                                                   ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsSubMenuAddonsManagerRobotKt.addonsMenu(SettingsSubMenuAddonsManagerRobot.kt:394)                                        ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.SettingsAddonsTest.noCrashWithAddonInstalledTest(SettingsAddonsTest.kt:113)                                                            ║
[task 2025-02-07T21:15:39.248Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.249Z] ╟──────────────────────────────────────────────────────────────────────────────────┼─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╢
[task 2025-02-07T21:15:39.249Z] ║ org.mozilla.fenix.ui.TranslationsTest#downloadLanguageWhileDataSaverModeIsOnTest │ Flaky   │ androidx.compose.ui.test.ComposeTimeoutException: Condition (at least one node matches (ContentDescription = 'Bosnian 17.11 MBDelete' (ignoreCase: false))) sti ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │ ll not satisfied after 15000 ms                                                                                                                                 ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │  at androidx.compose.ui.test.AndroidComposeUiTestEnvironment$AndroidComposeUiTestImpl.waitUntil(ComposeUiTest.android.kt:441)                                   ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │  at androidx.compose.ui.test.ComposeUiTestKt.waitUntilAtLeastOneExists(ComposeUiTest.kt:236)                                                                    ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │  at androidx.compose.ui.test.junit4.AndroidComposeTestRule.waitUntilAtLeastOneExists(AndroidComposeTestRule.android.kt:322)                                     ║
[task 2025-02-07T21:15:39.249Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.SettingsTranslationsRobot.verifyDownloadedLanguage(SettingsTranslationsRobot.kt:46)                                             ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.TranslationsTest$downloadLanguageWhileDataSaverModeIsOnTest$4.invoke(TranslationsTest.kt:413)                                          ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.TranslationsTest$downloadLanguageWhileDataSaverModeIsOnTest$4.invoke(TranslationsTest.kt:403)                                          ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.robots.TranslationsRobot$Transition.clickTranslationSettingsButton(TranslationsRobot.kt:279)                                           ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │  at org.mozilla.fenix.ui.TranslationsTest.downloadLanguageWhileDataSaverModeIsOnTest(TranslationsTest.kt:403)                                                   ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.256Z] ║                                                                                  │         │                                                                                                                                                                 ║
[task 2025-02-07T21:15:39.256Z] ╚══════════════════════════════════════════════════════════════════════════════════╧═════════╧═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
[task 2025-02-07T21:15:39.256Z] Command ./automation/taskcluster/androidTest/parse-ui-test-fromfile.py --results /builds/worker/artifacts/results failed with exit code 2
[taskcluster 2025-02-07 21:15:39.631Z] === Task Finished ===
[taskcluster 2025-02-07 21:16:03.259Z] Unsuccessful task run with exit code: 10 completed in 769.727 seconds

The Bugbug bot thinks this bug should belong to the 'Fenix::UI Tests' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: General → UI Tests
Keywords: intermittent-testcase, regression
Regressed by: 1765720

:ahmadkafeel94, since you are the author of the regressor, bug 1765720, could you take a look?

For more information, please visit BugBot documentation.

Flags: needinfo?(ahmadkafeel94)

Hello, the changes in Bug 1765720 only affect the about:logins desktop component. The regression here is related to Android so I don't believe the changes there caused this.

Flags: needinfo?(ahmadkafeel94) → needinfo?(abutkovits)

Set release status flags based on info from the regressing bug 1765720

status-firefox135: --- → unaffected
status-firefox136: --- → unaffected
status-firefox137: --- → affected

:aaronmt who may be the best person to reach out to for help on debugging this regression in the Fenix component? I'm not sure where to start.

Flags: needinfo?(mtigley) → needinfo?(aaron.train)

Hey there, this is the right spot and contact. I think I know what's happening here. You are right in identifying the desktop component unrelated. This is ironically an unintended regression from bug 1946780 (disabling another test) in the change-set right above it (where this test failure is logged).

Explanation: When we disable tests, it shuffles around the Firebase Test Lab's sharding (grouping) of tests and it ended up that that this test allowNotificationsPermissionTest runs in the same shard as verifyCustomTabToolbarWithAddressBarAtTheTopTest. Why does that matter? From the video it looks like verifyCustomTabToolbarWithAddressBarAtTheTopTest accidently doesn't reset device orientation and that might be affecting the outcome of allowNotificationsPermissionTest. If that's the problem, we just need to reset portrait orientation either in the teardown or rule.

My team will take a look on Monday, thanks for reaching out.

Flags: needinfo?(aaron.train) → needinfo?(ajoltan)
Regressed by: 1946780
No longer regressed by: 1765720

I think there was a confusion between this and Bug 1946939, which was caused by Bug 1946780.
Here the cause seems to be the known network connection issue (Bug 1875240) , causing problems with loading the extensions list. It's an intermittent issue, that might be solved in the meantime.

Flags: needinfo?(ajoltan)
No longer regressed by: 1946780

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: NEW → RESOLVED
Closed: 4 days ago
status-firefox137: affected → wontfix
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.