Closed Bug 1814032 Opened 2 years ago Closed 2 years ago

Intermittent UI test failure - < AddressAutofillTest.verifyAddressAutofillSelectionTest >

Categories

(Firefox for Android :: UI Tests, defect, P3)

Product:
Firefox for Android
Component:
UI Tests
Version:
Firefox 111
Platform:
All
Android
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
123 Branch
Tracking Flags:
Tracking Status
firefox111 --- wontfix
firefox112 --- wontfix
firefox113 --- wontfix
firefox114 --- wontfix
firefox121 --- wontfix
firefox122 --- wontfix
firefox123 --- fixed

People

(Reporter: ajoltan, Unassigned)

References

(Depends on 1 open bug)

Details

(Keywords: intermittent-failure, test-disabled)

Attachments

(2 files)

[mozilla-mobile/firefox-android] Bug 1814032 - Retry attempts for address autofill in UI tests
59 bytes, text/x-github-pull-request
Details | Review
[mozilla-mobile/firefox-android] Bug 1814032 - disable failing verifyAddressAutofillSelectionTest UI test (#1830)
59 bytes, text/x-github-pull-request
Details | Review

Firebase Test Run: Firebase link

Stacktrace:

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/select_address_header]
at androidx.test.uiautomator.UiObject.clickAndWaitForNewWindow(UiObject.java:456)
at org.mozilla.fenix.ui.robots.BrowserRobot.clickSelectAddressButton(BrowserRobot.kt:537)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:273)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:266)
at org.mozilla.fenix.ui.robots.NavigationToolbarRobot$Transition.enterURLAndEnterToBrowser(NavigationToolbarRobot.kt:138)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:266)

Build: 1/30 Main

Flaky 1x on 2/9 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/first_name_input]
at androidx.test.uiautomator.UiObject.setText(UiObject.java:666)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:212)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:248)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:231)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:179)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:231)

Severity: -- → S4
Priority: -- → P3
status-firefox111: --- → wontfix
Version: unspecified → Firefox 111

Flaky 1x on 3/5 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/select_address_header]
at androidx.test.uiautomator.UiObject.clickAndWaitForNewWindow(UiObject.java:456)
at org.mozilla.fenix.ui.robots.BrowserRobot.clickSelectAddressButton(BrowserRobot.kt:543)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:273)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:266)
at org.mozilla.fenix.ui.robots.NavigationToolbarRobot$Transition.enterURLAndEnterToBrowser(NavigationToolbarRobot.kt:138)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:266)

status-firefox112: --- → affected

Flaky 1x on 3/6 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/first_name_input]
at androidx.test.uiautomator.UiObject.setText(UiObject.java:666)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:259)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:248)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:231)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:176)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:231)

Flaky 1x on 3/10 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/select_address_header]
at androidx.test.uiautomator.UiObject.clickAndWaitForNewWindow(UiObject.java:456)
at org.mozilla.fenix.ui.robots.BrowserRobot.clickSelectAddressButton(BrowserRobot.kt:543)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:268)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:266)
at org.mozilla.fenix.ui.robots.NavigationToolbarRobot$Transition.enterURLAndEnterToBrowser(NavigationToolbarRobot.kt:140)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:266)

Assignee: nobody → ohorvath

PR open https://github.com/mozilla-mobile/firefox-android/pull/1209
This only fixes a part of the flakiness, there are still bugs affecting the feature, so issues will still appear.
Feel free to reopen the ticket if needed.

Attachment #9322763 - Attachment description: [mozilla-mobile/firefox-android] Bug 1814032 - run 1 (#1209) → [mozilla-mobile/firefox-android] Bug 1814032 - Retry attempts for address autofill in UI tests
Authored by oana.horvath https://github.com/mozilla-mobile/firefox-android/commit/3d9289368568f6eb3dd68506eba177078b59ece4 [main] Bug 1814032 - Retry attempts for address autofill in UI tests
Status: NEW → RESOLVED
Closed: 2 years ago
status-firefox113: affected → fixed
status-firefox113: affected → fixed
Flags: qe-verify+
Resolution: --- → FIXED
Target Milestone: --- → 113 Branch
Flags: qe-verify+

Flaky 1x on 3/17 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/first_name_input]
at androidx.test.uiautomator.UiObject.setText(UiObject.java:666)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:259)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:248)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:231)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:206)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:231)

Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Flaky 1x on 3/23 Main Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/first_name_input]
at androidx.test.uiautomator.UiObject.setText(UiObject.java:666)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:259)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:248)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:231)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:269)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:231)

Closing, as it didn't fail for a month.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
status-firefox113: affected → wontfix
status-firefox114: --- → unaffected
Resolution: --- → WORKSFORME

Flaky 1x on 4/27 Main Fireabase link

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.mozilla.fenix.ui.robots.BrowserRobot.verifyAutofilledAddress(BrowserRobot.kt:509)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:294)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$6.invoke(AddressAutofillTest.kt:275)
at org.mozilla.fenix.ui.robots.NavigationToolbarRobot$Transition.enterURLAndEnterToBrowser(NavigationToolbarRobot.kt:140)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:275)

Status: RESOLVED → REOPENED
status-firefox114: unaffected → affected
Resolution: WORKSFORME → ---

Failed 1x on 4/28
Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[RESOURCE_ID=org.mozilla.fenix.debug:id/first_name_input]
at androidx.test.uiautomator.UiObject.setText(UiObject.java:666)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:259)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:257)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:240)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:270)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:240)

Firebase link

androidx.test.uiautomator.UiObjectNotFoundException: UiSelector[CONTAINS_TEXT=Alaska]
at androidx.test.uiautomator.UiObject.click(UiObject.java:416)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.clickSubRegionOption(SettingsSubMenuAutofillRobot.kt:236)
at org.mozilla.fenix.ui.robots.SettingsSubMenuAutofillRobot.fillAndSaveAddress(SettingsSubMenuAutofillRobot.kt:265)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:243)
at org.mozilla.fenix.ui.AddressAutofillTest$verifyAddressAutofillSelectionTest$4.invoke(AddressAutofillTest.kt:240)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openAutofillSubMenu(SettingsRobot.kt:270)
at org.mozilla.fenix.ui.AddressAutofillTest.verifyAddressAutofillSelectionTest(AddressAutofillTest.kt:240)

Authored by https://github.com/AndiAJ
https://github.com/mozilla-mobile/firefox-android/commit/720d1be26d5b19b962b913524b5ea98fbe032d9b
[main] Bug 1814032 - disable failing verifyAddressAutofillSelectionTest Ui test

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
status-firefox114: affected → fixed
Flags: qe-verify+
Resolution: --- → FIXED
Target Milestone: 113 Branch → 114 Branch
Status: RESOLVED → REOPENED
status-firefox114: fixed → affected
Flags: qe-verify+
Keywords: test-disabled
Resolution: FIXED → ---
Assignee: ohorvath → nobody

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
status-firefox114: affected → wontfix
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
Resolution: --- → INCOMPLETE

Authored by https://github.com/AndiAJ
https://github.com/mozilla-mobile/firefox-android/commit/35507f417020a9d464b41c48d5a6c5397c28ba60
[main] Bug 1814032 - Re-enable verifyMultipleAddressesSelectionTest UI test

status-firefox122: --- → fixed
Resolution: INCOMPLETE → FIXED
Target Milestone: 114 Branch → 122 Branch
status-firefox121: --- → wontfix
status-firefox122: fixed → wontfix
status-firefox123: --- → fixed
Target Milestone: 122 Branch → 123 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: