1932366 - UI tests failures with Android fission enabled

Status: RESOLVED FIXED

(Firefox for Android :: UI Tests, defect)

Description

[Oana Horvath [:ohorvath - out of office]]

Try run with the MOZ_FORCE_ENABLE_FISSION env variable configuration:

https://console.firebase.google.com/u/0/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/5949212631303058815/details?stepId=bs.f563dad67d3f1120&testCaseId=4&tabId=video

Some https-only tests are failing when testing with the "http.badssl.com" test page.
The verifyCalendarFormInteractionsTest test fails to validate the text returned on the page when submitting the selected date.

Comment 1

[Oana Horvath [:ohorvath - out of office]]

Found the cause with the https-only mode tests.

First, to enable Fission:
Modified mobile/android/fenix/automation/taskcluster/androidTest/geckoview-configs/org.mozilla.fenix.debug-geckoview-config.yaml to add
env:
MOZ_FORCE_ENABLE_FISSION: 1
And pushed that file to device to /data/local/tmp

Steps to reproduce manually:

1. Enable the https-only mode in all tabs.
2. Open a website, like http://example.com in the test.
3. In the same tab, go to "http.baddssl.com"
4. Tap the "Go back (Recommended)" button on the bad ssl error page.

Result: Instead of returning directly to the example.com page, the image above the "Secure site not available" error disappears first. A second click is needed to navigate back to the first page.
*This is not reproducible with Fission disabled.

Comment 2

[Oana Horvath [:ohorvath - out of office]]

The calendar test failure seems to be just a timing issue. Replacing the way we assert the page content is gone will make the test pass.

Comment 3

[Aaron Train [:aaronmt]]

I can reproduce the issue. To confirm the URL in step 3 should be https://http.badssl.com

On 4, on Go Back (Recommended), the Go Back action looks like it's tied to something else instead of the expected first URL visit.

See attached screenshot.

Comment 4

[Aaron Train [:aaronmt]]

Attachment: screenshot.png

Comment 5

[Aaron Train [:aaronmt]]

Closing this out.