1716412 - Tests skipped in 'toolkit/components/passwordmgr/test/mochitest/mochitest.ini' for new fission xorigin platform triage

Status: NEW

(Toolkit :: Password Manager, task, P3)

Description

[Andrew Halberstadt [:ahal]]

Note: This bug was filed automatically via script.

The following tests are being disabled in toolkit/components/passwordmgr/test/mochitest/mochitest.ini due to new fission xorigin platform triage:

• test_autocomplete_basic_form_insecure.html
• test_autocomplete_autofill_related_realms_no_dupes.html
• test_autocomplete_https_downgrade.html
• test_autocomplete_basic_form.html
• test_autocomplete_basic_form_related_realms.html
• test_prompt_http.html

Full diff:

diff --git a/toolkit/components/passwordmgr/test/mochitest/mochitest.ini b/toolkit/components/passwordmgr/test/mochitest/mochitest.ini
--- a/toolkit/components/passwordmgr/test/mochitest/mochitest.ini
+++ b/toolkit/components/passwordmgr/test/mochitest/mochitest.ini
@@ -36,26 +36,37 @@ support-files =
   pwmgr_common.js
   pwmgr_common_parent.js
   ../authenticate.sjs
 skip-if = toolkit == 'android' && !is_fennec # Don't run on GeckoView
 
 # Note: new tests should use scheme = https unless they have a specific reason not to
 
 [test_autocomplete_autofill_related_realms_no_dupes.html]
+skip-if =
+  fission && xorigin  # {bug} - New fission platform triage
 scheme = https
 [test_autocomplete_basic_form.html]
-skip-if = toolkit == 'android' || debug && (os == 'linux' || os == 'win') || os == 'linux' && tsan # android:autocomplete. Bug 1541945, Bug 1590928
+skip-if =
+  toolkit == 'android'  # autocomplete
+  debug && (os == 'linux' || os == 'win')  # Bug 1541945
+  os == 'linux' && tsan  # Bug 1590928
+  fission && xorigin && (!debug || os == "mac")  # {bug} - New fission platform triage
 scheme = https
 [test_autocomplete_basic_form_insecure.html]
-skip-if = toolkit == 'android' || os == 'linux' # android:autocomplete., linux: bug 1325778
+skip-if =
+  toolkit == 'android'  # autocomplete
+  os == 'linux' # bug 1325778
+  fission && xorigin && (os == "win" || os == "mac")  # {bug} - New fission platform triage
 [test_autocomplete_basic_form_formActionOrigin.html]
 skip-if = toolkit == 'android' # android:autocomplete.
 scheme = https
 [test_autocomplete_basic_form_related_realms.html]
+skip-if =
+  fission && xorigin  # {bug} - New fission platform triage
 scheme = https
 [test_autocomplete_basic_form_subdomain.html]
 skip-if = toolkit == 'android' # android:autocomplete.
 scheme = https
 [test_autocomplete_hasBeenTypePassword.html]
 scheme = https
 skip-if = toolkit == 'android' # autocomplete
 [test_autocomplete_highlight.html]
@@ -64,17 +75,20 @@ skip-if = toolkit == 'android' # autocom
 [test_autocomplete_highlight_non_login.html]
 scheme = https
 skip-if = toolkit == 'android' # autocomplete
 [test_autocomplete_highlight_username_only_form.html]
 scheme = https
 skip-if = toolkit == 'android' # autocomplete
 [test_autocomplete_https_downgrade.html]
 scheme = http # Tests downgrading
-skip-if = toolkit == 'android'  || (os == 'linux' && debug) # autocomplete && Bug 1554959 for linux debug disable
+skip-if =
+  toolkit == 'android'  # autocomplete
+  os == 'linux' && debug  # Bug 1554959
+  fission && xorigin  # {bug} - New fission platform triage
 [test_autocomplete_https_upgrade.html]
 scheme = https
 skip-if = verify || toolkit == 'android'  || (os == 'linux' && debug) # autocomplete && Bug 1554959 for linux debug disable
 [test_autocomplete_password_generation.html]
 scheme = https
 skip-if = xorigin || toolkit == 'android' # autocomplete
 [test_autocomplete_password_generation_confirm.html]
 scheme = https
@@ -191,17 +205,20 @@ skip-if = toolkit == 'android' # bug 153
 [test_onsubmit_value_change.html]
 [test_passwords_in_type_password.html]
 [test_prompt.html]
 skip-if = os == "linux" || toolkit == 'android' # Tests desktop prompts
 [test_prompt_async.html]
 skip-if = toolkit == 'android' || (debug || asan || tsan) # Tests desktop prompts # Bug 1603659
 support-files = subtst_prompt_async.html
 [test_prompt_http.html]
-skip-if = os == "linux" || toolkit == 'android' # Tests desktop prompts
+skip-if =
+  toolkit == 'android'  # Tests desktop prompts
+  os == "linux"
+  fission && xorigin  # {bug} - New fission platform triage
 [test_prompt_noWindow.html]
 skip-if = toolkit == 'android' # Tests desktop prompts.
 [test_password_length.html]
 scheme = https
 skip-if = toolkit == 'android' # bug 1527403
 [test_prompt_promptAuth.html]
 skip-if = os == "linux" || toolkit == 'android' # Tests desktop prompts
 [test_prompt_promptAuth_proxy.html]

See this try push for failures. If failures are missing, they were
either discovered on a subsequent try push or this bug is invalid.

To run these failures in your own try push, first ensure the patches from bug 1700781
have landed, revert the skip-if annotations, then run:

$ ./mach try fuzzy --rebuild 3 toolkit/components/passwordmgr/test/mochitest/mochitest.ini

Finally use the fuzzy interface to select the task(s) which are relevant to the
removed skip-if expression(s).

Disclaimer

Adding new platforms is not an exact science, and in order to get to green and
enable coverage ASAP, we often err on the side of disabling when in doubt. For
this reason, it's possible that the annotation was added in error, is covered
by an existing intermittent, or was fixed sometime between now and when the
annotation was made.

If you believe this is the case here, please feel free to remove the
annotation. Sorry for the inconvenience and thanks for understanding.

Comment 1

[Chris Peterson [:cpeterson]]

@ Sam, here are some Password Manager test failures with Fission (in xorigin mode). Do you think fixing these test failures should block shipping Fission MVP? Looks like all the failures are variations of the same issue.

test_autocomplete_https_downgrade.html

@ Neil, this Fission+xorigin test failure looks like xorigin bug 1697771 fixed a few months ago. Any theories why it might have regressed?

TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/mochitest/test_autocomplete_https_downgrade.html | Checking form-basic username is: name1 - "" == "name1" - got "", expected "name1" (operator ==)

test_autocomplete_basic_form_insecure.html

test_autocomplete_basic_form.html
test_autocomplete_basic_form_related_realms.html

All these test failures look like some variation of Checking form2 password is: singlepass5 - got "", expected "singlepass5":

18:48:32 INFO - TEST-UNEXPECTED-FAIL | docshell/test/navigation/test_opener.html | Test timed out. -
7710	18:56:49 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/mochitest/test_autocomplete_basic_form_insecure.html | Wait for password manager to fill form
7714	18:56:49 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/mochitest/test_autocomplete_basic_form_insecure.html | Checking form2 username is: singleuser5 - got "", expected "singleuser5"
7719	18:56:49 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/mochitest/test_autocomplete_basic_form_insecure.html | Checking form2 password is: singlepass5 - got "", expected "singlepass5"

test_autocomplete_autofill_related_realms_no_dupes.html

This failure might be a duplicate of bug 1708578:

TEST-UNEXPECTED-FAIL | toolkit/components/passwordmgr/test/mochitest/test_autocomplete_autofill_related_realms_no_dupes.html | Checking form1 username is: exactUser1 - got "", expected "exactUser1"

Comment 2

[Chris Peterson [:cpeterson]]

Based on Tim's and Ethan's comments in Fission Password Manager meta bug 1711988, this test failures probably don't need to block shipping Fission MVP. So I will set Fission Milestone to "Future".

Comment 3

[Neil Deakin]

I filed bug 1717103 for a fix for test_autocomplete_https_downgrade.html

Comment 4

[Sam Foster [:sfoster] (he/him) out until 2024-05-26]

Cancelling the need-info per comment 2 - sorry, I was out on PTO.