1668902 - Investigate re-enabling passwordmgr mochitest-plain suite under GeckoView.

Status: NEW

(Toolkit :: Password Manager, task, P2)

Description

[Sam Foster [:sfoster] (he/him)]

In bug 1525959, all the mochitest-plain suite was disabled for gecko-view. We should figure out what it would take to re-enable at least some of those tests, as they provide important coverage for the core functionality that is being shared in this component.

Comment 1

[Sam Foster [:sfoster] (he/him)]

I'm not sure if this belongs in the gecko-view component, or password manager. From the look of the kinds of failures we get when re-enabling these tests[1], it looks like a mixed bag of missing APIs/incompatible test helpers, as well as some passwordmgr test specific details. :esawin, do you have any more details you can add here?

1. https://treeherder.mozilla.org/#/jobs?repo=try&revision=24f7341199001bf61059fce2dab3f5d7a66c57fc

Comment 2

[Eugen Sawin [:esawin]]

There is a variety of failure reasons.

Some are easy to work around, e.g., NS_ERROR_UNEXPECTED for prefs that are just not defined on the default branch for GV.

Others would need adding an Autocomplete test/mock delegate to TestRunnerActivity, e.g., NS_ERROR_NOT_IMPLEMENTED for tests managing logins, since GV delegates storage to the app rather than managing it.

Comment 3

[Bianca Danforth [:bdanforth]]

These are some notes from a conversation I had with MattN a long time ago when I asked about it:

Some of the reasons many of these tests are disabled are:

• Different storage backends between Desktop and Android (storage-json.js, storage-geckoview.js)
  • GeckoView doesn't provide storage, so any tests that go to storage-geckoview.js wouldn't have a storage backend to check.
  • storage-geckoview.js only implements async APIs, so it doesn't implement e.g. findLogins, countLogins, searchLogins etc. This is because of the async messaging to the app using GeckoView.
• At least for browser mochitests, nsILoginManagerPrompter interfaces are different based on platform.
• Also, password generation is not implemented yet in GeckoView
• Autocomplete works a little differently, as the Autocomplete UI on Desktop uses Custom Elements, so we'd need hooks to not go to our AC popup.

Ideally, we would run these tests on Fenix rather than GeckoView, but I don't know that Fenix runs mochitests in general. Also, since so many of our tests involve UI (e.g. autocomplete popups or doorhangers), it's hard to say how much we'd actually be testing.

Comment 4

[Eugen Sawin [:esawin]]

These are some notes from a conversation I had with MattN a long time ago when I asked about it:

Some of the reasons many of these tests are disabled are:

• Different storage backends between Desktop and Android (storage-json.js, storage-geckoview.js)
  • GeckoView doesn't provide storage, so any tests that go to storage-geckoview.js wouldn't have a storage backend to check.
  • storage-geckoview.js only implements async APIs, so it doesn't implement e.g. findLogins, countLogins, searchLogins etc. This is because of the async messaging to the app using GeckoView.
• At least for browser mochitests, nsILoginManagerPrompter interfaces are different based on platform.
• Also, password generation is not implemented yet in GeckoView
• Autocomplete works a little differently, as the Autocomplete UI on Desktop uses Custom Elements, so we'd need hooks to not go to our AC popup.

Ideally, we would run these tests on Fenix rather than GeckoView, but I don't know that Fenix runs mochitests in general. Also, since so many of our tests involve UI (e.g. autocomplete popups or doorhangers), it's hard to say how much we'd actually be testing.

That's a good summary. Let me add some clarifications:

• The GV API does support password generation
• Fenix doesn't run mochitests, it runs Android-specific UI tests via Firebase Test Lab

I would suggest investigating existing mochitests case-by-case to find if enabling the test is reasonable (although I won't have cycles for it before Q1).