1928096 - Add end-to-end test coverage for native messaging in snap

Status: RESOLVED FIXED

(WebExtensions :: General, task, P3)

Description

[Rob Wu [:robwu]]

Bug 1661935 adds the Firefox client side of the native messaging implementation, and adds unit tests that mock the D-Bus interface.

The D-Bus interface of the portal is provided in https://github.com/flatpak/xdg-desktop-portal/pull/705 , which also includes some limited unit tests.

There is however no end-to-end test coverage that verifies that the native messaging system as a whole is working as intended. This lack of test coverage incurs the risk of regressions reaching users on release, e.g. as seen in bug 1876447.

We should therefore add a test that consists of setting up native messaging manifests + binaries on the host system, and verify that Firefox can launch applications through this binary. There are other scenarios of interest at https://searchfox.org/mozilla-central/source/toolkit/components/extensions/test/xpcshell/test_ext_native_messaging.js

Additionally, unique to the snap setup, we should verify that Firefox is unable to write to the "real" filesystem (outside the portal), because the ability to do so would indicate a bug in portal. For more details on this, see https://github.com/flatpak/xdg-desktop-portal/pull/705#issuecomment-2262776394

Comment 1

[Nathan Teodosio :nteodosio]

I'm working on this and got a script for the test. As that extensions cannot be installed programatically in Firefox, it manually interacts with the Firefox GUI to install and execute the extension.

I'm only having a hard time setting up the environment for the test, as it would require the XDG desktop portal, Firefox and arguably Gnome running. I'll keep on that, but let me know if there is a preferred way (or prohibited ways) to go about this.

Comment 2

[Rob Wu [:robwu]]

Feel free to get in touch with me, whether by email, or via a private chat on Matrix (robwu:mozilla.org at chat.mozilla.org) if you need more context.

I'm only having a hard time setting up the environment for the test, as it would require the XDG desktop portal, Firefox and arguably Gnome running.

I wonder why Gnome would be needed. It sounds like the main requirements are XDG desktop portal + Firefox with snap native messaging enabled.
A true end-to-end test could test with the components already installed on the system, most importantly the XDG desktop portal (which is currently having a custom patch in the Ubuntu builds). For verification of new changes, it would be useful if the test can test with versioned/custom versions of XDG desktop portal, since it is not clear whether XDG desktop portal has already converged towards a specific implementation.

(I thought for a while that the XDG portal patch was final, but then people started forking the original PR and starting discussions on the directions, so as of now I am unsure - see https://github.com/flatpak/xdg-desktop-portal/pull/1537#issuecomment-2588197856)

Comment 3

[Nathan Teodosio :nteodosio]

Attachment: Bug 1928096 - Native messaging test with xdg-desktop-portal. r=robwu

Fails, log contains:

0:04.52 INFO "CONSOLE_MESSAGE: (error) [JavaScript Error: "[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: resource://gre/modules/NativeManifests.sys.mjs :: init :: line 41" data: no]"]
i

Comment 4

[Nathan Teodosio :nteodosio]

Attachment: Bug 1928096 - Native messaging test with xdg-desktop-portal. r?robwu

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/ddc1fb735721
https://hg.mozilla.org/integration/autoland/rev/89d3aa769ad4
Native messaging test with xdg-desktop-portal. r=robwu

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/99704b27a76e
https://hg.mozilla.org/integration/autoland/rev/5536d0de9633
Revert "Bug 1928096 - Native messaging test with xdg-desktop-portal. r=robwu" for causing xpcshell failures @ test_ext_native_messaging_portal_real.js

Comment 7

[amarc]

Backed out for causing xpcshell failures @ test_ext_native_messaging_portal_real.js

• Backout link
• Push with failures
• Failure Log

Comment 8

[Nathan Teodosio :nteodosio]

What Ubuntu is that test running on? The portal in 18.04 doesn't have WebExtensions so a failure would be expected.

Comment 9

[Rob Wu [:robwu]]

(In reply to Nathan Teodosio :nteodosio from comment #8)

What Ubuntu is that test running on? The portal in 18.04 doesn't have WebExtensions so a failure would be expected.

The "push with failures" link above shows that tests were scheduled for 18.04 and (after unhiding passed jobs) 24.04. If tests are not passing on 18.04 you could add a skip-if entry listing os == 'linux' && os_version == '18.04'.

Comment 10

[Nathan Teodosio :nteodosio]

Attachment: Bug 1928096 - Native messaging test with xdg-desktop-portal. r=robwu

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/e45b63e1062f
https://hg.mozilla.org/integration/autoland/rev/8fb4f2f9abf7
Native messaging test with xdg-desktop-portal. r=robwu

Comment 12

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/8fb4f2f9abf7