1661935 - Support NativeMessaging in Snap

Status: ASSIGNED

(WebExtensions :: General, defect, P3)

Description

[eduardo.rojasr]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0

Steps to reproduce:

Steps to reproduce:

Use the Firefox Snap build from Ubuntu Snap Store (Ubuntu software).

Install "chrome-gnome-shell" package.

Install Firefox GNOME Shell integration.

Go to "extensions.gnome.org" and choose any extension.

Actual results:

Actual results:

An error report shows that says the native host connector cannot be detected.

There is no option to install the GNOME extension.

Expected results:

Expected results:

I should be able to install the chosen GNOME extension directly from the website and manage it afterwards.

The behavior is correct on the current RPM/DEB builds.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Olivier Tilloy]

The problem is being discussed at https://forum.snapcraft.io/t/native-messaging-support-in-strictly-confined-browser-snaps/26849.

Comment 3

[David D Lowe]

This bug is also being tracked at https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/1741074

Ubuntu 21.10 ships the snap version of Firefox by default, (instead of the APT version), so we can expect more users to experience this bug.

Comment 4

[Olivier Tilloy]

Flatpak counterpart: bug 1621763.

Comment 5

[Darkspirit]

S2 (Serious) Major functionality/product severely impaired and a satisfactory workaround does not exist

Comment 6

[Olivier Tilloy]

Proposal for a new NativeMessaging portal to address this issue: https://github.com/flatpak/xdg-desktop-portal/issues/655.

Comment 8

[David D Lowe]

This bug report is getting triaged, I'm glad. I'll add that Ubuntu 22.04 LTS is planning on removing Firefox from the APT repositories altogether and to only provide the snap version of Firefox, which is Ubuntu's default browser, so this bug is going to be experienced by a large fraction of the Ubuntu userbase at some point.

Comment 9

[Olivier Tilloy]

Attachment: Bug 1661935 - Integration with a new WebExtensions XDG desktop portal for native messaging on Linux r=robwu

Comment 10

[William Woodruff]

Chiming in as a maintainer of an extension (ff2mpv) that uses native messaging: users have already reported challenges getting the extension to function correctly with the snap-ified Firefox distribution in the upcoming Ubuntu LTS release.

Downstream tracking: https://github.com/woodruffw/ff2mpv/issues/80

Comment 13

[Tommy Trussell]

I'm entering a comment here because I've not seen a tracker for the snap releases having native messaging enabled.

I've been using native messaging in FireFox to integrate with KeePassXC for several weeks now, through several FireFox beta releases (currently "beta" snap channel version 105.0b5-1). It is working as expected.

The only issue I noticed so far is each FireFox Profile needs its own separate connector to the KeePassXC database. This makes sense, but might warrant some documentation, somewhere.

Comment 14

[William Woodruff]

(In reply to William Woodruff from comment #10)

Chiming in as a maintainer of an extension (ff2mpv) that uses native messaging: users have already reported challenges getting the extension to function correctly with the snap-ified Firefox distribution in the upcoming Ubuntu LTS release.

Downstream tracking: https://github.com/woodruffw/ff2mpv/issues/80

Following up on this: I'm able to confirm that the extension I maintain works correctly on the "beta" channel of the Firefox snap.

However, to get it working, I had to run a manual flatpak command on the terminal:

flatpak permission-set webextensions ff2mpv snap.firefox yes

Is this documented somewhere? I had to dig through others' bug reports to figure out that this is what I needed, and I can imagine that a lot of other native extension users (and developers) are in a similar position.

Comment 15

[Olivier Tilloy]

As explained in the ff2mpv issue, desktop shells (such as GNOME Shell) and portal frontends (such as xdg-desktop-portal-gtk and xdg-desktop-portal-kde) should display a modal prompt the first time an extension requires this permission, and so you wouldn't need to use the flatpak command to make this choice "by hand". However minimal window managers without portal frontends (e.g. i3) are unlikely to implement that prompt, and the answer will default to "no".

Comment 16

[Olivier Tilloy]

The snap now supports native messaging through the use of the WebExtensions portal, which is available as a distro-patch in Ubuntu 22.04 and 22.10.

I am not closing this bug yet, because even though the patch has been cherry-picked in the snap, it hasn't landed in the tree yet.

Comment 17

[:gerard-majax]

Olivier, are you still able to actively work on that ?

Comment 19

[Rob Wu [:robwu]]

Olivier has left Canonical and the patch has been taken over by :bandali. I'm therefore re-assigning the bug for clarity.

Also, considering that the majority of the work is in extension code, I'm moving the bug to WebExtensions.

Comment 20

[renosifana.paksi]

(In reply to David D Lowe from comment #3)

This bug is also being tracked at https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/1741074

Ubuntu 21.10 ships the snap version of Firefox by default, (instead of the APT version), so we can expect more users to experience this bug.

I have same problem. Few days ago i have installed Kubuntu 23.10 "Mantic Minotaur" replacing Linux Mint 21.2 "Victoria". the extension doesn't work as well (Including KDE Browser Integration, a Download Manager (Persepolis Download Manager Integration, Aria-NG Integration)). I have grant Permission but nothing happen lol.

Comment 21

[renosifana.paksi]

Attachment: Screenshot_20231123_073703.png

KDE Plasma Integration on Firefox 120.

Comment 22

[david]

Same probleme here with ubuntu 24.04.1, and firefox snap.

"Failed to connect to the native host."

Comment 23

[Amin Bandali [:bandali]]

(In reply to david from comment #22)

Same probleme here with ubuntu 24.04.1, and firefox snap.

"Failed to connect to the native host."

Hello, could you please provide more information about this?

• Is this a fresh Ubuntu 24.04 LTS installation? Or did you upgrade from an existing Ubuntu release to this one? (If yes, from which version?)
• Please open a terminal, run the following commands, and provide the output here:

apt list --installed | grep xdg-desktop-portal
apt-cache policy xdg-desktop-portal
snap version

• Which Firefox version are you using?
• Which Firefox add-on are you trying to use here? And did you add a manifest for it under the ~/.mozilla/native-messaging-hosts/ directory?
• Are there any more error or warning messages in the logs beyond the one line you provided?

Thanks.

Comment 24

[david]

(In reply to Amin Bandali [:bandali] from comment #23)

(In reply to david from comment #22)

Same probleme here with ubuntu 24.04.1, and firefox snap.

"Failed to connect to the native host."

Hello, could you please provide more information about this?

• Is this a fresh Ubuntu 24.04 LTS installation? Or did you upgrade from an existing Ubuntu release to this one? (If yes, from which version?)
• Please open a terminal, run the following commands, and provide the output here:

apt list --installed | grep xdg-desktop-portal
apt-cache policy xdg-desktop-portal
snap version

• Which Firefox version are you using?
• Which Firefox add-on are you trying to use here? And did you add a manifest for it under the ~/.mozilla/native-messaging-hosts/ directory?
• Are there any more error or warning messages in the logs beyond the one line you provided?

Thanks.

Hi,

• This is not a fresh ubuntu install (its a upgraded version from ubuntu 22.04 LTS), BUT, I'm using a fresh account to test (new user, new /home).
• This is the output of the commands:

apt list --installed | grep xdg-desktop-portal

xdg-desktop-portal-gtk/noble,now 1.15.1-1build2 amd64 [installed,automatic]
xdg-desktop-portal-kde/noble,now 5.27.11-0ubuntu3 amd64 [installed,automatic]
xdg-desktop-portal/noble-updates,now 1.18.4-1ubuntu2 amd64 [installed,automatic]

apt-cache policy xdg-desktop-portal

xdg-desktop-portal:
  Installed: 1.18.4-1ubuntu2
  Candidate: 1.18.4-1ubuntu2
  Version table:
 *** 1.18.4-1ubuntu2 500
        500 http://fr.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.18.3-1ubuntu1 500
        500 http://fr.archive.ubuntu.com/ubuntu noble/main amd64 Packages

snap version

snap    2.63.1+24.04
snapd   2.63.1+24.04
series  16
ubuntu  24.04
kernel  6.8.0-41-generic

• Firefox version is 130.0 (64-bit)

• The firefox addon I'm trying to use is: https://addons.mozilla.org/en-US/firefox/addon/plasma-integration/. I did not add a manifest for it under ~/.mozilla/native-messaging-hosts/ directory.

• On the extension console, I can see the messages:

  Invalid url undefined action_popup.js:86:33
  Failed to check for whether media controls are blocked NO_ORIGINS action_popup.js:279:17

and in about:debugging#/runtime/this-firefox i see:

Warning details
Reading manifest: Warning processing key: An unexpected property was found in the WebExtension manifest.

if I try to reload the addon I get:

Host disconnected An unexpected error occurred extension.js:204:17
Not auto-restarting host as we haven't received any message from it before. Check that it's working/installed correctly extension.js:226:21
Background event page was not terminated on idle because a DevTools toolbox is attached to the extension. _generated_background_page.html

Comment 25

[:gerard-majax]

$ pip install --no-use-pep517 python-dbusmock==0.32.2
Collecting python-dbusmock==0.32.2
  Using cached python-dbusmock-0.32.2.tar.gz (104 kB)
ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml

Comment 26

[:gerard-majax]

Kagami in https://bugzilla.mozilla.org/show_bug.cgi?id=1868690#c10 you mentionned --no-use-pep517 as temporary? Can we remove it?

Comment 27

[:gerard-majax]

(venv-python-dbusmock) alex@portable-alex:~/tmp/mozillasupport$ pip install --upgrade setuptools==70.0.0 wheel==0.43.0
Collecting setuptools==70.0.0
  Using cached setuptools-70.0.0-py3-none-any.whl.metadata (5.9 kB)
Collecting wheel==0.43.0
  Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Using cached setuptools-70.0.0-py3-none-any.whl (863 kB)
Using cached wheel-0.43.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel, setuptools
  Attempting uninstall: wheel
    Found existing installation: wheel 0.44.0
    Uninstalling wheel-0.44.0:
      Successfully uninstalled wheel-0.44.0
  Attempting uninstall: setuptools
    Found existing installation: setuptools 75.2.0
    Uninstalling setuptools-75.2.0:
      Successfully uninstalled setuptools-75.2.0
Successfully installed setuptools-70.0.0 wheel-0.43.0
(venv-python-dbusmock) alex@portable-alex:~/tmp/mozillasupport$ pip install python-dbusmock==0.32.2
Collecting python-dbusmock==0.32.2
  Using cached python-dbusmock-0.32.2.tar.gz (104 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: dbus-python in ./venv-python-dbusmock/lib/python3.12/site-packages (from python-dbusmock==0.32.2) (1.3.2)
Building wheels for collected packages: python-dbusmock
  Building wheel for python-dbusmock (pyproject.toml) ... done
  Created wheel for python-dbusmock: filename=python_dbusmock-0.32.2-py3-none-any.whl size=73663 sha256=838a8a57b4240b19003b00188a1e63c90f6b4f5f19d0ca8e7c0803ffec0dcc95
  Stored in directory: /home/alex/.cache/pip/wheels/fd/95/73/9cc598f34c15d81a7a4b400328064875b72d637c10ab236ae7
Successfully built python-dbusmock
Installing collected packages: python-dbusmock
Successfully installed python-dbusmock-0.32.2

Not sure why it fails on CI: https://treeherder.mozilla.org/logviewer?job_id=478870258&repo=try

Comment 28

[Kagami Rosylight [:saschanaz] (they/them)]

(In reply to :gerard-majax from comment #26)

Kagami in https://bugzilla.mozilla.org/show_bug.cgi?id=1868690#c10 you mentionned --no-use-pep517 as temporary? Can we remove it?

That would require https://bugzilla.mozilla.org/show_bug.cgi?id=1868690#c3 option 2. I don't know whether anyone did it, maybe ahal knows.

Comment 29

[Andrew Halberstadt [:ahal]]

No work has been done or planned here afaik,

But yeah, if I'm understanding the situation, --no-use-pep517 is just a bandaid to allow our outdated infrastructure to chug along a little while longer. But at some point someone is going to have to do the actual work of updating our tooling to modern Python standards.

Comment 30

[:gerard-majax]

(In reply to Andrew Halberstadt [:ahal] from comment #29)

No work has been done or planned here afaik,

But yeah, if I'm understanding the situation, --no-use-pep517 is just a bandaid to allow our outdated infrastructure to chug along a little while longer. But at some point someone is going to have to do the actual work of updating our tooling to modern Python standards.

Thanks. Does it means we cannot install this package until that work is done?

Comment 31

[Andrew Halberstadt [:ahal]]

I think so.. I don't remember why we added --no-use-pep517 in the first place, it looks like it might have been to improve performance? If so we could alternatively back that patch out.

Comment 32

[Kagami Rosylight [:saschanaz] (they/them)]

Wasn't --no-use-pep517 to skip building wheel? It still builds wheel here and then fails, so not sure why the flag is relevant here.

Comment 33

[:gerard-majax]

So, how should such a package installed ?

Comment 34

[Kagami Rosylight [:saschanaz] (they/them)]

All I can say is that pip install --timeout 120 --no-index --find-links https://pypi.pub.build.mozilla.org/pub/ python-dbusmock==0.32.2 (the command from the CI log in comment #27) fails on my local machine too with the same error. Not sure why it behaves differently from pip install python-dbusmock==0.32.2.

Comment 35

[:gerard-majax]

interesting, it also fails for me, that's not something i would have expected

Comment 36

[:gerard-majax]

When it works i can see:

  Using cached setuptools-75.2.0-py3-none-any.whl (1.2 MB)
  Using cached setuptools_scm-8.1.0-py3-none-any.whl (43 kB)
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
  Installing collected packages: setuptools, packaging, setuptools-scm
  Successfully installed packaging-24.1 setuptools-75.2.0 setuptools-scm-8.1.0

When it does not we install other versions:

  Looking in links: https://pypi.pub.build.mozilla.org/pub/
  Collecting setuptools>=45
    Using cached https://pypi.pub.build.mozilla.org/pub/setuptools-69.0.3-py3-none-any.whl (819 kB)
  Collecting setuptools-scm
    Using cached https://pypi.pub.build.mozilla.org/pub/setuptools_scm-3.3.3-py2.py3-none-any.whl (23 kB)
  Installing collected packages: setuptools-scm, setuptools
  Successfully installed setuptools-69.0.3 setuptools-scm-3.3.3
  Installing build dependencies: finished with status 'done'

So maybe the setuptools version we have on our Pypi mirror is not recent enough? Or it's setuptools-scm?

Comment 37

[:gerard-majax]

Making a local dumb pypi mirror i could confirm:

• unable to install with setuptools-69.0.3 setuptools-scm-3.3.3
• able to install with packaging-23.1 setuptools-75.2.0 setuptools-scm-8.1.0

Maybe we could have the wheel instead of the tar of dbusmock a wheel on our pypi mirror?

Comment 38

[:gerard-majax]

Thanks to both :ahal and :saschanaz this seems to work now on this push https://treeherder.mozilla.org/jobs?repo=try&author=alissy%40mozilla.com&selectedTaskRun=LwPGCLVrQuyRDIDVvuJHVg.0

Comment 39

[:gerard-majax]

Attachment: Bug 1661935 - Native Messaging WebExtension documentation r?robwu!

Comment 40

[:gerard-majax]

Attachment: Bug 1661935 - Native Messaging WebExtensions portal tests r?robwu!

Comment 41

[:gerard-majax]

Attachment: GitHub Pull Request

Upstream removal of the patch PR against nightly

Comment 43

[:gerard-majax]

Attachment: GitHub Pull Request

Correct PR to remove the patch on nightly branch

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/594beeb5e424
Integration with a new WebExtensions XDG desktop portal for native messaging on Linux r=emilio,robwu,rpl,Gijs,andi,mach-reviewers
https://hg.mozilla.org/integration/autoland/rev/7b10b4e274db
Native Messaging WebExtension documentation r=robwu,rpl
https://hg.mozilla.org/integration/autoland/rev/620790051502
Native Messaging WebExtensions portal tests r=robwu,ahal

Comment 45

[agoloman]

Backed out for causing xpcshell failures @@CycleCollectedJSContext.

• Backout link
• Push with failures
• Failure Log

Comment 46

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/59ce3da9e504
Backed out 3 changesets for causing xpc failures @test_subprocess.js.

Comment 47

[:gerard-majax]

(In reply to Pulsebot from comment #46)

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/59ce3da9e504
Backed out 3 changesets for causing xpc failures @test_subprocess.js.

So it looks like we are facing some bad case of a race condition here. We create running spawning a process and then we connectRunning with its FDs and get a second proc process. There running.stdout is id: 1 and had fd: 16 when proc.stdin is id: 5 with fd: 16 which seems to be consistent with the expected behavior of those pipes. It means we have two InputPipe that shares the same FD. Upon creation they fillBuffer https://searchfox.org/mozilla-central/rev/b1cd7e191ac57f8dfe31806be2539e51843c03eb/toolkit/modules/subprocess/subprocess_common.sys.mjs#295. So both enqueues a read  call. Somehow the pollfd.revents https://searchfox.org/mozilla-central/rev/b1cd7e191ac57f8dfe31806be2539e51843c03eb/toolkit/modules/subprocess/subprocess_unix.worker.js#569 turns != 0for the InputPipe of id:5 before the id:1 and so we trigger reading the content of the pipe, which is then not available anymore to id:1.

Comment 48

[:gerard-majax]

Attachment: Bug 1661935 - Introduce ManagedProcess and connectRunning()

Comment 49

[:gerard-majax]

Attachment: Bug 1661935 - Properly close process from connectRunning() r?mossop!

Comment 50

[:gerard-majax]

Attachment: Bug 1661935 - Introduce Subprocess.connectRunning() and ManagedProcess r?robwu!