1946405 - Segmentation fault in GTKRemoteServer if DBus disabled and no display server running

Status: RESOLVED FIXED

(Firefox :: Headless, defect)

Description

[Robert Kirkman]

Attachment: This patch fixes the problem for me

1. Compile and install Firefox on any Linux distro, but add the ac_add_options --disable-dbus and ac_add_options --disable-necko-wifi lines to the mozconfig file
2. Shut down all running display servers, if applicable. For example, sudo systemctl stop sddm
3. Back up and delete the current user's Firefox profiles using mv ~/.mozilla ~/.mozilla-bak
4. Run the command firefox -headless

Comment 1

[Robert Kirkman]

Backtrace:

Thread 1 "firefox" received signal SIGSEGV, Segmentation fault.
0x00007ffff4fa9494 in XInternAtoms () from /usr/lib/libX11.so.6
(gdb) bt
#0  0x00007ffff4fa9494 in XInternAtoms () from /usr/lib/libX11.so.6
#1  0x00007fffef4c8303 in nsXRemoteServer::EnsureAtoms (this=0x7fffd92d15e0)
    at /usr/src/debug/firefox/firefox-135.0/toolkit/components/remote/nsXRemoteServer.cpp:154
#2  nsXRemoteServer::XRemoteBaseStartup (this=0x7fffd92d15e0, aAppName=0x7ffff7703708 "firefox", 
    aProfileName=0x7ffff7743488 "/home/tacokoneko/.mozilla/firefox/g3cc47es.default-release")
    at /usr/src/debug/firefox/firefox-135.0/toolkit/components/remote/nsXRemoteServer.cpp:60
#3  0x00007fffef4c4213 in nsGTKRemoteServer::Startup (this=0x7fffd92d15e0, aAppName=0x7ffff7703708 "firefox", 
    aProfileName=0x7ffff7743488 "/home/tacokoneko/.mozilla/firefox/g3cc47es.default-release")
    at /usr/src/debug/firefox/firefox-135.0/toolkit/components/remote/nsGTKRemoteServer.cpp:39
#4  0x00007fffef4c51fa in nsRemoteService::StartupServer (this=0x7ffff77a70a0)
    at /usr/src/debug/firefox/firefox-135.0/toolkit/components/remote/nsRemoteService.cpp:278
#5  0x00007fffef6f2434 in XREMain::XRE_mainRun (this=0x7fffffffd6c0)
    at /usr/src/debug/firefox/firefox-135.0/toolkit/xre/nsAppRunner.cpp:5771
#6  0x00007fffef6f2dba in XREMain::XRE_main (this=0x7fffffffd6c0, argc=2, argv=0x7fffffffea08, aConfig=...)
    at /usr/src/debug/firefox/firefox-135.0/toolkit/xre/nsAppRunner.cpp:6075
#7  0x00007fffef6f333c in XRE_main (argc=2, argv=0x7fffffffea08, aConfig=...)
    at /usr/src/debug/firefox/firefox-135.0/toolkit/xre/nsAppRunner.cpp:6148
#8  0x000055555557d281 in do_main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>)
    at /usr/src/debug/firefox/firefox-135.0/browser/app/nsBrowserApp.cpp:232
#9  main (argc=2, argv=0x7fffffffea08, envp=<optimized out>)
    at /usr/src/debug/firefox/firefox-135.0/browser/app/nsBrowserApp.cpp:464
(gdb) 

Comment 2

[Rares Doghi, Desktop QA]

Hi @Myk can you take a look at this issue ? Im not sure who to tag here for the Firefox Headless component.

Comment 3

[Myk Melez [:myk] [@mykmelez]]

Hi Rares, I haven't worked on this component for a long time (6+ years), so I'm not sure who should take a look at it either, but I see that whimboo fixed a headless issue in https://bugzilla.mozilla.org/show_bug.cgi?id=1936952 a few months ago, so perhaps he would know.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2] (away July 24th - August 16th)]

It was a very specific crash that my patch fixed. This one seems to be more broad and I don't now that much about GTK.

Karl, is it something you may be able to help with?

Comment 5

[Karl Tomlinson (:karlt)]

The -no-remote commandline option would workaround this, I assume.
I guess something should be checking whether a display is actually available before trying to use it.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2] (away July 24th - August 16th)]

(In reply to Karl Tomlinson (:karlt) from comment #5)

The -no-remote commandline option would workaround this, I assume.

Please note that the -no-remote command line option was removed in Firefox 131 via bug 1906260.

Comment 7

[Robert Kirkman]

I guess something should be checking whether a display is actually available before trying to use it.

Thank you for the response,

The patch I attached above seems to check whether a display is actually available successfully for me by using the mozilla::widget::GdkIsX11Display() function, and I have been using it to prevent the crash in headless, DBus-less mode, while still allowing the same build of Firefox, without having to recompile, to work normally after switching from headless mode to an X11 session, including preserving the ability to launch multiple Firefox tabs from the command line within the X11 session, using the GTK remote server.

(for example, firefox google.com & sleep 5 && firefox mozilla.org continues to work once the same build, using this patch and these mozconfig settings, is launched in X11 mode instead of headless mode, without ever encountering the message "firefox is already running but is not responding", which would happen if the GTK remote server and Dbus were both completely disabled at build-time)

However, because of the large scale of the Firefox project and this being my first issue opened, I am not completely confident that my patch is the absolute best way to fix the problem, just that I have been testing my patch with the situations I described so far and that I have not noticed any problems with it. It is possible that I could be overlooking and failing to account for other unknown edge cases in this area of the Firefox code.

Comment 8

[Karl Tomlinson (:karlt)]

Sorry, I somehow missed that you'd already posted a patch, thank you.
That works, and we'd no longer need the mozilla::widget::GdkIsWaylandDisplay() path above.

Using mozilla::DefaultXDisplay() instead would be more consistent with nsXRemoteServer::EnsureAtoms() and safer against a modification to the behavior of DefaultXDisplay().

If you are able to submit a patch to Phabricator, then I can review.
An automated test is not required because this is not built into the shipped product.

Comment 9

[Robert Kirkman]

Attachment: Bug 1946405 - Don't start nsGTKRemoteServer unless X11 server is present r=karlt

Comment 10

[Robert Kirkman]

Great, I have tested both of those changes (removing the call to mozilla::widget::GdkIsWaylandDisplay() and replacing the call to mozilla::widget::GdkIsX11Display() with a call to mozilla::DefaultXDisplay() instead), and it still seems to fix the problem adequately, as far as I can tell.
I have submitted the patch to Phabricator.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2f2962318683
Don't start nsGTKRemoteServer unless X11 server is present r=karlt

Comment 12

[amarc]

https://hg.mozilla.org/mozilla-central/rev/2f2962318683