2067776 - D-Bus remoting never finds the running instance when the profile path contains a symlink (server registers realpath, client asks for the literal path)

Status: UNCONFIRMED

(Toolkit :: Startup and Profile System, defect)

Description

[Sergii Grechko]

Patch attached (also at github.com/silencespeakstruth/firefox-remoting-symlink-fix); happy to resubmit via Phabricator with a bug number.

Summary

D-Bus remoting never finds the running instance when the profile path contains a
symlink and the running instance was started with --profile <path>: the server
registers its bus name from the realpath'd profile path, while a plain-launched
client computes its candidate names from the literal (non-canonicalized) path.
The client then falls through to a cold start against the locked profile and
hangs on the "already running" flow instead of handing the URL over.

Steps to reproduce

1. Have the profile directory reachable through a symlink, e.g.
   ~/.config/mozilla -> /data/user/.config/mozilla (any offloaded-to-another-
   filesystem layout; increasingly common on btrfs-root systems that keep heavy
   data on a separate disk).
2. Start Firefox with an explicit profile path through the symlinked location:
   firefox --profile ~/.config/mozilla/firefox/xxxxxxxx.default-release
3. From a terminal, run firefox some-file.pdf (or click a link in another app
   that execs plain firefox <url>).

Actual results

The second invocation never reaches the running instance. dbus-monitor shows it
trying exactly two destination names, both unowned, then giving up and starting
up as a new instance against the locked profile:

GetNameOwner "org.mozilla.firefox.L2hvbWUvc2lsZW5jZXNwZWFrc3RydXRoLy5jb25maWcvbW96aWxsYS9maXJlZm94L2dvMGRzbjR1LmRlZmF1bHQtcmVsZWFzZQ__"
  (= base64 of the LITERAL path  /home/user/.config/mozilla/firefox/xxxxxxxx.default-release)
GetNameOwner "org.mozilla.firefox.ZGVmYXVsdC1yZWxlYXNl"
  (= base64 of the profile NAME  "default-release")

Meanwhile the running server owns:

org.mozilla.firefox.L2RhdGEvc2lsZW5jZXNwZWFrc3RydXRoLy5jb25maWcvbW96aWxsYS9maXJlZm94L2dvMGRzbjR1LmRlZmF1bHQtcmVsZWFzZQ__
  (= base64 of the REALPATH     /data/user/.config/mozilla/firefox/xxxxxxxx.default-release)

No candidate matches, StartClient() fails, and the user gets the
profile-locked startup path instead of a tab in the running browser.

Expected results

The URL opens in the already-running instance, exactly as it does when no
symlink is involved.

Root cause

Both sides build the bus name symmetrically —
org.mozilla.<program>.<base64(profile)> — from whatever string
nsRemoteService::SetProfile() received
(toolkit/components/remote/nsDBusRemoteServer.cpp,
nsDBusRemoteClient.cpp). The asymmetry is in what that string is:

• A --profile <path> launch resolves the argument through
  XRE_GetFileFromPath(), which on Unix calls realpath()
  (toolkit/profile/nsToolkitProfileService.cpp), so mProfD — and therefore
  the registered server name — is fully canonicalized.
• Profile selection via profiles.ini builds mProfD by appending the
  relative entry to the profiles root and never canonicalizes, so the client's
  first candidate is the literal path. Its only fallback is the legacy
  profile-name form (toolkit/xre/nsAppRunner.cpp, "Older versions would use
  the profile name in preference to the path").

With no symlink the two forms coincide and nothing is visible. With a symlink
anywhere in the profile path, any pair of processes that arrived at the profile
by different routes disagree on the name forever.

Suggested fix (patch attached)

Canonicalize the profile identity once, where it is computed
(nsAppRunner.cpp, the MOZ_HAS_REMOTE block): after
mProfD->GetNativePath(profilePath), resolve it with realpath() on Unix.
Both the client candidates and the server registration flow from that one
string, so both sides agree regardless of how the profile was specified.

For compatibility with unpatched running instances, the client's fallback chain
becomes: canonical path → literal path (when different) → profile name,
mirroring the structure of the existing profile-name fallback:

• patched client vs. --profile-launched unpatched server: first candidate
  (realpath) matches — this is the reported case, fixed;
• patched client vs. profiles.ini-launched unpatched server: literal-path
  fallback matches;
• unpatched client vs. patched server: unchanged behavior for non-symlink
  layouts (the two forms coincide); symlink layouts remain broken until the
  client is updated, which is no worse than today.

Verified locally on Firefox 154.0.1: a patched build hands off to an unpatched
--profile-launched instance through a symlinked profile path; an unpatched
build reproducibly fails the same handoff.

Related code in the same class (why fixing at the source is right)

The same mProfile string feeds every remoting backend, so canonicalizing it
once in nsAppRunner.cpp also fixes the X11 path (_MOZILLA_PROFILE window
property, nsXRemoteServer.cpp), the GTK variant, and the macOS backend
(BuildClassName, nsMacRemoteServer.mm) — none of them need separate
changes.

Latent siblings this patch deliberately does NOT touch, listed for triage:

• Windows remoting (nsWinRemoteClient.cpp / nsWinRemoteServer.cpp)
  derives the message-window class name from the same string, but the
  XP_WIN branch of XRE_GetFileFromPath() uses _wfullpath(), which does
  not resolve junctions/symlinks — a junction-based profile layout can hit
  the same mismatch on Windows. The patch keeps realpath() Unix-only.
• nsToolkitProfileService::GetProfileByDir matches a --profile
  directory against known profiles with nsIFile::Equals (string equality),
  so a canonicalized path never matches the literal profiles.ini entry and
  the launch is treated as an unknown profile.
• GetProfileDescriptor decides relative-vs-absolute serialization via
  mAppData->Contains(aRootDir) (string prefix) and the loader re-serializes
  and rewrites Path= on mismatch — this is the mechanism behind the
  still-open bug 1800867 (symlinked profile serialized incorrectly).
• [Install...] Default= matching (installProfilePath.Equals(descriptor))
  is a descriptor string compare; a form flip (literal → canonical) silently
  unbinds the install's dedicated default profile.
• Downgrade/compatibility check compares the last and current GRE dirs
  with nsIFile::Equals — an application dir reached through a changed
  symlink route reads as an install change.

Checked and not affected: the remoting startup lock
($TMPDIR/<program>, profile-independent) and the profile lock itself
(fcntl on .parentlock, inode-based, symlink-transparent).

Comment 1

[Sergii Grechko]

Attachment: Canonicalize the profile path used for the remoting identity (against 154.0.1)

Patch against the 154.0.1 source tarball; applies to current mozilla-central toolkit/xre/nsAppRunner.cpp as well (the block is unchanged). Verified locally: see the A/B in the report. Happy to submit via moz-phab/Phabricator.