2054404 - Migrate the macOS privileged updater helper from NSConnection to NSXPCConnection

Status: ASSIGNED

(Toolkit :: Application Update, task)

Description

[Stephen A Pohl [:spohl]]

The macOS updater's privileged helper currently communicates with the unelevated updater over NSConnection, which Apple has long deprecated. This bug tracks moving that IPC to NSXPCConnection.

The new arrangement follows the standard SMJobBless pattern: the privileged helper exposes a system-domain mach service via its launchd plist's MachServices entry, and the unelevated updater connects to it as the XPC client, rather than the previous setup where the unelevated updater ran an NSConnection server that the helper connected back into.

Since the unelevated updater and the privileged helper are the same signed binary, both peers can authenticate each other against the updater's own code-signing identity. The helper validates each incoming connection's audit token against that requirement, and on macOS 13+ both sides additionally call setCodeSigningRequirement: for framework-level enforcement.

This should introduce no user-visible behavior change on its own. It is being split out from a larger piece of macOS updater work so it can be reviewed and land independently as an IPC modernization.

Comment 1

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Migrate the privileged updater helper from NSConnection to NSXPCConnection. r?#application-update-reviewers,yjuglaret!

Replace the NSConnection-based IPC between the unelevated updater and the privileged helper with NSXPCConnection. The previous arrangement had the unelevated updater register an NSConnection server and the helper connecting back to fetch arguments; the new arrangement is the standard SMJobBless pattern: the helper exposes a system-domain mach service via its launchd plist's MachServices entry, and the unelevated updater connects to it as the XPC client.

Mutual peer validation is built in from this commit. The unelevated updater (the XPC client) and the privileged helper are the same signed binary, so both peers authenticate each other against the updater's own code-signing identity (the SMPrivilegedExecutables requirement). The helper validates each incoming connection's audit token against that requirement before accepting it and, on macOS 13+, also calls setCodeSigningRequirement: for framework-level enforcement on every message. The client side likewise calls setCodeSigningRequirement: on macOS 13+ against the same requirement; on macOS 10.15-12 client-side validation relies on launchd's exclusivity guarantee for system-domain mach services, since only a daemon loaded by launchctl bootstrap system (which SMJobBless does after kernel-validated code-signing) can claim the registered name and a non-admin attacker cannot impersonate the helper. The SMAuthorizedClients requirement (matching the browser bundle) continues to gate SMJobBless installation only.

NSConnection is deprecated; NSXPCConnection is the supported mechanism for this kind of privileged-helper communication. This is purely an IPC modernization and introduces no user-visible behavior change on its own: SMJobBless still installs the helper per update, and the helper still tears itself down via launchctl remove after the update completes.

Notable supporting changes:

• The new XPC protocol (UpdaterHelperProtocol) carries applyUpdate and abortWithReply RPCs. The single applyUpdate call replaces what getArguments + getMARChannelID + shutdown previously carried over NSConnection.
• The code-signing requirement strings used for runtime peer validation are baked into a generated header (UpdaterCodeSigningRequirements.h) at build time. The strings reuse MOZ_SMAUTHORIZEDCLIENTS_REQUIREMENTS and MOZ_SMPRIVILEGEDEXECUTABLES_REQUIREMENTS configure-time values with MOZ_MACBUNDLE_ID substituted in.
• AbortElevatedUpdate now takes a pid_t and signals the unelevated child via SIGTERM. The previous NSConnection-based abort RPC isn't needed in the flipped topology: when SMJobBless fails the helper never appears and the unelevated client has no peer to disconnect from, so a process signal is the cleanest way to break it out of its blocking wait.
• MacRunFromDmgUtils now calls AbortElevatedUpdate before [task waitUntilExit] (otherwise we would deadlock waiting for an unelevated client that has nobody to talk to).

The listener delegate rejects concurrent connections via a gActiveClientConnection / gConnectionLock interlock, so two simultaneous updates from same-edition installs can't race on the shared pending-args/reply-block state or on overlapping filesystem mutations from applyUpdate. The rejected client sees a connection invalidation; its update fails this round and the user's normal update-retry path picks it up later, after the in-progress update finishes and the helper is available again. The connection's invalidation handler also wakes the helper's main thread if a client disconnects before sending applyUpdate or abort, so the helper cleans up and exits rather than blocking indefinitely.

Comment 2

[Stephen A Pohl [:spohl]]

Try run: https://treeherder.mozilla.org/jobs?repo=try&landoInstance=lando-prod-2025&landoCommitID=86293

Comment 3

[Stephen A Pohl [:spohl]]

I've requested QA coverage for this change before it lands: https://mozilla-hub.atlassian.net/browse/QA-5853. The plan is to test it on Pine first, with one Pine build that has the patch and a second one to update to, and to land it in Nightly for Firefox 161 once that checks out.

Setting a needinfo on myself to land the patch after QA is done.

Comment 4

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Remove a leftover helper instead of updating through it after the authentication dialog was dismissed. r?#application-update-reviewers,yjuglaret!

When the privileged helper can't be installed, for example because the user dismissed the authentication dialog, the unelevated updater still runs so that it can record ELEVATION_CANCELED and relaunch Firefox. If a helper from an earlier update was still registered with launchd, for example after it crashed, it accepted the updater's connection and applied the update anyway.

LaunchElevatedUpdate now tells the updater whether installation failed through MOZ_UPDATER_HELPER_NOT_INSTALLED. The updater reads and clears it at startup, and on failure it doesn't update. Instead it asks a leftover helper to remove itself through the new removeHelper method, which goes through the same peer validation as applyUpdate, then records ELEVATION_CANCELED and relaunches Firefox.

Comment 6

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Remove the updater's unused -dmgInstall code path. r?#application-update-reviewers,yjuglaret!

Since bug 2031817, an install from a mounted DMG goes through the separate DMG install helper, and nothing launches the updater with -dmgInstall any more. Most of the diff is NS_main losing one level of indentation.

Comment 7

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Retry an elevated update while the privileged helper handles another one. r?#application-update-reviewers,yjuglaret!

A client that reaches a helper that already took another request now records ELEVATED_UPDATE_IN_PROGRESS_ERROR, and Firefox prompts again later without counting a cancellation. After 10 such results in a row for the same update, they count as cancellations, so that a helper that stays busy can't keep the update pending forever.

Comment 8

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Don't stop an elevated update in progress by installing the privileged helper again. r?#application-update-reviewers,yjuglaret!

SMJobBless sends a running helper SIGTERM, even when it installs the same version, and launchd kills it a few seconds later. While the helper handles a request, it now records its process ID, and Firefox doesn't install the helper again while that process runs, so the client gets a busy result instead. The helper also ignores SIGTERM until it is done, and its plist sets ExitTimeOut to 60 seconds, which is how long launchd then waits before it kills the helper.

Comment 9

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Extract the manifest and patch files outside of the application bundle on macOS. r?#application-update-reviewers,yjuglaret!

The updater extracted them into an updating directory at the root of the bundle, which an interrupted update left behind, inside the signed bundle. On macOS they now go into a private directory in the temporary directory of the user who runs the updater. A failed update no longer changes the bundle's modification time, so the failure tests expect that now.

Comment 10

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Make the privileged helper refuse requests that a legitimate updater doesn't send. r?#application-update-reviewers,yjuglaret!

The helper now only runs the second updater invocation, and refuses staged and replace requests, which Firefox doesn't send for an update that needs elevation. It records ELEVATED_UPDATE_INVALID_REQUEST_ERROR for those.

Comment 11

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Pin the install directory that the privileged helper updates. r?#application-update-reviewers,yjuglaret!

The helper resolves the install directory once, opens it, and refuses it unless it is an app bundle (INVALID_INSTALL_DIR_PATH_ERROR) on a volume that root mounted with ownership enabled (the new ELEVATED_UPDATE_UNSUPPORTED_VOLUME_ERROR). Firefox uses the same volume check later in this stack. The working directory has to resolve to the same directory (INVALID_APPLYTO_DIR_ERROR), and from then on both are the resolved path.

Comment 12

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Make the privileged helper refuse installs that the client or other users can change. r?#application-update-reviewers,yjuglaret!

The helper now refuses an install that the client can change, through ownership, mode bits, group membership or an ACL, or that a user other than root and the install's owner can change, with the new ELEVATED_UPDATE_CHANGEABLE_INSTALL_ERROR. When the client isn't in the admin group, entries owned by or writable for admin members are accepted, because the helper gives that group write access after an update and another admin may have updated the install since.

The rule is in updaterfileutils_osx.mm, which Firefox also builds, and has a gtest.

Comment 13

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Don't ask for a password to update an install that the privileged helper refuses. r?#application-update-reviewers,yjuglaret!

Before asking for a password, Firefox now runs the helper's volume and ownership checks for the user, with the helper's code. When one fails, it doesn't ask, and the updater records the helper's error without contacting it. Entries that the user can't read are skipped, since the helper can check them.

Comment 14

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Make the privileged helper read update.mar through the descriptor that it checked. r?#application-update-reviewers,yjuglaret!

The client owns the patch directory. The helper now opens update.mar once without following a link, refuses anything but a regular file (READ_ERROR), and reads the update from that descriptor through the new mar_fdopen and ArchiveReader::Open(int).

Comment 15

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Make the privileged helper refuse an install whose version isn't its own. r?#application-update-reviewers,yjuglaret!

The updater refuses a MAR older than its own MOZ_APP_VERSION, and for the helper that is the version of the Firefox that last installed it, not necessarily that of the install it is asked to update. The helper now refuses an install whose application.ini gives another version, with the new ELEVATED_UPDATE_VERSION_MISMATCH_ERROR.

Comment 16

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Make the privileged helper read the accepted MAR channels from the install. r?#application-update-reviewers,yjuglaret!

The helper took the MAR channel from the client, which chooses it. It now reads the accepted channels from the clone's own UpdateSettings.framework, after checking the framework against its code-signing requirement, and the client no longer looks the channel up or sends it. This also keeps the channels right after a release-candidate update or a channel switch, where the install's framework lists the channels it accepts.

Comment 17

[Stephen A Pohl [:spohl]]

Attachment: Bug 2054404: Give the admin group write access to the updated clone before swapping it in. r?#application-update-reviewers,yjuglaret!

The helper changed the group and mode of the install by path after the update, following whatever was at each path by then. It now does that on its root-only clone, before the swap, through descriptors and without following links, so it only changes what it updated.