1927599 - [Gecko iOS] Get Gecko booting on-device

Status: RESOLVED FIXED

(Core :: IPC, task)

Comment 1

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 1: Don't build for arm64e, r=glandium!

Comment 2

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 2: Always codesign the XUL dylib, r=glandium!

The __IS_NOT_SIMULATOR environment variable I found some reference to online - it appears to not exist so everything falls over due to being unsigned if I continue to check it. This may break simulator builds, but I haven't looked into it yet.

Depends on D227098

Comment 3

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 3: Should not assert fds_to_remap size on iOS

Depends on D227099

Comment 4

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 1: Disable nsMemoryInfoDumper on iOS, r=glandium!

This depended on the libevent event loop, which is not going to work in iOS content processes due to the sandbox blocking the kevent syscall.

Depends on D227100

Comment 5

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 2: Parent process cannot have child task_t on iOS, r=#ipc-reviewers!

Depends on D227101

Comment 6

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 5: Properly set small alloc randomization for iOS content processes, r=jld!

Depends on D227102

Comment 7

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 6: Allow arbitrary XPC objects on the command-line, r=#ipc-reviewers!

PC will be used as the temporary IPC backend, so we need to be able to pass XPC endpoints on the command line.

Depends on D227103

Comment 8

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 7: Use XPC as an IPC backend on iOS, r=#ipc-reviewers!

This approach, especially in its current form, has quite a bit of overhead, but
should help us get some prototyping done for iOS. In the future, we'll probably
want to directly use mach ports, either with a kevent-based message loop (if
that syscall is unblocked) or with a libdispatch queue replacing our IPC IO
Thread.

This also required shimming out the process watcher, which also used the libevent pump.

Depends on D227104

Comment 9

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 8: Disable the BackgroundHangMonitor on iOS

This is a temporary hack because the I/O thread currently being used on iOS can't initialise BHR or it will crash - I want to fix this.

Depends on D227105

Comment 10

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 8: Require iOS 18.2 or newer

Depends on D227287

Comment 11

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 3: Enable extended virtual addressing for the WebContent process, r=glandium!

Depends on D227288

Comment 12

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 4: Inline JIT calls on iOS, r=jandem!

Depends on D227289

Comment 13

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 8: Use a libdispatch queue as the IO thread on iOS, r=#ipc-reviewers!

This reduces the number of required thread hops when sending messages over IPC
by having the IO thread be the libdispatch queue connected to the xpc
connection.

Depends on D227105

Comment 14

[Nika Layzell [:nika] (ni? for response)]

Status update: With this patch stack + the other patches on cedar, we're able to boot a proof-of-concept version of Gecko on an iOS 18.2 beta device. This requires us to build for aarch64, rather than arm64e. So far the main changes which are required in the stack are around the sandbox blocking some core IPC methods (specifically kevent and kevent64), and a bug with BrowserEngineCore on-device with aarch64, which we have filed in feedback assistant.

https://phabricator.services.mozilla.com/D227740 contains the code to work around the BrowserEngineCore bug using inline assembly - though hopefully Apple will fix this upstream and we won't need it. Without this change, the JIT crashes early during startup.

In terms of IPC sandboxing issues, we encountered 2 major issues:

1. We use mach task_t ports in order to transfer mach send rights between processes on iOS, combined with a unix domain socket for sending IPC payloads. Transferring a mach task port to the parent process over XPC appears to be blocked by the content process sandbox, so we cannot use this strategy on iOS (https://phabricator.services.mozilla.com/D227102)
2. Even without the ability to transfer mach ports, we cannot start our IPC connection, as the kevent syscall used by libevent for async I/O is blocked by the content process sandbox. It appears the only variant of this syscall not blocked by the sandbox is SYS_kevent_qos - an apple internal API used by libdispatch, so we need to use a libdispatch-based async I/O backend.
   • For simplicity in the initial implementation, this is currently being done by an XPC-based IPC channel implementation running on a libdispatch queue. There's a good chance we'll want to switch this to use a lower-level primitive like mach ports in the future, especially if the IPC overhead starts appearing in profiles.
   • This temporary implementation also ended up requiring some refactoring as we couldn't easily use our fork of the chromium IPC I/O loop, due to the libdispatch public interface. This is being done in bug 1928734.

Comment 15

[Nika Layzell [:nika] (ni? for response)]

One of the sources of complexity in this current patch stack is the need to change the model we use for the I/O thread to instead use a libdispatch event target. I took another stab at keeping the I/O thread by using a CFRunLoop-based thread and CFFileDescriptor (https://developer.apple.com/documentation/corefoundation/cffiledescriptor?language=objc) to watch file descriptors. Unfortunately, the CFFileDescriptorCreate call seems to be blocked by the iOS WebContent and Rendering sandboxes (but not the networking sandbox). AFAICT this is because it calls fcntl(.., F_DUPFD_CLOEXEC) under the hood, which is not an allowed fcntl call. I can't easily check for other issues past that point.

I expect that using CFRunLoop for watching mach ports will work OK. FD handling appears to not be in the swift-corelibs-foundation, but parts of the runloop backend are open source, and it appears CFRunLoop uses a mach port set and mach_msg to wait for messages (https://github.com/swiftlang/swift-corelibs-foundation/blob/21b3196b33a64d53a0989881fc9a486227b4a316/Sources/CoreFoundation/CFRunLoop.c#L3100). That being said, it might be simpler to directly call mach_msg in that case, though, rather than calling indirectly through CoreFoundation.

Based on the CFRunLoop source, it appears there's no handling for file descriptors in CFRunLoopRun. The CFRunLoopSource produced by CFFileDescriptorCreateRunLoopSource (in a non-sandboxed process) doesn't have a version 1 context with a mach_port_t (which would be added to the port set). This suggests that something like libdispatch is being used to watch FDs on a background thread, and notifications are then dispatched to the CFRunLoopSource. This can probably be done in user code as well if FD watchers are needed (though I believe Gecko could probably get away without them on iOS, if we switch to mach ports for IPC).

I'll probably look into a mach port set based IO thread more, as it's probably less invasive than using a libdispatch queue.

Comment 16

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 5: Disable JIT on iOS in LockdownMode, r=glandium

Depends on D227290

Comment 17

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 6: Get OpenGL-based rendering working on-device, r=gw!

This requires adding a call to texImageIOSurface, which is only available
on-device, so cannot be used for simulator builds. We will need to come up with
another soltuion for simulator, which may reqire a non-IOSurface-based backend.

In addition, the compositor code needs to be changed to use GL_TEXTURE_2D
instead of GL_TEXTURE_RECTANGLE, as that type doesn't exist for OpenGL ES on
iOS.

Depends on D229402

Comment 18

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 7: Get basic WebGL working on-device, r=gw!

Depends on D229403

Comment 19

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 14: Work around assembly label collision error in be_memory, r=glandium!,#spidermonkey-reviewers!

As of the iOS 18.2 SDK, the definition of
be_memory_inline_jit_restrict_rwx_to_rw_with_witness on non-arm64e uses both a
numeric 1: label, and a %= label. In the case where LLVM selects that the
%= label should be the number '1', this leads to a miscompilation (and
crash), as the adr x0, %=f expression loads the incorrect address into x0.

To work around this until the definition is fixed, this patch adds two empty
__asm__ blocks before the library call. This ensures that there are at least
2 LLVM IR call void asm sideeffect "${:uid}" blocks incrementing the value
for %= before the problematic function, avoiding the miscompilation. They are
placed within the same function to ensure that they cannot be eliminated or
re-ordered by the compiler, as the order of the blocks in the post-optimization
IR appears to be what controls the value of the %= label.

Depends on D230338

Comment 20

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 15: Initialize the localtime cache before process lockdown on iOS, r=glandium!

The locked down iOS sandbox appears to block attempts to read the localtime
after the lockdown is in place. As it appears localtime information is required
in content processes for HTTP header parsing, we need to access this
information (loading it into the cache), before the sandbox is locked down.

Depends on D233422

Comment 21

[Phabricator Automation]

Comment on attachment 9433778 [details]
WIP: Bug 1927599 - Part 5: Properly set small alloc randomization for iOS content processes, r=jld!

Revision D227103 was moved to bug 1945252. Setting attachment 9433778 [details] to obsolete.

Comment 22

[Nika Layzell [:nika] (ni? for response)]

Attachment: Bug 1927599 - Part 8: Dont build mozsandbox on iOS, r=haik!

This is the sandboxing implementation on macOS, and is not relevant on iOS
where sandboxing profiles are provided by the OS.

Depends on D230338

Comment 23

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1927599 - Part 10: Build CG parts of DrawTargetSkia on iOS, r=#gfx-reviewers!

Depends on D251641

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/9df584253d6c
https://hg.mozilla.org/integration/autoland/rev/30519656e5dd
Part 1: Disable nsMemoryInfoDumper on iOS, r=glandium
https://github.com/mozilla-firefox/firefox/commit/9f7cc5351f2e
https://hg.mozilla.org/integration/autoland/rev/09a79d174467
Part 2: Parent process cannot have child task_t on iOS, r=ipc-reviewers,profiler-reviewers,mstange,jld
https://github.com/mozilla-firefox/firefox/commit/141a27ea8b44
https://hg.mozilla.org/integration/autoland/rev/c5b8ecb22d85
Part 3: Enable extended virtual addressing for the WebContent process, r=glandium
https://github.com/mozilla-firefox/firefox/commit/c3e0176a0945
https://hg.mozilla.org/integration/autoland/rev/0e8627bb43f4
Part 4: Inline JIT calls on iOS, r=jandem
https://github.com/mozilla-firefox/firefox/commit/195873d7f314
https://hg.mozilla.org/integration/autoland/rev/9ac236f5314a
Part 5: Disable JIT on iOS in LockdownMode, r=glandium
https://github.com/mozilla-firefox/firefox/commit/5f46d005fa5d
https://hg.mozilla.org/integration/autoland/rev/e562cebaefcd
Part 6: Get OpenGL-based rendering working on-device, r=gfx-reviewers,ahale
https://github.com/mozilla-firefox/firefox/commit/7f392e31036c
https://hg.mozilla.org/integration/autoland/rev/19da8ee802cc
Part 7: Get basic WebGL working on-device, r=gfx-reviewers,ahale
https://github.com/mozilla-firefox/firefox/commit/3fb87ec935c4
https://hg.mozilla.org/integration/autoland/rev/18a01c664262
Part 8: Dont build mozsandbox on iOS, r=haik

Comment 25

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/30519656e5dd
https://hg.mozilla.org/mozilla-central/rev/09a79d174467
https://hg.mozilla.org/mozilla-central/rev/c5b8ecb22d85
https://hg.mozilla.org/mozilla-central/rev/0e8627bb43f4
https://hg.mozilla.org/mozilla-central/rev/9ac236f5314a
https://hg.mozilla.org/mozilla-central/rev/e562cebaefcd
https://hg.mozilla.org/mozilla-central/rev/19da8ee802cc
https://hg.mozilla.org/mozilla-central/rev/18a01c664262