Closed Bug 1598951 Opened 1 year ago Closed 1 year ago

Replay macOS recordings on linux

Categories

(Core Graveyard :: Web Replay, enhancement)

enhancement
Not set
normal

Tracking

(firefox73 fixed)

RESOLVED FIXED
mozilla73
Tracking Status
firefox73 --- fixed

People

(Reporter: bhackett1024, Assigned: bhackett1024)

References

(Blocks 1 open bug)

Details

Attachments

(2 files, 2 obsolete files)

As described in bug 1547081, being able to take recordings that were made on macOS and replay them on linux is a major part of the plan for cloud integration. I've looked into several ways of doing this, and I think I've found a pretty clean approach that doesn't require any changes to the firefox build system. I'd like an application targeting linux which takes as input the recording and the firefox binaries used to make the recording (plugin-container, XUL, libmozglue, libnss3, liblgpllibs, i.e. nothing that violates Apple's EULA). The application loads these binaries and links them to a set of stubs that either emulate the calls with native linux library functions (for pthreads etc.) or simply crash when being called. Functions in the latter bucket need to be redirected as part of the recording; when replaying these stubs will be redirected by the existing machinery and produce recorded information when they are called.

This design is similar to what Wine does (run windows software on linux by linking it against a set of functions emulating windows APIs). There is already a Wine-like project for running macOS software on linux, Darling (https://www.darlinghq.org/), though it doesn't support GUIs yet. The job here is much simpler than what Wine or Darling need to do, however, because we only need to emulate a small set of APIs and can rely on the recording to deal with the great bulk of the interactions between the replaying process and the system.

The main difficulty here is that macOS and linux have different binary formats (Mach-O vs. ELF). In order to load the binaries, the application needs to understand Mach-O and perform the appropriate dynamic linking steps. This seems intimidating at first but is actually not too hard and kind of fun. The Mach-O format is reasonably well documented, with open source headers and a linker implementation from Apple (APSL licensed, which is similar to BSD/MIT if you don't modify the code) and a number of third party tools for operating on binaries (none of which seem reusable here, but which are good for referencing).

The application performing this linking does not need to be compiled into firefox and its source can live in a separate place. I've started a github repository here:

https://github.com/bhackett1024/ReplayLinker

Some changes to mozilla-central/toolkit/recordreplay will be needed as well.

Disclaimer: work I'm doing on web replay is done on my own time, and is not part of my official work at mozilla. If you have a problem with me working on this, please talk to me directly.

Attached patch WIP (obsolete) — Splinter Review

WIP for Gecko changes, with a fair amount of debugging crud mixed in. With this patch and the ReplayLinker repository's current code, I can record a "Hello World" page on macOS and completely replay it on Linux.

Developing this went pretty smoothly. Besides emulating the linker, the main changes needed are ways to handle the various system libraries that are not currently redirected, either by redirecting them, emulating them, or simply forwarding them to the same Linux symbol (many of these are very basic C library functions like memcpy, snprintf, and so forth). The main unexpected bit is that there are a fair number of C++ symbols linked against when using the STL, for things like std::string, std::ostream, etc. (not STL containers though, I guess because they're so extensively templatized). At first I tried emulating the behavior of these symbols, but it became too annoying so I linked against libc++.dylib in addition to the mozilla dynamic libraries. This shouldn't cause licensing issues, as libc++ is part of LLVM and has a permissive open source license.

I'll also note that rr was a big help in developing this, it's nice to finally be able to use it for working on web replay!

Attached patch WIP (obsolete) — Splinter Review

Updated patch that deals with the changes that have been going on in bug 1597149 and other connected bugs. This works and is pretty much done but I'm going to hold off on asking for review until cloud support is further along.

Attachment #9112704 - Attachment is obsolete: true
Depends on: 1606447
Attachment #9117434 - Attachment is obsolete: true
Pushed by bhackett@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cf403935cf9b
Part 1 - Avoid connecting to parent when replaying remotely, r=gsvelto.
Pushed by bhackett@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2e225a8de1f9
Part 1 - Avoid connecting to parent when replaying remotely, r=gsvelto.
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla73
Pushed by bhackett@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/27a2f0bdfce9
Part 2 - Don't record random numbers in jemalloc, r=glandium.
Flags: needinfo?(bhackett1024)
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.