When capturing profiles for local Firefox builds, we use
symbolication-worker.js to dump symbols from the build.
This worker currently uses OS.File for file access.
It needs to read parts of a file into an existing Uint8Array, synchronously.
This synchronous requirement cannot be changed; the library we use for symbolication only supports synchronous reads. This code is not running on the main thread, so there are no performance concerns.
File opening can be asynchronous, but after the asynchronous "setup" is done, all reads need to happen synchronously.
Furthermore, it really needs to do partial reads. We cannot just read the entire file upfront and then synchronously copy parts of it out, because one of the files we're dealing with here is libxul, which is frequently around 2GB big (depending on the platform).
The current OS.File solution is rather inefficient: https://share.firefox.dev/2W93lnB
- Starting the OS.File worker is slow because of the JS module imports, which ping-pong to the main thread. Switching to IOUtils would solve this.
- For every read, OS.File creates a new buffer. This causes two problems:
- It requires a copy from that buffer into the existing Uint8Array (here: wasm memory).
- It creates garbage which then needs to be collected.
- It uses ctypes. The profile above spends 8.7% of the
readBytesInto call in
WorkerPrivate::SetGCTimerMode which gets called when entering and exiting ctypes calls.
An efficient solution would have the following properties:
- All reads should use the same file handle, so that the file doesn't need to be opened for every read.
- There should be minimal copies.
- The API should support reading into an existing destination buffer.
It would be nice to add the necessary APIs to IOUtils so that the OS.File dependency can be dropped from symbolication.