Closed Bug 1897498 Opened 27 days ago Closed 2 days ago

Build a single-file archive export method on BackupService

Categories

(Firefox :: Profile Backup, task, P3)

task

Tracking

()

RESOLVED FIXED
129 Branch
Tracking Status
firefox129 --- fixed

People

(Reporter: mconley, Assigned: mconley, NeedInfo)

References

(Blocks 4 open bugs, Regressed 1 open bug)

Details

(Whiteboard: [fidefe-device-migration])

Attachments

(1 file)

This method should:

  1. Take in a URL or handle to a local ZIP file. It might be a file URI, or maybe an nsIInputStream or ReadableStream.
  2. Have the method accept some items from an ArchiveEncryptionState if encryption is enabled. Those things are the public key, the salt, the backup auth key, and the wrapped static secrets. Make sure that the worker knows if encryption is expected to be enabled so that it can fail out if any of these things are not present or cannot be worked with.
  3. Have the method use some template URI for the single-file archive contents, as well as the backup metadata.
  4. Have the method fill in the template with items from the metadata, and then create a document comment block after the closing </html> tag, like this: <!-- Begin inline MIME --
  5. Create a MIME message header on the next line containing the content type data (Content-Type: multipart/mixed; boundary="<some generated bounday>")
  6. Create a application/json MIME section header, and then write a JSON structure. For now, it'll just be the metadata. In the future, it'll be the metadata alongside some cryptographic information.
  7. Create a new binary/octet-stream MIME section header, including the content length.
  8. Pump the base64 representation of the ZIP file into this region. Since the BackupService will be the only application that we intend to support reading this block, we don't need to break it up with newlines.
  9. On a newline, close the comment block with -- End inline MIME -->, followed by a newline.

We won't worry about doing the cryptography bit just yet - we'll just do the raw base64 encoding right now. We also won't worry about hooking this up to the createBackup mechanism just yet.

Summary: Build a single-file archive export worker → Build a single-file archive export method on BackupService

I'm waffling a bit here. I think I'm going back to the Worker model, mainly because I realized I can use IOUtils in a privileged ChromeWorker, which is nice - and also because base64 encoding large chunks of data on the main thread is never going to pass muster, performance-wise.

There are a number of interesting things going on this patch that I think are worth highlighting
here for my reviewers:

  1. The single-file archive format is an HTML file that uses an inlined multipart/mixed MIME
    message within a HTML document comment in order to embed the backup data into the archive.
  2. We use the multipart/mixed nsIStreamConverter to extract the JSON and binary data from
    the MIME block.
  3. We use a Archive Worker to do the archive creation, allowing us to do the work of construction
    off of the main thread.
  4. The Archive Worker is only parsing the header and getting the byte offset of the MIME block.
    Extraction is happening in the parent process. This is mainly for simplicity for now, since
    the Archive Worker cannot invoke an nsIStreamConverter. Down the line, if we determine that
    we'd prefer the Archive Worker do the base64 decoding off of the main thread, we may need
    to use a Message Channel to send the byte sfrom the nsIStreamConverter to it, and add
    stream-writing support to IOUtils so that the Archive Worker can take care of sending the
    decoded bytes to disk.
  5. The patch doesn't expose the extraction mechanism in any way except through the debug
    interface right now. That will come down the line. In the meantime, this mechanism
    can be manually tested in the debug interface by creating a backup, which should also
    create an "archive.html" file in the backups folder. Using the "Extract from archive"
    button in the debug tool will let you select that HTML file and extract the ZIP as
    a file in the backups folder called "extraction.zip".
  6. The test template contains Unicode characters because certain locales might involve
    us writing Unicode characters in the HTML template when generating the archive. The
    fun part about that is calculating where the byte offset is for the MIME block! See
    the comment in the Archive.worker.mjs script for how that works.
Blocks: 1900125
Pushed by mconley@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d88e9ba4a866
Add a mechanism for exporting backup snapshots to a single file archive. r=Gijs,valentin,backup-reviewers,frontend-codestyle-reviewers,Standard8,kpatenio
Regressions: 1901541

Backed out for causing xpcshell failures in test_BackupService_archive.js

Flags: needinfo?(mconley)
Blocks: 1901640

Pulse bot did not comment on the bug when this got pushed again to autoland because of bugzilla hiccup.
Push link: https://hg.mozilla.org/integration/autoland/rev/5f31ec57317e668a647da5299e4dee4809fc8ccf

Status: NEW → RESOLVED
Closed: 2 days ago
Resolution: --- → FIXED
Target Milestone: --- → 129 Branch
Blocks: 1902020
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: