1926560 - Add an interface to AddonManager to install an add-on from a string buffer (in-memory)

Status: NEW

(Toolkit :: Add-ons Manager, task, P3)

Description

[Alexandra Borovova [:Sasha]]

At the moment, when working with AddonManager it's only possible to request an add-on installation with a file saved to the file system. At the same time, in Marionette (and at some point in WebDriver BiDi as well) we have an option to install an add-on with base64 representation of the add-on file. Right now we decode the string, create a file out of it, save it and then request AddonManager to install it. This approach has a downside of requiring to identify an appropriate place where this file can be saved, and that is not straightforward on the Android devices.
So the idea here would be to have an API to request installation with just the content of the file (string, buffer, what ever works best), which would be created from a decoded base64 string sent by a client.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I'm going to suggest a change of the summary so that it is clear that it is not about any file contents but just a string/buffer representation of the addon archive in memory.

Comment 2

[Rob Wu [:robwu]]

Without any new additions to the AddonManager internals, it is already possible to install for a given string of binary data representing the add-on:

• The base64-data can be appended to data:application/x-xpinstall;base64, to create a data:-URL. A potential downside to this is that the install source of add-ons may be saved, and data:-URLs tend to be very long.
• The Blob constructor + URL.createObjectURL method can be used to create a blob:-URL. This does not have the length disadvantage of data:-URLs.

This approach has a downside of requiring to identify an appropriate place where this file can be saved, and that is not straightforward on the Android devices.

Could you elaborate this? I know that Android has some restrictions on filesystem access, and would like to understand which concern you have with using the local file.

/tmp/ or %TMP% on desktop is a world-readable/writable directory, and such a directory also exists on Android at /data/local/tmp/. If this is too open to you, /sdcard/Android/data/<appid>/ can also be an option, as it can be read&written by an app only (but requires knowing the app ID). If running within the app, the private /data/ directory of the app is also a viable option.

Comment 3

[Alexandra Borovova [:Sasha]]

(In reply to Rob Wu [:robwu] from comment #2)

Without any new additions to the AddonManager internals, it is already possible to install for a given string of binary data representing the add-on:

• The base64-data can be appended to data:application/x-xpinstall;base64, to create a data:-URL. A potential downside to this is that the install source of add-ons may be saved, and data:-URLs tend to be very long.
• The Blob constructor + URL.createObjectURL method can be used to create a blob:-URL. This does not have the length disadvantage of data:-URLs.

Thanks for the tip! Both options generally seem to be working.
But I have a question here. From what I've understood, I have to use getInstallForURL method to work with URLs, which is fine for non-temporary add-ons. But we also provide an option to install an add-on temporary, and I could only find installTemporaryAddon method which requires the nsIFile to be provided. Did I miss something, or maybe there is a different way to also install an add-on temporary from URL?

This approach has a downside of requiring to identify an appropriate place where this file can be saved, and that is not straightforward on the Android devices.

Could you elaborate this? I know that Android has some restrictions on filesystem access, and would like to understand which concern you have with using the local file.

/tmp/ or %TMP% on desktop is a world-readable/writable directory, and such a directory also exists on Android at /data/local/tmp/. If this is too open to you, /sdcard/Android/data/<appid>/ can also be an option, as it can be read&written by an app only (but requires knowing the app ID). If running within the app, the private /data/ directory of the app is also a viable option.

Maybe Henrik can add more details here, but we have other places where we now already create files to save some information, and yes it's probably generally working fine for desktop and Android, but we have bug reports from users with some Android devices for whom it doesn't work.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Rob Wu [:robwu] from comment #2)

This approach has a downside of requiring to identify an appropriate place where this file can be saved, and that is not straightforward on the Android devices.

Could you elaborate this? I know that Android has some restrictions on filesystem access, and would like to understand which concern you have with using the local file.

/tmp/ or %TMP% on desktop is a world-readable/writable directory, and such a directory also exists on Android at /data/local/tmp/. If this is too open to you, /sdcard/Android/data/<appid>/ can also be an option, as it can be read&written by an app only (but requires knowing the app ID). If running within the app, the private /data/ directory of the app is also a viable option.

It's mainly an issue for geckodriver to find the right path to push the XPI onto the device. Some devices have kinda strange locking mechanisms causing us to fail pushes. See https://github.com/mozilla/geckodriver/issues/2092.

The problem is different when we are running code in Firefox itself. Then we could use the Firefox profile directory to store the file temporarily for installation. That should always work. And here it's mainly a question of performance, and that we unnecessarily have to write the XPI to disk and for reading it immediately.

Comment 5

[Rob Wu [:robwu]]

(In reply to Alexandra Borovova [:Sasha] from comment #3)

But I have a question here. From what I've understood, I have to use getInstallForURL method to work with URLs, which is fine for non-temporary add-ons.

The process for a regular installation is roughly:

1. Download specified URL to a temporary file on disk.
2. Validate installation and put it at a permanent location within the profile directory on disk.
3. Delete the temporary file (even if step 2 failed).

But we also provide an option to install an add-on temporary, and I could only find installTemporaryAddon method which requires the nsIFile to be provided. Did I miss something, or maybe there is a different way to also install an add-on temporary from URL?

For temporary installations, we do not make any copies. The implementation loads the extension directly from the specified nsIFile, whether a zip/xpi file, or even a directory containing the unpacked extension.

Ultimately, the extension internals operate on files (or resource://-URLs resolving to files). Supporting in-memory non-file extension packages would require significant work. (just to see how much, I tried implementing logic that ultimately creates a jar:-URL from a blob or data:-URL, and both fail with NS_ERROR_UNSAFE_CONTENT_TYPE because jar: really expects a file (https://searchfox.org/mozilla-central/rev/964b8aa226c68bbf83c9ffc38984804734bb0de2/modules/libjar/nsJARChannel.cpp#1166))

Just reluctance to creating local files does not feel like a sufficiently strong motivation to introduce first class support for file-less add-on installations, especially because the implementation complexity to support it is substantial.

Ultimately, there needs to be a file somewhere. I guess that the question boils down to whether Firefox should manage it, or the test driver.
I'm leaning towards letting the test driver manage it, to minimize complexity in Firefox itself.

Comment 6

[Rob Wu [:robwu]]

Attachment: Bug 1926560 - prototype install from string WIP DO NOT LAND

This does not work - whether using blob: or data:-URL, the
test_temporary_in_memory.js test fails with a network error.

The cause for the error is internally NS_ERROR_UNSAFE_CONTENT_TYPE
from https://searchfox.org/mozilla-central/rev/964b8aa226c68bbf83c9ffc38984804734bb0de2/modules/libjar/nsJARChannel.cpp#1166