Closed Bug 865444 Opened 11 years ago Closed 11 years ago

Implement Java-side data provider for add-ons

Categories

(Firefox for Android Graveyard :: Data Providers, defect)

Product:
Firefox for Android Graveyard
Component:
Data Providers
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 875400

People

(Reporter: rnewman, Unassigned)

References

Details

For Firefox Health Report, we need a way to *inexpensively* determine the set of installed add-ons, quite probably once per session plus once per day.

The "inexpensive" part implies that change events in Gecko need to be propagated into some kind of storage that the Java side can read. No Gecko runtime dependency. Slather with ContentProvider sauce.

Questions:

* Is there a better way to do this than to replicate data? (Can we read raw add-on manifests off disk, perhaps with some timestamping or something?)
* Does something like this already exist?
* Are there any other (potential) consumers who ought to be taken into account?

I'm tracking this for 23, ideally.
I don't know if it helps, but bug 853388 is converting the current SQLite to JSON. Since we do already use an Sqlite DB to store the installed add-ons, we could wrap it the way we do FormData and Passwords.
(Follow-on from comment in Bug 853388.)

Irving:

We almost certainly will be depending on internals -- the goal here is to provide access to the set of installed add-ons without having to run Gecko. So we'll be opening .sqlite files read-only and wrapping them in Java.

If that's ill-advised (and I admit that it seems so), we'll replicate into a separate store while Gecko is running by tromping over the add-ons manager, pushing JSON over the Gecko-Android bridge, and just keep that mirror up-to-date as add-ons change.

Do you have either a visceral reaction or a reasoned opinion (or, better yet, an awesome third option) on how to achieve the goal of this bug?
Flags: needinfo?(irving)
Can we just iterate over the extensions folder, listing files/dirs?
Probably not. We need records like:

        "places-maintenance@bonardo.net": {
          "userDisabled": false,
          "appDisabled": false,
          "version": "1.3",
          "type": "extension",
          "scope": 1,
          "foreignInstall": false,
          "hasBinaryComponents": false,
          "installDay": 15268,
          "updateDay": 15379
        },
I would not recommend writing this now, while the Addons database is in a bit of flux (likely moving to JSON). But I also wonder if we can just have this sort of data collection only happen when Gecko is running rather than trying to be fancy about it. Then we can basically use the exact same code on Android that we use on desktop...
I would love to not write this now. But FHR is aiming for 23.

It's not just data collection; Fennec will need to know the set of installed add-ons in order to begin collecting FHR data, because it's part of the 'environment'.

(And the desktop code doesn't implement the new contextual storage, as well as being unworkable for some other reasons, not least of which being that we don't ship any of the desktop FHR code on Android.)

So we need to replicate *something*, whether that be a hash of installed add-ons (input to env) or the whole bundle. And given that we also need to report the whole bundle without launching Gecko, we might as well make that replicated state… the whole bundle.

We can do that with some storage and some little replicator, or by opening the Gecko-side storage directly from Java. But we need data that's in the add-on manager.
The active extensions are listed in extensions.ini, so you can get some of the information necessary from there. The best data is available inside Gecko, or by directly reading the main table from extensions.sqlite and/or addons.sqlite; they will be replaced with JSON files by bug 853388 / bug 853389.

There won't be significant changes to this data when gecko isn't running, so pulling the data into FHR when gecko is running and caching it would also work.

I suppose we could split out the bits of bug 853388 that write a JSON copy of the extensions database and land that early; I don't expect changes to the format of that data, so it would be future proof.

I assume you're looking for all forms of add on, not just XPI format? Information for lightweight themes and plugins isn't stored in extensions.sqlite; there may be enough info in addons.sqlite or we may need to look elsewhere.
Depends on: 853388
Flags: needinfo?(irving)
(In reply to :Irving Reid from comment #7)

Thanks for the quick reply!

> I suppose we could split out the bits of bug 853388 that write a JSON copy
> of the extensions database and land that early; I don't expect changes to
> the format of that data, so it would be future proof.

That would work for me. If I were to start hacking on this today, I'd write some JS (probably based on the desktop FHR code) that dumps a JSON copy of installed add-ons on change, and notifies Java accordingly.

(An example of that JSON is in Comment 4.)


> I assume you're looking for all forms of add on, not just XPI format?
> Information for lightweight themes and plugins isn't stored in
> extensions.sqlite; there may be enough info in addons.sqlite or we may need
> to look elsewhere.

We upload some or all of everything returned by AddonsManager.getAllAddons.

We upload full details for all add-ons of type "plugin", "extension", and "service", and increment a counter for all other types.

I'm not sure if that answers your question!
I'm not even sure you can open the DB - if you ever do it when Gecko is running, I think it will fail as we open it in exclusive mode.

If a good-enough approach is acceptable until we get the JSON backend landed, you could look at the following prefs:
* extensions.bootstrappedAddons - all enabled restartless add-ons
* extensions.enabledAddons - all enabled non-restartless add-ons

They don't contain all the info you eventually want (just id, version, type), but it's better than nothing. But again, that's only XPI-based add-ons (ie, no plugins, lightweight themes, or services).

For services, you look at the social.manifest.* prefs.
Failing that, I'd recommend replicating the data when Gecko is running. That's probably the most resilient method long-term anyway, given the requirements.
Yeah, we really want plugins -- they have a large effect on browser health -- and themes. I'll aim to replicate.

Thanks for the input, Blair!
Blocks: 868447
(In reply to Richard Newman [:rnewman] from comment #6)
> I would love to not write this now. But FHR is aiming for 23.

Does the add-on collection block releasing FHR for Android? I'd push for "no".
Also, why does Java need to be notified on changes? If this is a "send data once a day" kind of thing, it doesn't seem critical to let Java know when any changes happen.
Re Comment 12: Add-ons are part of the computed environment for each recorded datapoint, as is CPU, memory, Firefox version, etc. We can omit them for now, but it heavily impacts the validity of our longitudinal analysis: for example, if I install a restartless Crash-a-lot® add-on, it won't cause that day's crashes to be partitioned. Add-ons will be the second thing I attack after core work.

Re Comment 13: we can't have to boot Gecko and init add-ons manager to find out the set of installed add-ons, so we absolutely do need to notify Java on change so that it can persist the current data for later retrieval. This should be infrequent, but it's the crux of how we're going to tackle this.
Depends on: 873360
I implemented pretty much this on top of ProfileInformationCache in Bug 875400. We add a listener for pref and add-on changes in browser.js, passing those through to Java, and storing them in a blob of JSON. Duping this over.
Status: NEW → RESOLVED
Closed: 11 years ago
No longer depends on: 853388
Resolution: --- → DUPLICATE
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.