1214790 - Come up with a data migration plan from XUL/SDK to WebExtensions

Status: RESOLVED FIXED

(WebExtensions :: Untriaged, defect)

Description

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Let's say you have a XUL or SDK add-on right now and there's user data stored in simple-storage, preferences, or something like that. Then you have a new version of your add-on that's a WebExtension. Now you can store your data using localStorage or browser.storage. But right now there's now way to migrate user data, so it will all be lost as part of the upgrade. We should fix this.

One idea would be to allow the old extension to store some data in a place that's accessible to WebExtensions. Then the developer could publish an intermediate version that would change the data format before switching to WebExtensions.

Comment 1

[Blake Winton (:bwinton) (:☕️)]

Could existing extensions store their data in localStorage or browser.storage, for later getting out by the WebExtension version?
(If so, or if that's something we can enable, we should do that pretty quickly to give people a chance to react, I think.)

Comment 2

[Johann Hofmann [:johannh]]

> One idea would be to allow the old extension to store some data in a place that's accessible to 
> WebExtensions. Then the developer could publish an intermediate version that would change the data format 
> before switching to WebExtensions.

> Could existing extensions store their data in localStorage or
> browser.storage, for later getting out by the WebExtension version?

I think a problem that many add-on authors would be facing with that is that they'd have to push a new version through AMO review before they can ever publish their WebExtension update (which will hopefully be way easier to get reviewed).

Another idea would be some (read-only) API to access simple-storage in WebExtensions. But I can see how that might not be beautiful to implement.

Comment 3

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

(In reply to Blake Winton (:bwinton) (:☕️) from comment #1)
> Could existing extensions store their data in localStorage or
> browser.storage, for later getting out by the WebExtension version?

We'd have to do some extra work to enable that. Right now everything is stored based on a per-install UUID, which the pre-WebExtension add-on won't have.

> (If so, or if that's something we can enable, we should do that pretty
> quickly to give people a chance to react, I think.)

Thinking about this more, it would really be better if the user didn't have to use a recent version of the XUL add-on before updating to the WebExtension. If someone didn't use Firefox for a while, they might "miss" the migration version of the XUL extension and suffer data loss. So it would be nicer if the WebExtension could include some extra migration code that would be ignored if you're not upgrading. This code would have access to XPCOM.

Comment 4

[Thomas Oberndörfer]

(In reply to Johann Hofmann from comment #2)
> Another idea would be some (read-only) API to access simple-storage in

For us write access to simple-storage would be also important to be able to delete the data after migration.

Comment 5

[Erik Vold]

There should be an easy way for add-on developers to write tests to verify that these upgrades work correctly... which might depend on bug 1223616

Comment 6

[sblask]

I think it would be great to have something in WebExtensions that is compatible with simple-preferences. Not having simple-preferences would be quite a step backwards. Now you just have to declare and use, no need for custom ui and storage and the look and feel is the same for all extensions using simple-preferences...


That would solve the data migration problem at least for those who use that. And I doubt there will be an easy solution for someone who did something very custom. This bug is kind of very broad, maybe there should be sub-tickets for different use cases?

Comment 7

[Andy McKay]

We've got the embedded extension which allows this to happen. There hasn't been much (well any) contributions come in for accessing simple-preferences so I'm thinking at this point this is the best bet for developers. Based on that I think we should close this.

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Embedded_WebExtensions

Comment 8

[l.mai]

(In reply to Andy McKay [:andym] from comment #7)
> We've got the embedded extension which allows this to happen. There hasn't
> been much (well any) contributions come in for accessing simple-preferences
> so I'm thinking at this point this is the best bet for developers. Based on
> that I think we should close this.

That sounds like "we don't have a good solution to this problem, so let's just close it" to me. It's unsatisfying.

Also, this issue is still unaddressed:

(In reply to Bill McCloskey (:billm) from comment #3)
> Thinking about this more, it would really be better if the user didn't have
> to use a recent version of the XUL add-on before updating to the
> WebExtension. If someone didn't use Firefox for a while, they might "miss"
> the migration version of the XUL extension and suffer data loss. So it would
> be nicer if the WebExtension could include some extra migration code that
> would be ignored if you're not upgrading. This code would have access to
> XPCOM.

Comment 9

[Andy McKay]

> Also, this issue is still unaddressed:
> 
> (In reply to Bill McCloskey (:billm) from comment #3)
> > Thinking about this more, it would really be better if the user didn't have
> > to use a recent version of the XUL add-on before updating to the
> > WebExtension. If someone didn't use Firefox for a while, they might "miss"
> > the migration version of the XUL extension and suffer data loss. So it would
> > be nicer if the WebExtension could include some extra migration code that
> > would be ignored if you're not upgrading. This code would have access to
> > XPCOM.

Well they wouldn't technically suffer data loss, the data is still their in the profile its just that its not accessible to the new add-on so that means that are multiple work arounds available (e.g. going back versions, converting manually). Part of the problem is that legacy add-ons store their data all over the place so even just adding in simple-storage fixes some use cases but not all of them.

Comment 10

[Andy McKay]

We have embedded extensions with examples and documentation.

I filed https://github.com/mozilla/addons-server/issues/4129 to request AMO to force an embedded extension update.