1288858 - Should custom containers be synced across devices?

Status: NEW

(Core :: DOM: Security, enhancement, P3)

Description

[Tanvi Vyas[:tanvi]]

If a user creates a custom container, should the custom container also show up in Firefox on a different device that is logged into the same sync account?

I think yes.  But I don't know what it would take implementation wise and if it is feasible to change sync to collect this information.  Sync would have to store: int userContextId, int color-or-icon-used, string name-given-by-user

If we allowed for custom icons, then that would get much more complicated, since sync would have to store an icon.  Fortunately, we aren't offering custom icons right now so we don't need need to worry about this.

Comment 1

[Andrea Marchesini [:baku]]

This requires to introduce containers in FF for Android, does it?

Comment 2

[Richard Newman [:rnewman]]

It doesn't necessarily involve supporting containers on iOS or Android (neither of which use Places, and iOS doesn't even use Gecko).

It requires defining a cross-platform future-proof definition for a container definition, and defined behaviors for what happens when they disappear or change (e.g., if I upload a record with the same GUID but a different numeric ID, must this client re-map?).

Further, if this is a dependent engine for another datatype, we'd need to figure out how that works in Sync; it's never been done before.

iOS and Android would need to be able to round-trip data -- that means that if container references would be used in Sync's formats for bookmarks or history in the future, those platforms' data stores would need to be made aware of them.

So, some questions:

* What's the real, cross-platform definition of a container? The icon and name? What defines the icon? Can new icons be added (even if not custom)? What does a platform do when it doesn't have that icon? Are names localized? If my desktop is running Firefox in English, and my phone is in Armenian, should they both show "Work", or should my phone show "աշխատանք"?

* Are these things immutable, or can they change?

* Does the integer value need to be synced? Can that ever change? If it changes, what would happen?

* Are containers only going to be used for History, or might they also apply to bookmarks, form data, or something else?

Comment 4

[Jonathan Kingston [:jkt] he/him]

Richard missed this Need Info for tanvi.

> * What's the real, cross-platform definition of a container? The icon and name? What defines the icon? Can new icons be added (even if not custom)? What does a platform do when it doesn't have that icon? Are names localized? If my desktop is running Firefox in English, and my phone is in Armenian, should they both show "Work", or should my phone show "աշխատանք"?

Containers are a unique storage isolation for browser history, cookies, localstorage which resolves to a unique id.

A synced setup might look like:
- If a user browses to google.com in personal, their other machine would contain history in their personal container for google.com
- If a user renames Personal to MyStuff, their other machine would also contain MyStuff and not Personal.

Icon and Name are changeable whenever the user wants, there are a set of a few icons to pick from and they can rename the container.
Currently the layout shows no icon where there isn't that icon. This happens sometimes in menus that are newer in Nightly that use container icons and our experiment hasn't fixed them with our new icon set. This would be the same issue if users didn't have the icons (assuming you are accounting for Firefox ESR).
Perhaps if this became an issue long term we would have to push new icons to older versions.

By default users get the first 4 containers localised, if you rename "work" to "workthings" we would likely have to take that over the Armenian, however up until that point the containers name is stored as the e10s string which would represent the same container.

I suspect syncing containers to be similar to syncing bookmark labels, if the user changes it on one device all devices would reflect the same change.


> * Are these things immutable, or can they change?

The IDs are unique per profile, same as the other attributes pinned to this ID, these properties like name and colour are mostly unimportant at the moment. However in our experiments we are using assignment of URLs to an ID also.


> * Does the integer value need to be synced? Can that ever change? If it changes, what would happen?

If the ID changed of a container then the cookies and storage would no longer work.


> * Are containers only going to be used for History, or might they also apply to bookmarks, form data, or something else?

This is very likely, bookmarks is the most requested feature certainly.

Comment 5

[Jonathan Kingston [:jkt] he/him]

Let me know if you have further questions.

Comment 6

[Richard Newman [:rnewman]]

(In reply to Jonathan Kingston [:jkt] from comment #4)

> Icon and Name are changeable whenever the user wants, there are a set of a
> few icons to pick from and they can rename the container.

Should icons sync between devices?

Can the set of icons ever change? Can they be supplied by the user, or only picked from an enumeration? Will icons ever be removed from the distribution?

> By default users get the first 4 containers localised, if you rename "work"
> to "workthings" we would likely have to take that over the Armenian, however
> up until that point the containers name is stored as the e10s string which
> would represent the same container.

To make sure I understand you:

- If I never impose a name on a container, and I have two devices in two locales, the name of the container is different on each device, and is the localized default name.
- If I impose a name, it is the same on all devices.


> The IDs are unique per profile, same as the other attributes pinned to this
> ID, these properties like name and colour are mostly unimportant at the
> moment. However in our experiments we are using assignment of URLs to an ID
> also.

If I create two profiles on two different devices, do they have the same ID for their 'work' container?

If I delete and recreate the work container, what happens? (And what happens to the history visits etc. tagged with that collection?)

  -- can containers ever be deleted from a data model perspective?

If I create a new container, "Foo", does it (should it?) have the same ID as a "Foo" container created on another device?

If I create a container "Foo" on one device, and "Foo" on another device, do you expect them to be merged when synced? Or can I have two containers with the same name?

If I create two containers on different devices, and I rename one of them to have the same name, do you expect them to merge? 

If I have "Work" on one device, and "Mozilla" on another, can I merge them after setting up Sync?


> > * Does the integer value need to be synced? Can that ever change? If it changes, what would happen?
> 
> If the ID changed of a container then the cookies and storage would no
> longer work.

Depending on your answers to the above questions, containers might need to be renumbered, or a conceptual 'container' might need to have more than one ID. There's no way to merge two containers without somehow reconciling the identifiers.


> > * Are containers only going to be used for History, or might they also apply to bookmarks, form data, or something else?
> 
> This is very likely, bookmarks is the most requested feature certainly.

Did you consider any other implementation approaches other than adding an integer to (potentially) every data type in the profile?

Has anyone asked for syncing different containers to different accounts? E.g., having my home computer have a "work" container that syncs with my work computer, and a "personal" container that doesn't?

Comment 7

[Jonathan Kingston [:jkt] he/him]

> Should icons sync between devices?

Yes it's how users identify their containers along with colour.

> Can the set of icons ever change?

Yes, they changed for the test pilot. They may change again when evolving based on user needs.

> Can they be supplied by the user, or only picked from an enumeration?

Currently only from an enumerated set

>  Will icons ever be removed from the distribution?

Possibly also.

> - If I never impose a name on a container, and I have two devices in two locales, the name of the container is different on each device, and is the localized default name.
> - If I impose a name, it is the same on all devices.

This is currently how the code is yes the initial strings are stored as i18n identifiers.

> If I create two profiles on two different devices, do they have the same ID for their 'work' container?

Currently the initial profile set would have 1-4 containers which would all be named the same yes.

> If I delete and recreate the work container, what happens? (And what happens to the history visits etc. tagged with that collection?)

I don't think we clear the history of deleted containers, I would have to check. However if you deleted work it would likely go from id 2 to id 5 (or the next id available).

>  -- can containers ever be deleted from a data model perspective?

I'm not sure I understand the question?

> If I create a new container, "Foo", does it (should it?) have the same ID as a "Foo" container created on another device?

Not really, if you have sync setup the container name should already exist in your container list. Creating another container with the same name would have two separate containers with the same name. Currently we don't restrict users doing this or even warn them.

> If I create two containers on different devices, and I rename one of them to have the same name, do you expect them to merge?

Personally I wouldn't, especially as the history would all be tagged to different IDs

> If I have "Work" on one device, and "Mozilla" on another, can I merge them after setting up Sync?
 
Currently we don't have a merge feature. It would require merging histories and storage etc. Maybe this could be a manual step later. We certainly couldn't split containers later. There is also the issue of conflicting cookies from the same domains to resolve if we went down the merging front. Maybe we could restrict merging to cross device perhaps?

> Depending on your answers to the above questions, containers might need to be renumbered, or a conceptual 'container' might need to have more than one ID. There's no way to merge two containers without somehow reconciling the identifiers.

Yeah I can imagine that sync might need to capture other device containers that might be the same. We could potentially remap all containers to new id's when sync is setup which would involve changing all storages to have the new id.

> Did you consider any other implementation approaches other than adding an integer to (potentially) every data type in the profile?

My assumption was it would be the integer again yeah this is how we tag everything else.

> Has anyone asked for syncing different containers to different accounts? E.g., having my home computer have a "work" container that syncs with my work computer, and a "personal" container that doesn't?

Yes we have had this request however I personally think it adds a lot more complexity that we don't need right now.

Comment 8

[Richard Newman [:rnewman]]

(In reply to Jonathan Kingston [:jkt] from comment #7)
> > Should icons sync between devices?
> 
> Yes it's how users identify their containers along with colour.

I mean: let's say you have two devices, one running Firefox 61, and one running Firefox 62. This is very common.

Firefox 62 introduces a "Travel" container, with an icon of an aircraft. You visit a bunch of travel sites on your 62 device.

Those sync to Firefox 61. Your 61 device is now aware of a Travel container, but Firefox 61 doesn't include the iconography, because it shipped two months earlier.

Hence my question: do we sync the icons? The alternative is to show no icons until you upgrade the software, or to use a different mechanism (e.g., emoji) for icons.


There are a whole set of domain concepts here:

- Icons, which can change between versions and by user action
- Names, including default and localized names, which can change between versions and by user action
- The default container set, which can change between versions
- History visits
- Cookies
- …

Each of these has their own life cycle, their own distribution mechanism, and their own data ownership.

The reason for all of my questions is to establish these scopes, so that we can define the appropriate behaviors when users install different versions of Firefox, downgrade Firefox, delete and recreate and rename containers, wipe their server data, sync down to a clean profile, etc.


> > If I delete and recreate the work container, what happens? (And what happens to the history visits etc. tagged with that collection?)
> 
> I don't think we clear the history of deleted containers, I would have to
> check. However if you deleted work it would likely go from id 2 to id 5 (or
> the next id available).
> 
> >  -- can containers ever be deleted from a data model perspective?
> 
> I'm not sure I understand the question?

These two questions are related.

If I had 'Work' visits tagged with ID '2', and I delete the 'Work' container, are those visits still tagged '2', or do we reset them to a different container, or delete them?

What does '2' mean if I delete the container with ID '2'?

What happens when those visits sync, each tagged '2'?

What happens to the tens of thousands of records on your Sync server, potentially tagged '2'? What happens if I then re-create the container?

If we locally drop those visits _and only those visits_ on the floor after deleting 'Work', we have no way in Sync to represent that -- it can only represent deletion of a URI and all of its visits.

If we re-container those visits, we need to reupload the record, and perhaps _still_ don't know how to represent that without rewriting the history data format on all three platforms.

That is: there is a foreign key relationship between visits and containers, and some of the operations we support violate that relationship.

If I delete the 'Work' container, one reasonable approach is to _hide_ the container, so the data is still consistent… but it'll still pop up on your other devices. 

Another approach is to move every Work-tagged data item into the default container so that the other can be deleted… but that requires reuploading tens of thousands of history records to refer to a different container, and doing the Sync work to make that a meaningful concept.

Furthermore, changes to data and containers need to be atomic, otherwise another device will see old history which refers to a container that isn't present on the server.

How containers are modeled has potentially huge implications on how they are synchronized and how expensive all of these user actions are.


> Personally I wouldn't, especially as the history would all be tagged to
> different IDs

Consider this situation.

I set up my work laptop. I create a 'Travel' container. (It happens to have ID 5.)

I set up my home laptop. I also create a 'Travel' container. (It happens to have ID 7.)

I discover Firefox Sync, and set up my Firefox Account. I now have two containers named 'Travel', two named 'Work', two named 'Personal', ….

We can't merge on ID, because integer IDs are arbitrary -- '5' on my home laptop is "Consulting".

If we merge on name, we'll need to renumber data, potentially reuploading it, depending on how we model containers on the server.


> Maybe we could restrict merging to cross device perhaps?

Neither Sync nor the container mechanism have any notion of a device. If you sign out and sign back in to the same account, Firefox Sync will re-process all of the data that any device ever uploaded, in one undifferentiated lump.


> > Did you consider any other implementation approaches other than adding an integer to (potentially) every data type in the profile?
> 
> My assumption was it would be the integer again yeah this is how we tag
> everything else.

Unfortunately, tagging data with arbitrary integers introduces substantial coupling across data stores, and is -- as you can probably tell from these questions -- quite complicated to synchronize, particularly with a record-oriented store like Sync. This kind of trouble is why we don't synchronize the internal Places IDs, and instead use stable GUIDs.

Another approach would be to model each container as a discrete data store -- e.g., to maintain a separate visits table for each container, or a separate cookie store for each container. Synchronization could then take place against discrete server collections; renaming and merging containers would then be a (re-)association problem, and the data model itself wouldn't change. It's a relatively small step from there to syncing your work data with your work Firefox Account, just like Chrome's user switching.

If synchronization is important to you, and if the set of datatypes that you want to associate with containers is likely to grow, then you might want to re-evaluate your data modeling approach to better reflect the lifecycles of synced data.

Otherwise, it sounds like syncing is _possible_, albeit fiddly, but very difficult in the presence of changing containers; whoever ends up doing this work will need to carefully coordinate how data is associated with a container across server collections.

Comment 9

[Jonathan Kingston [:jkt] he/him]

I mean I don't think we are that coupled to the IDs being used. The issue as I see it are that we expose this to web extensions also.

From your comments it seems that having local IDs for containers makes the most sense based upon their current setup and how the code currently works. Then sync can use a GUID across machines and we could have a mapping table to correlate all GUIDS that correlate to the same ID.

This would likely look like this:

Machine     | Container | ID | GUID
Home Desktop| Work      |  2 | 840e56e4-e48c-46d5-a2a9-2004382fc284
Work Laptop | Work      |  7 | 887df67c-54c8-4284-ab73-a248db7a8887
Home Desktop| Personal  |  1 | 236fd266-6a13-4a8b-9ab7-14e390135cfa
Work Laptop | Stuff     |  1 | b0ffb563-e796-4bc9-9dcf-e10dc7497061

Mapping:
Home Desktop:
ID | GUID
1  | b0ffb563-e796-4bc9-9dcf-e10dc7497061
2  | 887df67c-54c8-4284-ab73-a248db7a8887


Work laptop:
ID | GUID
1  | 236fd266-6a13-4a8b-9ab7-14e390135cfa
7  | 840e56e4-e48c-46d5-a2a9-2004382fc284

This way the devices know the GUID sync ID of the other devices and able to map different device IDs to different computer container IDs.

> Another approach would be to model each container as a discrete data store -- e.g., to maintain a separate visits table for each container, or a separate cookie store for each container.

I don't think changing how all storages are isolated. This is a lot more like profiles and defeats the advantages we have. Maybe this could be a long term goal but the simplicity comes from creating a container is just an ID.

Reshaping how the browser uses IDs seems like a lot of reengineering over perhaps carefully building a mapping table which seems how places does it.

Comment 10

[Jonathan Kingston [:jkt] he/him]

I guess we could provide some UI for first sync to ask if they want to sync by name with a box ticked by default. I'm not entirely sure on the UX for this. It seems like something we should ask UX how enabling sync would work.

I will share the problem with our designer to see if they could think of some solutions for this when they have some spare cycles.

I think I answered all of your questions, will check tomorrow.

Comment 11

[Mark Hammond [:markh] [:mhammond]]

(In reply to Jonathan Kingston [:jkt] from comment #9)
> From your comments it seems that having local IDs for containers makes the
> most sense based upon their current setup and how the code currently works.
> Then sync can use a GUID across machines and we could have a mapping table
> to correlate all GUIDS that correlate to the same ID.

I don't see how this helps - all you are doing is assigning a random GUID to an integer that has, effectively, a random meaning when considering multiple devices. For example, it doesn't help answer what should be done when a user renames a context.

Such a mapping only makes sense when a container of "Work" has a consistent GUID across all devices, and it's impossible for that  GUID to ever mean anything other than "Work" on any device.

> Maybe this could be a long
> term goal but the simplicity comes from creating a container is just an ID.

While there was clearly local simplicity in that decision, the complexity we are discussing here arises precisely from that same decision.

> perhaps carefully building a mapping table which seems how places does it.

That's not how places does it - your proposal would be analogous if places allowed you to "rename" a URL giving it new canonical "ID" and have it treated identically on multiple devices, even if it was given a different ID on each. Places doesn't allow that, and if it did, I'm missing how that mapping would work in practice.

(In reply to Jonathan Kingston [:jkt] from comment #10)
> I think I answered all of your questions, will check tomorrow.

I might be missing some finer points of the mapping proposal, but I don't think it addresses the points Richard raised at all (eg, renaming, train skipping, reconcilliation, etc)

Comment 12

[Morpheus Chen [:morpheus] UX]

After consulted with John, who is the PM for Containers, our conclusion for this bug is "pending." We will revisit this bug according to the result of the Shield study.

Comment 13

[Firefox Bug Husbandry Bot]

Bulk change per https://bugzilla.mozilla.org/show_bug.cgi?id=1401020

Comment 14

[Antoine Turmel [:GeekShadow]]

In my user case, I want to have the same Firefox desktop experience both on my computer and laptop (and across Linux & Windows)
So the ability to sync containers would be nice (along with search providers, see bug 1407859)

Comment 16

[omrigold]

I want to second the importance of this feature.

I use versions of Firefox across PC desktop (home), Macbook Pro (work), Android, iPad, and occasionally Linux. Multi-Account Containers are a massive privacy win over other browsers. However, if I spent all my time setting up those containers on my PC, I expect them to sync to my Mac.

Seems like an oversight. Is there a technical reason this isn't yet offered or it has simply been de-prioritized? Thanks

Comment 17

[paul.pehrson]

As a recently re-converted Firefox user, containers was one of the killer features for privacy for me. The fact that I can't sync the settings between desktop/laptop devices is intensely frustrating. This bug has been open for years, and not even an extension developer has solved it! (The only extension that tries, synchronizes the containers, but not the websites assigned to the containers, which is the whole point of synchronization.)

Comment 18

[Luke Sheldrick]

Was there any update after the testing? is this going to happen?

I notice the third party plug in 'containers sync' has been pulled.

Without this, this feautre/product is totally crippled. I use it extensively and thing it's a brilliant solution (I'm in and out of Microsoft tenants all day and is a perfect solution). However not being able to Sync or even import/export is a killer.

Comment 19

[Ryan Kelly [:rfkelly]]

Cross-linking for context, the "multi-account containers" adding is working on sync support via the chrome.storage.sync webextension API here: https://github.com/mozilla/multi-account-containers/pull/1611

Comment 20

[Kagami Rosylight [:saschanaz] (they/them)]

The PR has been merged and it's working now. Does it make this issue fixed?