998077 - [meta] Sync support for autofill profile data

Status: RESOLVED DUPLICATE of bug 1361010

(Firefox :: Sync, defect, P3)

Description

[Brian Nicholson (:bnicholson)]

Currently, the plan for bug 939351 is to use a JSON file that can be synced across devices. The format might look something like this:

{
  address: {
    mtime: <timestamp>
    entries: [
      {
        guid: <guid>,
        mtime: <timestamp>,
        lastUsed: <timestamp>,
        data: {
          // autofill address fields
        }
      }
    ]
  },

  payment: {
    mtime: <timestamp>
    entries: [
      {
        guid: <guid>,
        mtime: <timestamp>,
        lastUsed: <timestamp>,
        data: {
          // autofill payment fields
        }
      }
    ]
  }
}

* All platforms are expected to use the same autofill provider/JSON storage module (no platform-specific database implementations)
* We'll want conflict resolution to prevent autofill data from being clobbered across devices, thus the guid/mtime fields for each entry.
* The UI will want to know the recency of entries to determine order for the autofill list, thus the lastUsed fields for each entry.
* We want to prevent unnecessary syncing of unchanged data, thus the mtime field for the top-level address/payment containers. Any mtime updates to children will bubble to the parent.
* We'll need a way to keep track of deleted data. Similar to existing synced DBs, Nick suggested using a tombstone object with a "deleted" property to keep track of deletions.
* Since we don't want Sync to be dependent on Gecko to access/modify the JSON database, we'll need to use a file locking mechanism to prevent concurrent reads/writes.

Comment 1

[Richard Newman [:rnewman]]

Morphing this into a meta bug, which will handle defining the record format. We can file new bugs for each supported platform.


> Currently, the plan for bug 939351 is to use a JSON file that can be synced
> across devices. The format might look something like this:

Note that Sync is record-oriented. If you can split data into individual records that change, great. If not, there'll be one big record, and you'll have to put more effort into conflict resolution.



> * All platforms are expected to use the same autofill provider/JSON storage
> module (no platform-specific database implementations)

Is this a hard requirement? If so, where from?

We're planning to move in the opposite direction for passwords on Android.


> * We'll want conflict resolution to prevent autofill data from being
> clobbered across devices, thus the guid/mtime fields for each entry.
> ...
> * We want to prevent unnecessary syncing of unchanged data, thus the mtime
> field for the top-level address/payment containers. Any mtime updates to
> children will bubble to the parent.

This is largely handled for you by Sync, but you need to get the record abstraction right. All Sync records already have a modification time, which is a server timestamp corresponding to the instant of upload.

If you additionally want to layer on a secondary conflict detection approach, you need to figure out a scheme for correcting for clock skew, or declare that you aren't worried about (potentially huge) skews.


> * The UI will want to know the recency of entries to determine order for the
> autofill list, thus the lastUsed fields for each entry.

Sync also has a concept of a sortindex, which will give you slightly more responsive syncing behavior. You can reuse your recency for this.


> * We'll need a way to keep track of deleted data. Similar to existing synced
> DBs, Nick suggested using a tombstone object with a "deleted" property to
> keep track of deletions.

Yep, that's how Sync does it.


> * Since we don't want Sync to be dependent on Gecko to access/modify the
> JSON database, we'll need to use a file locking mechanism to prevent
> concurrent reads/writes.

I'd suggest going the other way on Android: on all platforms, abstract away storage ("AutofillStorage"). On desktop, it can directly read and write a file. On Android, it would talk to a Java ContentProvider.

That is, unless you're very confident in your ability to get file locking semantics right :D

Comment 2

[Richard Newman [:rnewman]]

There's one additional concern here: if you're adding a new Sync engine, we need a plan for surfacing that to users. If you're not adding this as a top-level checkbox (e.g., you're adding this as a sibling to form history sync), then that has a different, smaller set of things to tackle.

Comment 3

[Mark Hammond [:markh] [:mhammond]]

While bug 1361010 post-dates this one, bug 1361010 has more relevant and up-to-date information on what needs to be done for Sync.