Finish Sync state machine: partial server, blank server, node reassignments

VERIFIED FIXED

Status

()

Firefox for iOS
Sync
VERIFIED FIXED
3 years ago
2 years ago

People

(Reporter: rnewman, Assigned: nalexander)

Tracking

(Depends on: 1 bug)

unspecified
All
iOS
Dependency tree / graph

Firefox Tracking Flags

(fxios1.1+)

Details

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
We currently support the success path through the state machine: download meta/global, download crypto/keys, start syncing.

We haven't finished implementing the rest:

* No meta/global.
* Changed meta/global (syncID, version, per-engine syncID, per-engine version).
* Changed set of engines.
* No keys.
* Changed keys (including added collection keys).
* Error states.


These will be necessary before users "in the wild" can use Sync.
(Reporter)

Updated

3 years ago
Summary: Finish Sync state machine: partial server, blank server → Finish Sync state machine: partial server, blank server, node reassignments
(Reporter)

Updated

3 years ago
Depends on: 1167738
(Reporter)

Updated

3 years ago
Blocks: 1168512
(Reporter)

Updated

3 years ago
tracking-fxios: --- → ?

Updated

3 years ago
tracking-fxios: ? → +
(Reporter)

Updated

3 years ago
tracking-fxios: + → 1.1+
(Reporter)

Updated

3 years ago
Blocks: 1193612
(Reporter)

Updated

3 years ago
Blocks: 1196241
(Reporter)

Updated

3 years ago
Blocks: 1196256
(Reporter)

Updated

3 years ago
Blocks: 1196258
(Reporter)

Updated

3 years ago
Whiteboard: [nicetohave105]
(Reporter)

Updated

3 years ago
Assignee: nobody → rnewman
Status: NEW → ASSIGNED
Is this a must have for 1.1 and is it related to signup?
Flags: needinfo?(rnewman)
(Reporter)

Comment 2

3 years ago
I wrote some words that outline what needs to happen here. Reproduced for posterity.

https://github.com/mozilla-services/syncto/issues/22#issuecomment-141523612


The token server is responsible for taking an assertion and a client state — `X-Client-State`, derived from `kB` — and returning an `api_endpoint` and a token to use with it.

The API endpoint (Sync server) might be new or it might be a previous one. It might be the same physical server but with a different UID. The client doesn't care.

X-C-S is there to make sure that a client with a new sync key (e.g., after a password change) never ends up at the same server+uid combination; this avoids some classes of data integrity bugs.

Populating a blank server is an involved and slightly risky business.

There are a few definitions of 'blank':

* Missing everything.
* Missing data, but has `m/g` and `c/k`.
* Missing `c/k`, but has `m/g`.
* Potentially: missing `m/g` but has other stuff.

There are two ways of detecting 'blank':

* Not present in `info/collections`.
* 404 on fetch.
* Record fetches, but is malformed or encrypted with the wrong key.

A client needs to get the server into a consistent state, which might involve wiping `/storage/` in order to e.g., delete data that might be encrypted with a lost key.

The client needs to make sure that the server ends up with four things:

* A meta/global with the right engine selections and declined list.
* A `crypto/keys` object, encrypted with the correct `kB`, containing a (new, I think) bulk key bundle.
* Our encrypted client record.
* Collection data encrypted with the right keys from the bulk key bundle. This might take some time to upload.

The reason this is tricky: doing this involves 4+ HTTP requests, including potentially very slow bulk deletions, and clients need to be careful to avoid races. For example, two clients might race with meta/global, leading to a collection being written without having an engine entry, and two clients might race with key upload, leading to collection data being encrypted with the wrong bulk keys (eek!).

`X-If-Unmodified-Since` and careful re-fetches is the right approach. It's not as simple as "oh, it's an empty server, I'll run four `PUT`s".

The above is a fairly complete picture of what's supposed to happen, but I recommend you read the Android Sync source, and maybe the desktop `service.js` code to get more angles.
Flags: needinfo?(rnewman)
Whiteboard: [nicetohave105]
(Reporter)

Comment 3

3 years ago
(In reply to Stephan Leroux [:sleroux] from comment #1)
> Is this a must have for 1.1 and is it related to signup?

When we say "support sign up for a Firefox Account" (Bug 1196241), what we really mean is:

We removed the sign up links from FxA and empty panels because the Sync state machine doesn't support populating a blank server, and certain other edge cases.

Those omissions lead to a bad user experience after creating an account: a user with two iOS devices won't see them syncing until they connect a desktop or Android device.

We need to finish the state machine so we can deal with a blank server correctly; then we can restore those links.

Given that we want to support account creation for 1.1, this bug is the real meat. The blocked bugs are mopping up when we're done.
(Reporter)

Comment 4

3 years ago
Nick wanted to get to this towards the end of September.
(Reporter)

Updated

3 years ago
Depends on: 1207203
(Reporter)

Updated

2 years ago
Assignee: rnewman → nalexander
(Reporter)

Updated

2 years ago
Depends on: 1210625

Updated

2 years ago
No longer depends on: 1210625

Updated

2 years ago
Depends on: 1210625
(Assignee)

Comment 5

2 years ago
Created attachment 8669148 [details] [review]
Link to Github pull-request: https://github.com/mozilla/firefox-ios/pull/1115
Attachment #8669148 - Flags: review?(rnewman)
(Reporter)

Comment 6

2 years ago
Comment on attachment 8669148 [details] [review]
Link to Github pull-request: https://github.com/mozilla/firefox-ios/pull/1115

r+ with comments addressed. Excellent work as always; very intrinsically complicated stuff reduced to clarity.
Attachment #8669148 - Flags: review?(rnewman) → review+
(Reporter)

Updated

2 years ago
OS: iOS 8 → iOS
Duplicate of this bug: 1196256
(Assignee)

Updated

2 years ago
Depends on: 1211682
(Assignee)

Comment 8

2 years ago
Created attachment 8670545 [details] [review]
Link to Github pull-request: https://github.com/mozilla/firefox-ios/pull/1134

Back to you.  I haven't done any in-App testing; figure you can look at the changes, especially around `LocalCommand`, and then I can do some final testing before landing tomorrow AM.
Attachment #8670545 - Flags: review?(rnewman)
(Assignee)

Updated

2 years ago
Depends on: 1212426
(Assignee)

Updated

2 years ago
Depends on: 1212431
(Assignee)

Updated

2 years ago
Blocks: 1139045
(Reporter)

Updated

2 years ago
No longer depends on: 1212431
(Reporter)

Comment 9

2 years ago
Comment on attachment 8670545 [details] [review]
Link to Github pull-request: https://github.com/mozilla/firefox-ios/pull/1134

Nits on the PR. Next up: responding to the changes!
Attachment #8670545 - Flags: review?(rnewman) → review+
(Assignee)

Comment 10

2 years ago
https://github.com/mozilla/firefox-ios/commit/82fb1cfc80fdba9961224ef650f1971f4bb6335d
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
(Assignee)

Comment 11

2 years ago
AaronMT: this could use some focus testing.

You should be able to create a Firefox Account on iOS, and start syncing correctly.  The Choose What to Sync button is not yet functional, and there is no UI showing you what is currently syncing.  (That's 1.2 territory).

If you sync against Desktop, you should see everything syncing initially.  If you turn things off and on in Desktop, you should see your iOS device responding (although not reflecting the changes anywhere in the UI; and remote tabs, bookmarks, etc are not yet purged when an engine is disabled: see Bug 1212431).
Flags: needinfo?(aaron.train)
(Reporter)

Updated

2 years ago
Depends on: 1213534

Updated

2 years ago
Status: RESOLVED → VERIFIED
Flags: needinfo?(aaron.train)
You need to log in before you can comment on or make changes to this bug.