1387351 - iOS validation info in the sync ping should include number of bookmarks

Status: RESOLVED FIXED

(Firefox for iOS :: Sync, enhancement)

Description

[Mark Hammond [:markh] [:mhammond]]

When desktop validates bookmarks it records the number of records it considers in a "checked" attribute.

http://searchfox.org/mozilla-central/rev/30a47c4339bd397b937abdb2305f99d3bb537ba6/services/sync/modules/telemetry.js#184

This might help when interpreting validation results (eg, see bug 1386953, where knowing how many bookmarks exist could help prove or disprove some theories)

Comment 1

[Richard Newman [:rnewman]]

There might be a slight ontological mismatch here. iOS has a few different things it can count:

- How many records do we have in the buffer? (count buffer)
- How many records have we successfully synced? (count mirror)
- How many records do we think the server has? (count overlay buffer on mirror, though see the next item)
- How many non-deleted records do we think the server has? (The client keeps all remote deletions until a sync has eventually completed.)

What are we trying to measure?

Comment 2

[Mark Hammond [:markh] [:mhammond]]

(In reply to Richard Newman [:rnewman] from comment #1)
> What are we trying to measure?

I admit that I don't understand the implementation for iOS, but on desktop a key part of validation is grabbing every server record and checking against what we have locally. I hope iOS does something similar as the point isn't to ask "do you think you are consistent?" but "are you actually consistent?". Desktop records the total number of server records including tombstones.

Comment 3

[Richard Newman [:rnewman]]

(In reply to Mark Hammond [:markh] from comment #2)

> I admit that I don't understand the implementation for iOS, but on desktop a
> key part of validation is grabbing every server record and checking against
> what we have locally. I hope iOS does something similar as the point isn't
> to ask "do you think you are consistent?" but "are you actually
> consistent?".

iOS has a full copy of the server state* -- either everything is loaded into the buffer, or the changes since the last sync are loaded into the buffer and the previous server state is in the mirror. That's transparent to the validator.

> Desktop records the total number of server records including
> tombstones.

OK, so this sounds like

  SELECT (SELECT COUNT(*) FROM buffer)
       + (SELECT COUNT(*) FROM mirror WHERE is_overridden = 0)
      AS c;

-- everything in the buffer, deleted or not, plus whatever we have sticking around in the mirror.

This won't count deleted records on the server that have already been merged (we don't keep those) but I think that's OK -- right now we're not merging at all, so we will have those records.



* With one caveat: a totally invalid record that doesn't pass rudimentary validation -- e.g., {'bmkUri': 42} -- will be dropped on the floor to protect the integrity of later parts of the system. We expect this to be rare these days -- we flushed out all the over-strictness cases that we know of -- but we don't report or count those.

Comment 4

[Justin D'Arcangelo [:justindarc]]

NI? from triage

Comment 5

[Edouard Oger [:eoger]]

Attachment: https://github.com/mozilla-mobile/firefox-ios/pull/3044