Closed Bug 720304 Opened 8 years ago Closed 8 years ago

Limit number of processed history and bookmarks records

Categories

(Firefox for Android :: Android Sync, defect, P1)

ARM
Android
defect

Tracking

()

VERIFIED FIXED
mozilla13
Tracking Status
firefox12 --- fixed
blocking-fennec1.0 --- beta+

People

(Reporter: rnewman, Unassigned)

References

Details

Desktop limits both downloads and uploads to 5,000 records. We should do the same or fewer.
Blocks: 722379
Potential issues:

* If Fennec truncates history during insertion during a sync, data loss could occur: history items created on the device could be pruned before they've ever been uploaded. This is very likely on a first sync: we'll be inserting 250 of your really, really frecent history (Twitter, Facebook), and they'll all beat the few pages you've visited on your phone.

=> Fennec must not actively prune.

* If there are lots of new records on the server, and we limit how many we fetch, then it's possible that we won't download a record that we're about to upload. This will cause a record on the server to be overwritten with stale data. We will then rely on some desktop somewhere fixing our mistake, which currently does not occur: Firefox trusts that server data is accurate.

* The local store on the device will converge towards a collection of high-frecency items. (I hope.) This will eventually cause all history fetches to be pointless; they'll be grabbing the 250 most frecent recent items (limit, sort, newer), most of which won't be more frecent than the local data, and will thus be immediately discarded.

In this case it might be better to use index_above to ensure a fetch of only items that are more frecent than Fennec's data, but this would rely on Fennec's conception of frecency being comparable to Firefox's.
Depends on: 723235
Priority: -- → P1
Tacking on a sanity limit for bookmarks, too.

In my code (https://github.com/mozilla-services/android-sync/pull/51) I've set these at 500 history items and 5000 bookmarks, just as a sane upper bound.

XUL Fennec had those limits set at 5,000 for history and no limit for bookmarks, FWIW.
Summary: Limit number of processed history records → Limit number of processed history and bookmarks records
Depends on: 723240
Blocks: 720934
https://hg.mozilla.org/mozilla-central/rev/8d922468a7fe
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
rggh. I somehow botched up my gargantuan testing profile.  It's no longer exceeding the volume of data required to verify this.   I will see if I can track down something to use.   

Are thees limits prefs or hard coded?
(In reply to Tracy Walker [:tracy] from comment #5)

> Are thees limits prefs or hard coded?

There are no prefs on Android. Hard-coded.
I'm seeing only 100 history items on my mobile device. Desktop has much more than that.  I don't have enough bookmarks to get to 5000.  Seeing ~1500 on mobile device.
Blocks: 726045
Ok, the 100 history items is a "view" limit of awesomebar.  Inspecting this closer by searching for older history shows more than that viewed 100.  If I search back far enough for an item existing on desktop, I can verify that there is a limit that isn't large.  I'm not going to go through my history to determine what are the 250th and 251st oldest history item.  Eyeballing it, I would say it is in the correct range.
Status: RESOLVED → VERIFIED
Thanks, Tracy! That sounds good to me.
blocking-fennec1.0: --- → beta+
Target Milestone: --- → mozilla13
Product: Mozilla Services → Android Background Services
Product: Android Background Services → Firefox for Android
You need to log in before you can comment on or make changes to this bug.