459281 - index/work with sub entries by add chunk id, not host key

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, defect)

Description

[Dave Camp (:dcamp)]

Attachment: wip

A new rev of the safebrowsing protocol (v2.2) will be coming shortly.  This is a fairly minor update.

Attached patch handles the only significant change - applying an add chunk should expire all subs that apply to that add chunk, even if they don't actually match an add entry.  This required adding an index on moz_subs, but allowed us to remove an index too.  In my measurements against a v2.1 server, this speeds up an update by about 30%, and has about the same filesize.  I'll need to re-measure that with the new server once it's available, though.

Removing the domain index from subs makes domain queries against subs really slow.  The patch moves any methods that rely on that index to nsUrlClassifierAddStore to prevent accidentally relying on them.

There are also a few tiny fixes to the stream updater that I needed to get the performance testing right.

This patch resets the db, so isn't really appropriate for a 3.0.x release.  We'll need a scaled-back version for 3.0.x.

Comment 1

[Dave Camp (:dcamp)]

Attachment: v1

OK, here's a completed version of the earlier patch.

Comment 2

[Tony Chang (Google)]

Comment on attachment 343658 [details] [diff] [review]
v1

looks good to me

Comment 3

[Dave Camp (:dcamp)]

Updating the summary since I didn't do the actual 2.1->2.2 protocol switch in this patch (and the changes in the patch are valid for 2.1 too).  I'll open a new bug for the 2.1->2.2 switch

Comment 4

[Dave Camp (:dcamp)]

Pushed as http://hg.mozilla.org/mozilla-central/rev/7825b962281d