index/work with sub entries by add chunk id, not host key

RESOLVED FIXED

Status

()

Toolkit
Safe Browsing
RESOLVED FIXED
10 years ago
4 years ago

People

(Reporter: dcamp, Unassigned)

Tracking

({perf})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

v1
26.25 KB, patch
Tony Chang (Google)
: review+
Details | Diff | Splinter Review
(Reporter)

Description

10 years ago
Created attachment 342482 [details] [diff] [review]
wip

A new rev of the safebrowsing protocol (v2.2) will be coming shortly.  This is a fairly minor update.

Attached patch handles the only significant change - applying an add chunk should expire all subs that apply to that add chunk, even if they don't actually match an add entry.  This required adding an index on moz_subs, but allowed us to remove an index too.  In my measurements against a v2.1 server, this speeds up an update by about 30%, and has about the same filesize.  I'll need to re-measure that with the new server once it's available, though.

Removing the domain index from subs makes domain queries against subs really slow.  The patch moves any methods that rely on that index to nsUrlClassifierAddStore to prevent accidentally relying on them.

There are also a few tiny fixes to the stream updater that I needed to get the performance testing right.

This patch resets the db, so isn't really appropriate for a 3.0.x release.  We'll need a scaled-back version for 3.0.x.

Updated

10 years ago
Keywords: perf
(Reporter)

Comment 1

10 years ago
Created attachment 343658 [details] [diff] [review]
v1

OK, here's a completed version of the earlier patch.
Attachment #342482 - Attachment is obsolete: true
Attachment #343658 - Flags: review?(tony)

Comment 2

10 years ago
Comment on attachment 343658 [details] [diff] [review]
v1

looks good to me
Attachment #343658 - Flags: review?(tony) → review+
(Reporter)

Comment 3

10 years ago
Updating the summary since I didn't do the actual 2.1->2.2 protocol switch in this patch (and the changes in the patch are valid for 2.1 too).  I'll open a new bug for the 2.1->2.2 switch
Summary: safebrowsing v2.2 changes → index/work with sub entries by add chunk id, not host key
(Reporter)

Comment 4

10 years ago
Pushed as http://hg.mozilla.org/mozilla-central/rev/7825b962281d
Status: NEW → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED
(Assignee)

Updated

4 years ago
Component: Phishing Protection → Phishing Protection
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.