If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

safebrowsing server generating way too many subs

RESOLVED FIXED

Status

()

Toolkit
Safe Browsing
RESOLVED FIXED
10 years ago
3 years ago

People

(Reporter: dcamp, Unassigned)

Tracking

3.0 Branch
Points:
---
Bug Flags:
blocking-firefox3 -
blocking-firefox3.5 -
blocking1.9.0.1 -
wanted1.9.0.x +

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [external dependency])

(Reporter)

Description

10 years ago
Looking at a snapshot of the safebrowsing data from last week, we found some significant inefficiency in the add/sub data - by the time we were done, only 13% of the malware data we processed ended up as relevant data on the system - the rest were subs from the list and the adds that matched these subs.

Comparing that last week's snapshot to yesterday's data, it appears that 3.8megs (400,000 entries) have been added to the sub data.  This is a lot more than the expected weekly volume (which is supposed to be closer to ~ 1000 subs per week, as I understand it).

When I talked to the google guys they were surprised by this - I think we need to understand what's going on here quickly, because this is asking users to chew through a whole lot of data unnecessarily.
Flags: blocking-firefox3?
Adding [external dependency] -- Dave, we wouldn't be changing our processing here, right? It's just a matter of asking the list maintainers to more aggressively find +/- pairs and remove them from the list?
Flags: blocking-firefox3? → blocking-firefox3+
Whiteboard: [external dependency]
(Reporter)

Comment 2

10 years ago
Yeah, shouldn't need code changes on our end.

Comment 3

10 years ago
There were a couple of different issues that were causing these problems. We have fixed most of them, so new data coming out should be relatively good.  However, older data that has not expired is still going to have way more subs than is necessary.  We are working on ways to clean up this old data.  We will update here when we start actually editing this data.
dcamp: update?
Moving off to branch blocking nomination list.
Flags: blocking1.9.0.1?
Flags: blocking-firefox3.1?
Flags: blocking-firefox3-
Flags: blocking-firefox3+
Version: unspecified → 3.0 Branch
dcamp: is this even wanted anymore? there's no plan and little detail here ...
Flags: wanted1.9.0.x?
Flags: blocking1.9.0.1?
Flags: blocking1.9.0.1-
After discussion, this is wanted. What's the progress? How do we get motion?
Flags: wanted1.9.0.x? → wanted1.9.0.x+
dcamp: whom do I have to bribe for updates? :)
Flags: blocking-firefox3.1? → blocking-firefox3.1-
Should we set closeme status this bug for a month from now? No one wants to acknowledge beltzner :(

Comment 10

8 years ago
Sorry about the lag, this thread somehow got muted in my e-mail.  This issue is fixed.  We didn't end up specifically changing the older data, but it has been automatically cleaned up at this point.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Component: Phishing Protection → Phishing Protection
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.