Fragmentation in Safe Browsing chunks leads to 413 (Request Entity Too Large) during list updates

RESOLVED FIXED

Status

()

Toolkit
Safe Browsing
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: francois, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

2 years ago
Monica started looking into bug 1147212 and noticed that she would sometimes get HTTP response 413 (Request Entity Too Large) in response to adding goog-unwanted-shavar. Sample request body is attached.

That's a very fragmented chunk space. It could be a regression on our end.
(Reporter)

Comment 1

2 years ago
Created attachment 8587137 [details]
Example request body
Created attachment 8587833 [details]
Logfile from first update

This is a log taken from within ProtocolParser showing what the server sends us the very first update. 

You can see for example that sub chunk number 47176 is missing, although 47175 and 47177 are there. (see first 3 lines)

Adds chunks are continuous, sub chunks are all over the place.

This makes me suspect this may not be at our end.
I've done a build of Firefox ESR 10, which more or less uses the original SafeBrowsing implementation dating back to Firefox 3 (and likely dates back to Google's original implementation in their toolbar).

It has the same problem, i.e. it ends up with a totally fragemented subchunk list. 

I conclude this is not a regression in our code.
Our alerts did pick up that something happened to the distribution of SafeBrowsing data at Firefox users:
http://alerts.telemetry.mozilla.org/index.html#/detectors/1/metrics/244/alerts/?from=2015-02-26&to=2015-02-26
Google's team pointed out in email conversation that the missing data is in the first redirect URL of the server responses. Looking in that direction now.
Further investigation didn't turn up much interesting. WireShark confirms that the list of redirects we process matches what the server sends back.

I can demonstrate that the list is fragmented via curl + grep, so I think that is more confirmation this is likely not our regression.
(Reporter)

Comment 7

2 years ago
The request size limit on the server will be increased from 20KB to 64KB sometime this week or early
next week.
(Reporter)

Comment 8

2 years ago
The fix is apparently in production, so I'll assume this is fixed.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.