investigate "IOError: request data read error" exceptions on production collectors

RESOLVED WONTFIX

Status

Socorro
Backend
RESOLVED WONTFIX
3 years ago
8 months ago

People

(Reporter: rhelmer, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
This doesn't look like a new issue (however logs only go back to 2015-03-02), and crash volume seems normal, but we seem to be getting more "IOError: request data read error" exceptions on prod collectors than I remember in the past.

The most likely cause of this exception is clients disconnecting while sending data, while we are reading form data based on the stack traces.

I'd like to make sure there isn't any problem on our end. We might want to add more logging to figure out what's going on - the loadbalancers might be timing slow clients out or something.
[lars@socorro-collector1.webapp.phx1 httpd]$ grep "Fri Mar 13" error_log | grep accepted | wc -l
244546
[lars@socorro-collector1.webapp.phx1 httpd]$ grep "Fri Mar 13" crash-reports.mozilla.com/error_log_2015-03-13* | grep IOError | wc -l
24424

that's roughyl 10%.  Historically it's been 8%
We just did a rewrite of the collector. I've been watching it in production and haven't seen any IOErrors like this one. There are a bunch of possible reasons for that, one of which being that the infrastructure is different, so maybe something deals with disconnecting clients before the data gets to Antenna.

Anyhow, I'm going to close this as WONTFIX since we're not going to look into fixing it in the collector code.
Status: NEW → RESOLVED
Last Resolved: 8 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.