change deepcopy in processor

RESOLVED FIXED

Status

RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: willkg, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Right now, we make a deepcopy of every raw crash for every crash storage class in the processor. That does a lot of extra work because not all the crash storage classes actually mutate the raw crash.

This bug covers changing the code so that we only do deep copies for crash storage classes that mutate the crash.

Comment 2

2 years ago
Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/f409048a12c7d4159383b880394da8bcdc9b6643
Fixes bug 1318448: reduces deepcopies we do in the processor (#3594)

* Fixes bug 1318448: reduces deepcopies we do in the processor

This changes the PolyCrashStorage code such that it only does a deep
copy of the processed crash if the crash storage says it mutates the
crash data.

Currently, we have 5 crash storage classes configured in production.
This change will reduce the number of deepcopies we're doing from 5 to
1.

* Fix for statsd wrapped crash stores

* Fix the fix

The previous fix didn't do the right thing. This does.

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
I missed that StatsdCounter doesn't derive from CrashStorageBase.

PR to handle StatsdCounter: https://github.com/mozilla/socorro/pull/3596
This is on -stage now.

Before the changes:

* processor save_raw_and_processed maxes out at 1.95k crashes every 30m
* CPU hangs out in the high 80s%/low 90s %

After the changes:

* processor save_raw_and_processed maxes out at 2.1k crashes every 20min
* CPU hangs out in the low 50s%

Datadog graph with the appropriate range is here:

https://app.datadoghq.com/dash/187676/socorro-stage-perf?live=false&page=0&is_auto=false&from_ts=1479358800000&to_ts=1479531599999&tile_size=m

Seems like it's good.

We're in a change freeze, so this will go to -prod in a couple of weeks.
This was just pushed to -prod. I added a processor CPU graph to the datadog socorro prod dashboard and watched it for a bit. I don't see any changes in processing (which is expected and fine). Everything seems ok. I think we're done here.
You need to log in before you can comment on or make changes to this bug.