Closed Bug 1495741 Opened 2 years ago Closed 2 years ago

Multiple fixes for issues related to memory usage + configuration options for managing memory usage

Categories

(bugzilla.mozilla.org :: General, enhancement, P1)

Production
enhancement

Tracking

()

RESOLVED FIXED

People

(Reporter: dylan, Assigned: dylan)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 1 obsolete file)

07:07 <ckolos> the bugzilla memory issue is causing the service to die in such a way that the health check from the elb *isn't* failing
07:08 <ckolos> or at least failing in a way that's triggering autoscaling.

My hunch is that this is the /rest/product API.
On -dev, we observed this using 2G of memory (and subsequently being killed by the oom-killer). Looking in the logs, it seems that (probably recently) this API has seen more use, up to 250 times per day.
Component: Infrastructure → General
Priority: -- → P1
QA Contact: mcote
There's only 600ish flags. For each flag we turn them into a hash.
This becomes a combinatorial explosion and it's the reason this request uses so much RAM.

This patch caches the hash per flag-type ID and saves about 200-300M
This patch adds a Bugzilla::WebService::JSON class.
This class responds to encode/decode like JSON::MaybeXS but its encode() method returns a Bugzilla::WebService::JSON::Box object,
and its decode() method when passed a "Box" will simply return the box's value.
This avoids round-tripping the JSON value multiple times and doesn't require any changes to the WebService API methods.

Note that one copy is likely to remain -- generating an etag() will call Storable::freeze(). I'll address that in another patch.
Generating etags requires cloning the entire perl data structure into a Storable string. So let's not do that.
add size limit bits and obsolete backed out patch
Attachment #9015896 - Attachment is obsolete: true
Summary: Recent memory issues → memory issues
Summary: memory issues → Multiple fixes for issues related to memory usage + configuration options for managing memory usage
All merged to master.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Blocks: 1513921
You need to log in before you can comment on or make changes to this bug.