Closed Bug 1543319 Opened 6 months ago Closed 6 months ago

Reduce memory used during Safe Browsing update

Categories

(Toolkit :: Safe Browsing, enhancement, P1)

enhancement

Tracking

()

RESOLVED FIXED
mozilla68
Tracking Status
firefox68 --- fixed

People

(Reporter: dimi, Assigned: dimi)

References

Details

Attachments

(2 files)

During Safe Browsing update, some used memory can be free ASAP and we can also replace the byte-reordering with in-place algorithm to save memory.

Here is the flow how prefixes are handled during an V4 update:

  1. Prefixes are received from Safe Browsing update, stored in ProtocolBuffer
  2. Copy the prefixes from ProtocolBuffer to TableUpdate structure
  3. Prefixes in TableUpdate are merged with local prefixes (stored in LookupCacheV4)
  4. Merged prefixes are processes by PrefixSet to generate the in-memory prefix
    set data structure (MakePrefixSet).

In this patch, we free the prefixes stored in TableUpdate right after step3.
This reduces the peak memory used during an update (peak happens in step 4).

We don't need an additional array just for byte reordering, replace
it with in-place processing.

Testcase are modified because the LookupCacheV4::Build API now clears the
input parameter.

Pushed by dlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/95e20c4ed6c3
P1. Free intermediate memory as early as possible during Safe Browsing update. r=gcp
https://hg.mozilla.org/integration/autoland/rev/85e806016727
P2. Swap the byte order in-place. r=gcp
Status: ASSIGNED → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.