1543319 - Reduce memory used during Safe Browsing update

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, enhancement, P1)

Description

[Dimi Lee [:dimi]]

During Safe Browsing update, some used memory can be free ASAP and we can also replace the byte-reordering with in-place algorithm to save memory.

Comment 1

[Dimi Lee [:dimi]]

Attachment: Bug 1543319 - P1. Free intermediate memory as early as possible during Safe Browsing update. r?gcp

Here is the flow how prefixes are handled during an V4 update:

1. Prefixes are received from Safe Browsing update, stored in ProtocolBuffer
2. Copy the prefixes from ProtocolBuffer to TableUpdate structure
3. Prefixes in TableUpdate are merged with local prefixes (stored in LookupCacheV4)
4. Merged prefixes are processes by PrefixSet to generate the in-memory prefix
   set data structure (MakePrefixSet).

In this patch, we free the prefixes stored in TableUpdate right after step3.
This reduces the peak memory used during an update (peak happens in step 4).

Comment 2

[Dimi Lee [:dimi]]

Attachment: Bug 1543319 - P2. Swap the byte order in-place. r?gcp

We don't need an additional array just for byte reordering, replace
it with in-place processing.

Testcase are modified because the LookupCacheV4::Build API now clears the
input parameter.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/95e20c4ed6c3
P1. Free intermediate memory as early as possible during Safe Browsing update. r=gcp
https://hg.mozilla.org/integration/autoland/rev/85e806016727
P2. Swap the byte order in-place. r=gcp

Comment 4

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/95e20c4ed6c3
https://hg.mozilla.org/mozilla-central/rev/85e806016727