1715316 - (oom-fuzz) [meta] (oom-fuzz) Memory usage bugs in the browser found while fuzzing

Status: NEW

(Core :: Fuzzing, defect)

Description

[Tyson Smith [:tsmith]]

This is a meta bug to track memory usage issues found while fuzzing.

The primary goal is to identify unique memory consumption issues fuzzers trigger that negatively impact fuzzing and address them. This primarily includes OOMs due to multiple allocations.

Issues that are reported aren't necessarily bugs, instead they may highlight an area that could benefit from optimization. They may be magnifying an issue that would otherwise go unnoticed. Addressing this would help make the browser more fuzzing friendly.

Heap profiles are the most effective method of bucketing we have at the moment. A consistent method of bucketing enables our tools to automatically reduce test cases.

A bug with a reliable test case and a heap profile are provided to help determine if an issue truly is a bug or if there is potential for an optimization.

A note about memory usage bugs from the fuzzing perspective

When there is unconstrained memory usage triggered due to a bug it can affect fuzzing in multiple ways.

It can affect the current fuzzing task directly triggering a (browser process) crash due to OOM. These can be very easy to trigger due to the nature of fuzzing. Depending on the type of OOM they can be difficultly to bucket and ignore. Avoiding unnecessary browser restarts helps make fuzzing more effective. This is important in the case of all easily triggerable issues. See Fuzz Blockers.

When fuzzing it is common to run multiple instances of a fuzzer on the same hardware to maximize resource usage.

These issues also affect the system the fuzzers are running on and this can affect multiple fuzzers at once. It can trigger high system load due to paging or OOM the system which can lead to false positives or bogus results. It can also cause the automation responsible for running fuzzers on the system to fail.

In some cases it might not be realistic to enforce limits or "fix" the issue. Making allocations fallible can prevent the fuzzers from crashing the browser and might be a reasonable solution is these cases.

(UPDATED) Browser fuzzing typically uses the following limits

• Allocation size limit: 12GB (NULL is returned for allocations greater than the limit. ASan & TSan only)
• Maximum process size: 12GB (heap profile is collected and browser is terminated. ASan & TSan only)