Avoid ASAN-only stack overflows when reflowing deep frame trees, by e.g. using a lower MAX_REFLOW_DEPTH in ASAN builds
Categories
(Core :: Layout, task)
Tracking
()
People
(Reporter: dholbert, Unassigned)
References
(Blocks 3 open bugs)
Details
We have MAX_REFLOW_DEPTH[1] to prevent stack overflow (i.e. stack exhaustion) crashes -- running out of stack space due to too much recursion in deep frame trees. But our choice of a limit is apparently too high for ASAN builds, since ASAN builds have extra allocations per stack-level as discussed in bug 1735919 comment 5.
We should use an ASAN-specific lower threshold for MAX_REFLOW_DEPTH to avoid having fuzzers trip over these stack overflows and abort in ways that official builds would not. (We probably need a little bit of trial-and-error to find the right non-crashy threshold.)
(Alternately, if it's possible to do the reverse and just request a larger stack size, we could do that instead.)
|
||
Comment 1•1 year ago
|
||
In case it's helpful, bugs where we recently hit stack exhaustion on win32 in regular builds and we worked around it: bug 1827428, bug 1824565, bug 1828239. From my memory, these seemed to be caused by the compiler choosing to create larger stack frames for certain functions. So perhaps there is a way to convince the asan builds to not have as big stack frames?
Description
•