799494 - AddressSanitizer: jit-test/tests/debug/Environment-identity-03.js fails with "InternalError: too much recursion"

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Christian Holler (:decoder)]

When running the jit-tests under AddressSanitizer, there is one failure popping up (both on the try server and locally, even with Clang revision 164207). The test in question is "jit-test/tests/debug/Environment-identity-03.js" and it is failing as follows:

jit-test/tests/debug/Environment-identity-03.js:75:0 InternalError: too much recursion


This happens in both debug+opt and opt-only builds, but only with AddressSanitizer, since I assume that the stack instrumentation increases the stack size a bit more so the failure gets visible.

A GDB stack trace in a debug+opt build shows 296 stack frames on the stack when hitting the recursion limit, and the recursion is between these four functions:

#263 0x0000000000b0e2ad in EmitFunc
#264 0x0000000000afa6bf in js::frontend::EmitTree
#265 0x0000000000aff9bc in js::frontend::EmitTree
#266 0x0000000000af649f in js::frontend::EmitFunctionScript


I was trying to figure out which of the function frames is the largest, and it seems that the size of js::frontend::EmitFunctionScript is 2688 bytes (I measured the difference of rsp between frame 265 and 266, not sure if I did that right).

This failure is not new btw, it has been there for a longer time, but I haven't reported it because other failures where more important. Right now this is the last blocker for green jit-tests.

Comment 1

[Terrence Cole [:terrence]]

Is this jit-tests in the shell?  If so, could you check if MOZ_ASAN is defined when building the shell in ASAN mode?  In the browser we double the stack size limit when MOZ_ASAN is set and it would be nice if we could just copy this code from there.

Comment 2

[Christian Holler (:decoder)]

Confirmed that with stacksize 2 * 128 * sizeof(size_t) * 1024 in js/src/shell/js.cpp the test passes.

Comment 3

[Ed Morley [:emorley]]

Is occurring on the new releng ASan builds.

Comment 4

[Christian Holler (:decoder)]

I think this is fixed by bug 803182 now, at least I haven't seen this failure anymore since then.