1638764 - Intermittent AddressSanitizer: SEGV /build/glibc-6V9RKT/glibc-2.19/string/../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:131

Status: RESOLVED FIXED

(Core :: JavaScript: GC, defect, P3)

Description

Push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=ac9f8166fa65f84f404575292dcbae15affb2827&searchStr=linux%2Cx64%2Copt%2Cspidermonkey%2Cbuilds%2Cspidermonkey-sm-fuzzing-linux64%2Fopt%2Csm%28f%29&selectedTaskRun=LmXLQP_jQI-bO64KCC8gzA-0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302694213&repo=autoland&lineNumber=12656

>[task 2020-05-18T07:13:09.506Z] AddressSanitizer:DEADLYSIGNAL
[task 2020-05-18T07:13:09.506Z] =================================================================
[task 2020-05-18T07:13:09.506Z] ==12253==ERROR: AddressSanitizer: SEGV on unknown address 0x000000003ef0 (pc 0x7ffff6e98519 bp 0x7fffffdf4f70 sp 0x7fffffdf4728 T0)
[task 2020-05-18T07:13:09.506Z] ==12253==The signal is caused by a READ memory access.
[task 2020-05-18T07:13:09.506Z]     #0 0x7ffff6e98519  /build/glibc-6V9RKT/glibc-2.19/string/../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:131
[task 2020-05-18T07:13:09.506Z]     #1 0x55555625c117 in __asan_memcpy /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:22:3
[task 2020-05-18T07:13:09.506Z]     #2 0x555557a57054 in std::enable_if<__and_<std::__not_<std::__is_tuple_like<js::gc::ChunkBitmap> >, std::is_move_constructible<js::gc::ChunkBitmap>, std::is_move_assignable<js::gc::ChunkBitmap> >::value, void>::type std::swap<js::gc::ChunkBitmap>(js::gc::ChunkBitmap&, js::gc::ChunkBitmap&) /builds/worker/fetches/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/7.4.0/../../../../include/c++/7.4.0/bits/move.h:198:19
[task 2020-05-18T07:13:09.506Z]     #3 0x555557a57054 in js::gc::MarkingValidator::nonIncrementalMark(js::gc::AutoGCSession&) /builds/worker/workspace/build/src/js/src/gc/Verifier.cpp:637:7
[task 2020-05-18T07:13:09.506Z]     #4 0x55555798301b in js::gc::GCRuntime::beginSweepPhase(JS::GCReason, js::gc::AutoGCSession&) /builds/worker/workspace/build/src/js/src/gc/GC.cpp:5468:3
[task 2020-05-18T07:13:09.506Z]     #5 0x55555798f5e0 in js::gc::GCRuntime::incrementalSlice(js::SliceBudget&, mozilla::Maybe<JSGCInvocationKind> const&, JS::GCReason, js::gc::AutoGCSession&) /builds/worker/workspace/build/src/js/src/gc/GC.cpp:6665:7
[task 2020-05-18T07:13:09.506Z]     #6 0x55555799311c in js::gc::GCRuntime::gcCycle(bool, js::SliceBudget, mozilla::Maybe<JSGCInvocationKind> const&, JS::GCReason) /builds/worker/workspace/build/src/js/src/gc/GC.cpp:7085:3
[task 2020-05-18T07:13:09.506Z]     #7 0x555557996b70 in js::gc::GCRuntime::collect(bool, js::SliceBudget, mozilla::Maybe<JSGCInvocationKind> const&, JS::GCReason) /builds/worker/workspace/build/src/js/src/gc/GC.cpp:7296:9
[task 2020-05-18T07:13:09.506Z]     #8 0x55555799fdb8 in js::gc::GCRuntime::runDebugGC() /builds/worker/workspace/build/src/js/src/gc/GC.cpp:7885:5
[task 2020-05-18T07:13:09.506Z]     #9 0x555557902518 in js::gc::GCRuntime::gcIfNeededAtAllocation(JSContext*) /builds/worker/workspace/build/src/js/src/gc/Allocator.cpp:442:5
[task 2020-05-18T07:13:09.506Z]     #10 0x555557902518 in bool js::gc::GCRuntime::checkAllocatorState<(js::AllowGC)1>(JSContext*, js::gc::AllocKind) /builds/worker/workspace/build/src/js/src/gc/Allocator.cpp:404:10
[task 2020-05-18T07:13:09.506Z]     #11 0x555557902518 in js::Shape* js::Allocate<js::Shape, (js::AllowGC)1>(JSContext*) /builds/worker/workspace/build/src/js/src/gc/Allocator.cpp:329:28
[task 2020-05-18T07:13:09.506Z]     #12 0x555556dc27e8 in js::NativeObject::toDictionaryMode(JSContext*, JS::Handle<js::NativeObject*>) /builds/worker/workspace/build/src/js/src/vm/Shape.cpp:505:47
[task 2020-05-18T07:13:09.506Z]     #13 0x555556dd0ea3 in js::NativeObject::maybeToDictionaryModeForPut(JSContext*, JS::Handle<js::NativeObject*>, JS::MutableHandle<js::Shape*>) /builds/worker/workspace/build/src/js/src/vm/Shape.cpp:994:8
[task 2020-05-18T07:13:09.506Z]     #14 0x555556dd1d65 in js::NativeObject::putDataProperty(JSContext*, JS::Handle<js::NativeObject*>, JS::Handle<JS::PropertyKey>, unsigned int) /builds/worker/workspace/build/src/js/src/vm/Shape.cpp:1062:8
[task 2020-05-18T07:13:09.506Z]     #15 0x555556c725b8 in bool AddOrChangeProperty<(IsAddOrChange)1>(JSContext*, JS::Handle<js::NativeObject*>, JS::Handle<JS::PropertyKey>, JS::Handle<JS::PropertyDescriptor>) /builds/worker/workspace/build/src/js/src/vm/NativeObject.cpp:1459:15
[task 2020-05-18T07:13:09.506Z]     #16 0x555556c725b8 in js::NativeDefineProperty(JSContext*, JS::Handle<js::NativeObject*>, JS::Handle<JS::PropertyKey>, JS::Handle<JS::PropertyDescriptor>, JS::ObjectOpResult&) /builds/worker/workspace/build/src/js/src/vm/NativeObject.cpp:1903:8
[task 2020-05-18T07:13:09.506Z]     #17 0x555556b8c631 in js::DefineDataProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::PropertyKey>, JS::Handle<JS::Value>, unsigned int, JS::ObjectOpResult&) /builds/worker/workspace/build/src/js/src/vm/JSObject.cpp:2737:10
[task 2020-05-18T07:13:09.506Z]     #18 0x555556b8c631 in js::DefineDataProperty(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::PropertyKey>, JS::Handle<JS::Value>, unsigned int) /builds/worker/workspace/build/src/js/src/vm/JSObject.cpp:2758:8
[task 2020-05-18T07:13:09.507Z]     #19 0x555556b66c39 in js::DefineDataProperty(JSContext*, JS::Handle<JSObject*>, js::PropertyName*, JS::Handle<JS::Value>, unsigned int) /builds/worker/workspace/build/src/js/src/vm/JSObject.cpp:2772:10
[task 2020-05-18T07:13:09.507Z]     #20 0x5555569a0adb in js::LinkConstructorAndPrototype(JSContext*, JSObject*, JSObject*, unsigned int, unsigned int) /builds/worker/workspace/build/src/js/src/vm/GlobalObject.cpp:916:10
[task 2020-05-18T07:13:09.507Z]     #21 0x55555699f14e in js::GlobalObject::resolveConstructor(JSContext*, JS::Handle<js::GlobalObject*>, JSProtoKey, js::GlobalObject::IfClassIsDisabled) /builds/worker/workspace/build/src/js/src/vm/GlobalObject.cpp:357:17
[task 2020-05-18T07:13:09.507Z]     #22 0x555556d5f3fc in js::GlobalObject::ensureConstructor(JSContext*, JS::Handle<js::GlobalObject*>, JSProtoKey) /builds/worker/workspace/build/src/js/src/vm/GlobalObject.h:179:12
[task 2020-05-18T07:13:09.507Z]     #23 0x555556d5f3fc in js::GlobalObject::getOrCreateSavedFramePrototype(JSContext*, JS::Handle<js::GlobalObject*>) /builds/worker/workspace/build/src/js/src/vm/GlobalObject.h:422:10
[task 2020-05-18T07:13:09.507Z]     #24 0x555556d5f3fc in js::SavedFrame::create(JSContext*) /builds/worker/workspace/build/src/js/src/vm/SavedStacks.cpp:557:22
[task 2020-05-18T07:13:09.507Z]     #25 0x555556d77eb8 in js::SavedStacks::createFrameFromLookup(JSContext*, JS::Handle<js::SavedFrame::Lookup>) /builds/worker/workspace/build/src/js/src/vm/SavedStacks.cpp:1730:30
[task 2020-05-18T07:13:09.507Z]     #26 0x555556d77eb8 in js::SavedStacks::getOrCreateSavedFrame(JSContext*, JS::Handle<js::SavedFrame::Lookup>) /builds/worker/workspace/build/src/js/src/vm/SavedStacks.cpp:1716:30
[task 2020-05-18T07:13:09.507Z]     #27 0x555556d6f767 in js::SavedStacks::insertFrames(JSContext*, JS::MutableHandle<js::SavedFrame*>, mozilla::Variant<JS::AllFrames, JS::MaxFrames, JS::FirstSubsumedFrame>&&) /builds/worker/workspace/build/src/js/src/vm/SavedStacks.cpp:1570:15
[task 2020-05-18T07:13:09.507Z]     #28 0x555556d6c9d7 in js::SavedStacks::saveCurrentStack(JSContext*, JS::MutableHandle<js::SavedFrame*>, mozilla::Variant<JS::AllFrames, JS::MaxFrames, JS::FirstSubsumedFrame>&&) /builds/worker/workspace/build/src/js/src/vm/SavedStacks.cpp:1307:10
[task 2020-05-18T07:13:09.507Z]     #29 0x55555724bce2 in JS::CaptureCurrentStack(JSContext*, JS::MutableHandle<JSObject*>, mozilla::Variant<JS::AllFrames, JS::MaxFrames, JS::FirstSubsumedFrame>&&) /builds/worker/workspace/build/src/js/src/jsapi.cpp:5852:29
[task 2020-05-18T07:13:09.507Z]     #30 0x555556799af5 in js::CaptureStack(JSContext*, JS::MutableHandle<JSObject*>) /builds/worker/workspace/build/src/js/src/jsexn.cpp:219:10
[task 2020-05-18T07:13:09.507Z]     #31 0x555556799af5 in js::ErrorToException(JSContext*, JSErrorReport*, JSErrorFormatString const* (*)(void*, unsigned int), void*) /builds/worker/workspace/build/src/js/src/jsexn.cpp:329:8
[task 2020-05-18T07:13:09.507Z]     #32 0x555556964008 in ReportError(JSContext*, JSErrorReport*, JSErrorFormatString const* (*)(void*, unsigned int), void*) /builds/worker/workspace/build/src/js/src/vm/ErrorReporting.cpp:164:3
[task 2020-05-18T07:13:09.507Z]     #33 0x555556964008 in js::ReportErrorNumberVA(JSContext*, js::IsWarning, JSErrorFormatString const* (*)(void*, unsigned int), void*, unsigned int, js::ErrorArgumentsType, __va_list_tag*) /builds/worker/workspace/build/src/js/src/vm/ErrorReporting.cpp:477:3
[task 2020-05-18T07:13:09.507Z]     #34 0x5555571eced7 in JS_ReportErrorNumberASCIIVA(JSContext*, JSErrorFormatString const* (*)(void*, unsigned int), void*, unsigned int, __va_list_tag*) /builds/worker/workspace/build/src/js/src/jsapi.cpp:4777:3
[task 2020-05-18T07:13:09.507Z]     #35 0x5555571eced7 in JS_ReportErrorNumberASCII(JSContext*, JSErrorFormatString const* (*)(void*, unsigned int), void*, unsigned int, ...) /builds/worker/workspace/build/src/js/src/jsapi.cpp:4767:3
[task 2020-05-18T07:13:09.507Z]     #36 0x555556b6312d in js::ReportOverRecursed(JSContext*, unsigned int) /builds/worker/workspace/build/src/js/src/vm/JSContext.cpp:309:7
[task 2020-05-18T07:13:09.507Z]     #37 0x5555565605eb in js::CheckRecursionLimit(JSContext*, unsigned long) /builds/worker/workspace/build/src/js/src/jsfriendapi.h:988:5
[task 2020-05-18T07:13:09.507Z]     #38 0x5555565605eb in js::CheckRecursionLimit(JSContext*) /builds/worker/workspace/build/src/js/src/jsfriendapi.h:1014:10
[task 2020-05-18T07:13:09.507Z]     #39 0x5555565605eb in js::RunScript(JSContext*, js::RunState&) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:427:8
[task 2020-05-18T07:13:09.507Z]     #40 0x555556594a41 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) /builds/worker/workspace/build/src/js/src/vm/Interpreter.cpp:620:13
[task 2020-05-18T07:13:09.507Z]     #41 0x555557b831eb in js::jit::DoCallFallback(JSContext*, js::jit::BaselineFrame*, js::jit::ICCall_Fallback*, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) /builds/worker/workspace/build/src/js/src/jit/BaselineIC.cpp:2990:10
[task 2020-05-18T07:13:09.507Z]     #42 0x124ec8567bd7  (<unknown module>)
[task 2020-05-18T07:13:09.507Z] 
[task 2020-05-18T07:13:09.507Z] AddressSanitizer can not provide additional info.
[task 2020-05-18T07:13:09.507Z] SUMMARY: AddressSanitizer: SEGV /build/glibc-6V9RKT/glibc-2.19/string/../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:131 
[task 2020-05-18T07:13:09.507Z] ==12253==ABORTING
[task 2020-05-18T07:13:09.507Z] Exit code: 1
[task 2020-05-18T07:13:09.507Z] FAIL - gc/bug-1459860.js

Comment 1

[Daniel Veditz [:dveditz]]

This looks like it might be in validator code, and maybe not affect shipping Firefox?

Comment 2

[Jon Coppeard (:jonco)]

Yes, this looks like a bug in the incremental marking validator.

Comment 3

[Jon Coppeard (:jonco)]

Attachment: Bug 1638764 - Wait for background allocation before copying chunk mark bits r?sfink

I think the problem is that background allocation is creating a new chunk after we copy the mark bits for the existing chunks, so we fail to find the new chunk in the map. If so then we need to wait for that first, and after we know any off-thread parsing has finished.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d583fafc8eda
Wait for background allocation before copying chunk mark bits r=sfink

Comment 5

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/d583fafc8eda

Comment 6

[Intermittent Failures Robot]

2 failures in 4727 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1638764&startday=2020-05-18&endday=2020-05-24&tree=all

Comment 8

[Jon Coppeard (:jonco)]

Looks like that didn't fix it.

Comment 9

[Intermittent Failures Robot]

1 failures in 5458 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1638764&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 10

[Intermittent Failures Robot]

2 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1638764&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 11

[Intermittent Failures Robot]

1 failures in 4035 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1638764&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 12

[Jon Coppeard (:jonco)]

Attachment: Bug 1638764 - Make the incremental marking validator check that all chunks are found in its map r?sfink

I still don't know why this is happening, but this should make the problem a release mode assertion failure rather than an illegal memory access.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/27a7ed0c155a
Make the incremental marking validator check that all chunks are found in its map r=sfink

Comment 14

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/27a7ed0c155a

Comment 16

[Jon Coppeard (:jonco)]

The assertion added by the previous patch is now failing. I still can't see how this is possible though.

Probably we should just ignore new chunks in the pool since this doesn't happen very often and the marking verifier is not a critical piece of functionality.

Before I do that however, Steve do you think you could take a look and see if you can see what's going wrong here?

Comment 18

[Ted Campbell [:tcampbell]]

Reproduces with ./mach jit-test --debugger="rr record -h" gc/bug-1459860.js. The bug is that the Chunk is created during an off-thread parse (for an off-thread Zone) while MarkingValidator::nonIncrementalMark is running, it doesn't exist yet when the map is initially filled in.

(Bug 1658800 is a dupe of this)

Comment 20

[Jon Coppeard (:jonco)]

(In reply to Ted Campbell [:tcampbell] from comment #18)

The bug is that the Chunk is created during an off-thread parse

The marking validator waits for all helper thread tasks to finish first, by calling WaitForAllHelperThreads(). The bug is that that doesn't work.

Comment 21

[Jon Coppeard (:jonco)]

Attachment: Bug 1638764 - Make WaitForAllHelperThreads() wait for all queued tasks to be processed r?tcampbell

I added an assertion that there were no queued tasks present on exit from this
function and observed that it failed a few times while running jit-tests.

This can happen when a task is queued and the helper thread is signalled (and
made runnable by the OS) but hasn't started running yet.

Comment 22

[Intermittent Failures Robot]

3 failures in 2956 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform breakdown:

• linux64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1638764&startday=2021-01-04&endday=2021-01-10&tree=all

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f91af44347aa
Make WaitForAllHelperThreads() wait for all queued tasks to be processed r=tcampbell

Comment 24

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/f91af44347aa

Comment 25

[Intermittent Failures Robot]

1 failures in 3810 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1638764&startday=2021-01-11&endday=2021-01-17&tree=all

Comment 26

[Ted Campbell [:tcampbell]]

Do we need to uplift to ESR?

Comment 27

[Jon Coppeard (:jonco)]

This only really affects testing so I don't think it needs an uplift.