1720772 - SUMMARY: AddressSanitizer: bad-free /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:127:3 in free

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect)

Description

[Mike Hommey [:glandium]]

With bug 1704766 applied, asan builds fail with:

[task 2021-07-15T08:54:43.478Z] 08:54:43     INFO -  package> ==10340==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x7f5bb2ae4bb0 in thread T17 (BgIOThr~Pool #1)
[task 2021-07-15T08:54:43.479Z] 08:54:43     INFO -  package>     #0 0x564c3b9ef082 in free /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:127:3
[task 2021-07-15T08:54:43.479Z] 08:54:43     INFO -  package>     #1 0x564c3ba2b376 in _M_dispose /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/basic_string.h:3251:5
[task 2021-07-15T08:54:43.479Z] 08:54:43     INFO -  package>     #2 0x564c3ba2b376 in std::string::reserve(unsigned long) /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/basic_string.tcc:961:14
[task 2021-07-15T08:54:43.479Z] 08:54:43     INFO -  package>     #3 0x564c3ba2af41 in std::string::append(char const*, unsigned long) /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/bits/basic_string.tcc:758:9
[task 2021-07-15T08:54:43.480Z] 08:54:43     INFO -  package>     #4 0x7f5b9784f201 in std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, char) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xac201)
[task 2021-07-15T08:54:43.480Z] 08:54:43     INFO -  package>     #5 0x7f5b9bb9f9bc in SimpleParseKeyValuePairs /builds/worker/checkouts/gecko/xpcom/base/nsSystemInfo.cpp:89:26
[task 2021-07-15T08:54:43.480Z] 08:54:43     INFO -  package>     #6 0x7f5b9bb9f9bc in CollectProcessInfo(ProcessInfo&) /builds/worker/checkouts/gecko/xpcom/base/nsSystemInfo.cpp:731:5
[task 2021-07-15T08:54:43.480Z] 08:54:43     INFO -  package>     #7 0x7f5b9ce3fef1 in operator() /builds/worker/checkouts/gecko/ipc/glue/IdleSchedulerParent.cpp:60:15
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #8 0x7f5b9ce3fef1 in mozilla::detail::RunnableFunction<mozilla::ipc::IdleSchedulerParent::IdleSchedulerParent()::$_4>::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.h:532:5
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #9 0x7f5b9bd3812f in nsThreadPool::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadPool.cpp:303:14
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #10 0x7f5b9bd2a59b in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1146:16
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #11 0x7f5b9bd34a4c in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:466:10
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #12 0x7f5b9ce21c38 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
[task 2021-07-15T08:54:43.481Z] 08:54:43     INFO -  package>     #13 0x7f5b9cd2dea1 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:331:10
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #14 0x7f5b9cd2dea1 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:324:3
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #15 0x7f5b9cd2dea1 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:306:3
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #16 0x7f5b9bd24052 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:392:10
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #17 0x7f5bb2ceb70e in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #18 0x7f5bb2d1bfa2 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7fa2)
[task 2021-07-15T08:54:43.482Z] 08:54:43     INFO -  package>     #19 0x7f5b9752d4ce in clone (/lib/x86_64-linux-gnu/libc.so.6+0xf94ce)
[task 2021-07-15T08:54:43.483Z] 08:54:43     INFO -  package>

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1720772 - Upgrade sysroot libstdc++ to 7.5.0-6.

7.3.0-1 was used because it was the last 7.x that provided the libstdc++6
package. Unfortunately, once libstdc++ is removed from the clang
toolchain, which makes the one from the sysroot used instead, it causes
an ASan issue which goes away when using a more recent version (which the
libstdc++ from clang was: it was 7.5).

So we take the last 7.x that exists, and tweak it so that it generates
the necessary libstdc++6 package.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/561cbc6c105d
Upgrade sysroot libstdc++ to 7.5.0-6. r=firefox-build-system-reviewers,mhentges

Comment 3

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8596258dd030
Backed out changeset 561cbc6c105d on the suspicion of causing the builds to not be scheduled. CLOSED TREE

Comment 4

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/integration/autoland/rev/561cbc6c105d caused the builds to not be scheduled anymore as you ca see form the below link.
After backing out builds have started again.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&fromchange=8b1c17e122712005dc57542e8e8b8edc4c9b1839&searchStr=build&tochange=8596258dd030379c8cf02f589671da004fd439e9

Backout link: https://hg.mozilla.org/integration/autoland/rev/8596258dd030379c8cf02f589671da004fd439e9

LE: It looks like it needed a toolchains rebuild. Now builds are showing up on the pushes they were missing.
Please let the sheriffs on duty know on element or on slack when these toolchains need to be rebuilt so we can avoid future cases like this. Thank you.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/196b9fb506b7
Upgrade sysroot libstdc++ to 7.5.0-6. r=firefox-build-system-reviewers,mhentges

Comment 6

[Mike Hommey [:glandium]]

See bug 1066272

Comment 7

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/196b9fb506b7