Closed Bug 1557113 Opened 6 years ago Closed 5 years ago

Intermittent SUMMARY: AddressSanitizer: SEGV /build/glibc-LK5gWL/glibc-2.23/stdlib/getenv.c:75 in getenv

Categories

(Core :: IPC, defect)

Product:
Core
Component:
IPC
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1556599

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, regression, Whiteboard: [stockwell unknown])

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=250196670&repo=mozilla-inbound
Full log: https://queue.taskcluster.net/v1/task/b6LoZdk7SweLOqgqUbu46A/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/b6LoZdk7SweLOqgqUbu46A/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-06-05T18:06:46.489Z] 18:06:46 INFO - SUMMARY: AddressSanitizer: SEGV /build/glibc-LK5gWL/glibc-2.23/stdlib/getenv.c:75 in getenv
[task 2019-06-05T18:06:46.489Z] 18:06:46 INFO - Thread T2 (Gecko_IOThread) created by T0 here:
[task 2019-06-05T18:06:46.508Z] 18:06:46 INFO - #0 0x55606d465add in pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:210:3
[task 2019-06-05T18:06:46.509Z] 18:06:46 INFO - #1 0x7fc3de310bcc in CreateThread /builds/worker/workspace/build/src/ipc/chromium/src/base/platform_thread_posix.cc:123:14
[task 2019-06-05T18:06:46.510Z] 18:06:46 INFO - #2 0x7fc3de310bcc in PlatformThread::Create(unsigned long, PlatformThread::Delegate*, unsigned long*) /builds/worker/workspace/build/src/ipc/chromium/src/base/platform_thread_posix.cc:134
[task 2019-06-05T18:06:46.511Z] 18:06:46 INFO - #3 0x7fc3de31df93 in base::Thread::StartWithOptions(base::Thread::Options const&) /builds/worker/workspace/build/src/ipc/chromium/src/base/thread.cc:97:8
[task 2019-06-05T18:06:46.512Z] 18:06:46 INFO - #4 0x7fc3dd3aaca7 in NS_InitXPCOM /builds/worker/workspace/build/src/xpcom/build/XPCOMInit.cpp:315:9
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #5 0x7fc3e9451a50 in Initialize /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:1281:8
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #6 0x7fc3e9451a50 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4771
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #7 0x7fc3e94533be in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4856:21
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #8 0x55606d4b004d in do_main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:212:22
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #9 0x55606d4b004d in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:291
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - #10 0x7fc3fcf2882f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
[task 2019-06-05T18:06:46.519Z] 18:06:46 INFO - ==5384==ABORTING
[task 2019-06-05T18:09:43.744Z] 18:09:43 ERROR - TEST-UNEXPECTED-FAIL | Last test finished | application terminated with exit code 1

The log excerpt in comment #0 inexplicably omits the stack for the actual crash; this is where it should have started:

[task 2019-06-05T18:06:45.786Z] 18:06:45    ERROR - ==5384==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fc3fcf4181d bp 0x616000000c80 sp 0x7fc3d9de17c0 T2)
[task 2019-06-05T18:06:45.787Z] 18:06:45     INFO - ==5384==The signal is caused by a READ memory access.
[task 2019-06-05T18:06:45.788Z] 18:06:45     INFO - ==5384==Hint: address points to the zero page.
[task 2019-06-05T18:06:46.067Z] 18:06:46     INFO -     #0 0x7fc3fcf4181c in getenv /build/glibc-LK5gWL/glibc-2.23/stdlib/getenv.c:75
[task 2019-06-05T18:06:46.070Z] 18:06:46     INFO -     #1 0x7fc3fcf31cb8 in __gconv_load_cache /build/glibc-LK5gWL/glibc-2.23/iconv/gconv_cache.c:57
[task 2019-06-05T18:06:46.071Z] 18:06:46     INFO -     #2 0x7fc3fcf2b3d5 in __gconv_read_conf /build/glibc-LK5gWL/glibc-2.23/iconv/gconv_conf.c:548
[task 2019-06-05T18:06:46.072Z] 18:06:46     INFO -     #3 0x7fc3fdf8da98 in __pthread_once_slow (/lib/x86_64-linux-gnu/libpthread.so.0+0xea98)
[task 2019-06-05T18:06:46.073Z] 18:06:46     INFO -     #4 0x7fc3fcf2a9c3 in __gconv_find_transform /build/glibc-LK5gWL/glibc-2.23/iconv/gconv_db.c:747
[task 2019-06-05T18:06:46.075Z] 18:06:46     INFO -     #5 0x7fc3fcfc218a in __wcsmbs_getfct /build/glibc-LK5gWL/glibc-2.23/wcsmbs/wcsmbsload.c:91
[task 2019-06-05T18:06:46.077Z] 18:06:46     INFO -     #6 0x7fc3fcfc218a in __wcsmbs_load_conv /build/glibc-LK5gWL/glibc-2.23/wcsmbs/wcsmbsload.c:186
[task 2019-06-05T18:06:46.077Z] 18:06:46     INFO -     #7 0x7fc3fcfc234a in get_gconv_fcts /build/glibc-LK5gWL/glibc-2.23/wcsmbs/./wcsmbsload.h:75
[task 2019-06-05T18:06:46.078Z] 18:06:46     INFO -     #8 0x7fc3fcfc234a in __wcsmbs_clone_conv /build/glibc-LK5gWL/glibc-2.23/wcsmbs/wcsmbsload.c:223
[task 2019-06-05T18:06:46.080Z] 18:06:46     INFO -     #9 0x7fc3fcf7d332 in _IO_fwide /build/glibc-LK5gWL/glibc-2.23/libio/iofwide.c:124
[task 2019-06-05T18:06:46.083Z] 18:06:46     INFO -     #10 0x7fc3fcf7983d in vswprintf /build/glibc-LK5gWL/glibc-2.23/libio/vswprintf.c:108
[task 2019-06-05T18:06:46.442Z] 18:06:46     INFO -     #11 0x7fc3de31ceb1 in vswprintf /builds/worker/workspace/build/src/ipc/chromium/src/base/string_util_posix.h:39:10
[task 2019-06-05T18:06:46.446Z] 18:06:46     INFO -     #12 0x7fc3de31ceb1 in vsnprintfT /builds/worker/workspace/build/src/ipc/chromium/src/base/string_util.cc:341
[task 2019-06-05T18:06:46.447Z] 18:06:46     INFO -     #13 0x7fc3de31ceb1 in StringAppendVT<std::basic_string<wchar_t> > /builds/worker/workspace/build/src/ipc/chromium/src/base/string_util.cc:361
[task 2019-06-05T18:06:46.448Z] 18:06:46     INFO -     #14 0x7fc3de31ceb1 in StringAppendV(std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >*, wchar_t const*, __va_list_tag*) /builds/worker/workspace/build/src/ipc/chromium/src/base/string_util.cc:534
[task 2019-06-05T18:06:46.449Z] 18:06:46     INFO -     #15 0x7fc3de31ccda in StringPrintf(wchar_t const*, ...) /builds/worker/workspace/build/src/ipc/chromium/src/base/string_util.cc:550:3
[task 2019-06-05T18:06:46.450Z] 18:06:46     INFO -     #16 0x7fc3de327b86 in IPC::Channel::GenerateUniqueRandomChannelID() /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel.cc:36:10
[task 2019-06-05T18:06:46.451Z] 18:06:46     INFO -     #17 0x7fc3de3310f4 in IPC::Channel::GenerateVerifiedChannelID(std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&) /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:978:20
[task 2019-06-05T18:06:46.452Z] 18:06:46     INFO -     #18 0x7fc3de320906 in ChildProcessHost::CreateChannel() /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/child_process_host.cc:29:17

This looks like a duplicate of bug 1556599; see the analysis in bug 1556599 comment #1.

This isn't really an IPC bug — we're just the first thing to touch locales — but it certainly isn't Layout.

Component: Layout → IPC

There have been 26 total failures in the last 7 days on linux1804-64-asan opt and linux1804-64-asan-qr opt:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2020-03-10&endday=2020-03-17&tree=trunk&bug=1557113

Recent log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=293444024&repo=autoland&lineNumber=4677

Jed, any progress on this or Bug 1556599?

Flags: needinfo?(jld)
Whiteboard: [stockwell needswork:owner]

There are 21 failures associated to this bug in the last 7 days. These are occurring on linux1804-64-asan and linux1804-64-asan-qr opt builds.

recent log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=294460192&repo=autoland&lineNumber=2135
[task 2020-03-24T02:53:10.057Z] 02:53:10 INFO - AddressSanitizer can not provide additional info.
[task 2020-03-24T02:53:10.057Z] 02:53:10 INFO - SUMMARY: AddressSanitizer: SEGV /build/glibc-OTsEL5/glibc-2.27/stdlib/getenv.c:75 in getenv
[task 2020-03-24T02:53:10.058Z] 02:53:10 INFO - Thread T4 (Gecko_IOThread) created by T0 here:
[task 2020-03-24T02:53:10.061Z] 02:53:10 INFO - #0 0x564dce3b706a in pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:209:3
[task 2020-03-24T02:53:10.062Z] 02:53:10 INFO - #1 0x7fc8ecc03cdc in CreateThread /builds/worker/checkouts/gecko/ipc/chromium/src/base/platform_thread_posix.cc:123:14
[task 2020-03-24T02:53:10.062Z] 02:53:10 INFO - #2 0x7fc8ecc03cdc in PlatformThread::Create(unsigned long, PlatformThread::Delegate*, unsigned long*) /builds/worker/checkouts/gecko/ipc/chromium/src/base/platform_thread_posix.cc:134:10
[task 2020-03-24T02:53:10.063Z] 02:53:10 INFO - #3 0x7fc8ecc1684d in base::Thread::StartWithOptions(base::Thread::Options const&) /builds/worker/checkouts/gecko/ipc/chromium/src/base/thread.cc:97:8
[task 2020-03-24T02:53:10.065Z] 02:53:10 INFO - #4 0x7fc8ebcc16fb in NS_InitXPCOM /builds/worker/checkouts/gecko/xpcom/build/XPCOMInit.cpp:316:9
[task 2020-03-24T02:53:10.067Z] 02:53:10 INFO - #5 0x7fc8f6434203 in ScopedXPCOMStartup::Initialize(bool) /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:1266:8
[task 2020-03-24T02:53:10.068Z] 02:53:10 INFO - #6 0x7fc8f64474cd in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:4745:22
[task 2020-03-24T02:53:10.068Z] 02:53:10 INFO - #7 0x7fc8f6448223 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:4800:21
[task 2020-03-24T02:53:10.069Z] 02:53:10 INFO - #8 0x564dce3ff48b in do_main /builds/worker/checkouts/gecko/browser/app/nsBrowserApp.cpp:217:22
[task 2020-03-24T02:53:10.069Z] 02:53:10 INFO - #9 0x564dce3ff48b in main /builds/worker/checkouts/gecko/browser/app/nsBrowserApp.cpp:331:16
[task 2020-03-24T02:53:10.073Z] 02:53:10 INFO - #10 0x7fc90a865b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
[task 2020-03-24T02:53:10.073Z] 02:53:10 INFO - ==2279==ABORTING
[task 2020-03-24T02:56:08.336Z] 02:56:08 ERROR - TEST-UNEXPECTED-FAIL | Last test finished | application terminated with exit code 1
[task 2020-03-24T02:56:08.340Z] 02:56:08 INFO - Traceback (most recent call last):
[task 2020-03-24T02:56:08.340Z] 02:56:08 INFO - File "/builds/worker/workspace/build/tests/reftest/runreftest.py", line 1030, in <module>
[task 2020-03-24T02:56:08.341Z] 02:56:08 INFO - sys.exit(run_test_harness(parser, options))
[task 2020-03-24T02:56:08.342Z] 02:56:08 INFO - File "/builds/worker/workspace/build/tests/reftest/runreftest.py", line 1022, in run_test_harness
[task 2020-03-24T02:56:08.342Z] 02:56:08 INFO - result = reftest.runTests(options.tests, options)
[task 2020-03-24T02:56:08.342Z] 02:56:08 INFO - File "/builds/worker/workspace/build/tests/reftest/runreftest.py", line 613, in runTests
[task 2020-03-24T02:56:08.343Z] 02:56:08 INFO - return self.runSerialTests(manifests, options, cmdargs)
[task 2020-03-24T02:56:08.343Z] 02:56:08 INFO - File "/builds/worker/workspace/build/tests/reftest/runreftest.py", line 970, in runSerialTests
[task 2020-03-24T02:56:08.343Z] 02:56:08 INFO - status = run(tests=tests)
[task 2020-03-24T02:56:08.344Z] 02:56:08 INFO - File "/builds/worker/workspace/build/tests/reftest/runreftest.py", line 944, in run
[task 2020-03-24T02:56:08.344Z] 02:56:08 INFO - **kwargs)

:jld any updates for this?

There are 21 total failures in the last 7 days on linux1804-64-asan-qr, linux1804-64-asan opt.

An example of a recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=296345786&repo=autoland&lineNumber=7123

There are 20 failures associated to this bug in the last 7 days. These are occurring on linux1804-64-asan-qr and linux1804-64-asan.

:jld any updates on this?

If there was progress there would be updates in bug 1556599.

Flags: needinfo?(jld)

Going to set the depends to reflect that we expect that bug to make this problem go away.

Depends on: 1556599

In hindsight I should have closed this as a duplicate in comment #29 — it's the same stack and no reason to believe the cause is different.

And the patches I have, if/when the landing sticks, will resolve this crash, because some of the code (frames 16 and 17 in both bugs) will no longer exist in Linux builds.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.