1633169 - Frequent beta LeakSanitizer | leak at Realloc, nsTArray_base, nsTArray_Impl, AppendElement

Status: RESOLVED INCOMPLETE

(Testing :: web-platform-tests, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=299392556&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZKQ7zzKdRWGQ2Orkvsjv6A/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-04-25T11:12:54.239Z] 11:12:54 INFO - TEST-START | /encoding/legacy-mb-japanese/euc-jp/eucjp-encode-form-cseucpkdfmtjapanese.html?2001-3000
[task 2020-04-25T11:12:54.241Z] 11:12:54 INFO - PID 12086 | #19 0x7f2d14840ba2 in mozilla::net::ChannelEventQueue::MaybeFlushQueue() /builds/worker/workspace/obj-build/dist/include/mozilla/net/ChannelEventQueue.h:350:5
[task 2020-04-25T11:12:54.241Z] 11:12:54 INFO - PID 12086 | #20 0x7f2d14c24e72 in mozilla::net::ChannelEventQueue::CompleteResume() /builds/worker/workspace/obj-build/dist/include/mozilla/net/ChannelEventQueue.h:329:5
[task 2020-04-25T11:12:54.241Z] 11:12:54 INFO - PID 12086 | #21 0x7f2d14c24b8f in mozilla::net::ChannelEventQueue::ResumeInternal()::CompleteResumeRunnable::Run() /builds/worker/checkouts/gecko/netwerk/ipc/ChannelEventQueue.cpp:148:17
[task 2020-04-25T11:12:54.252Z] 11:12:54 INFO - PID 12086 | #22 0x7f2d13e50f2d in mozilla::SchedulerGroup::Runnable::Run() /builds/worker/checkouts/gecko/xpcom/threads/SchedulerGroup.cpp:146:20
[task 2020-04-25T11:12:54.252Z] 11:12:54 INFO - PID 12086 | #23 0x7f2d13e84bab in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1200:14
[task 2020-04-25T11:12:54.252Z] 11:12:54 INFO - PID 12086 | #24 0x7f2d13e8e8cc in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:481:10
[task 2020-04-25T11:12:54.252Z] 11:12:54 INFO - PID 12086 | #25 0x7f2d14ec5efa in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:87:21

[task 2020-04-25T11:12:59.289Z] 11:12:59 INFO - Browser exited with return code 0
[task 2020-04-25T11:12:59.289Z] 11:12:59 INFO - PROCESS LEAKS None
[task 2020-04-25T11:12:59.289Z] 11:12:59 INFO - ERROR | LeakSanitizer | SUMMARY: AddressSanitizer: 89080 byte(s) leaked in 817 allocation(s).
[task 2020-04-25T11:12:59.290Z] 11:12:59 INFO - LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2020-04-25T11:12:59.290Z] 11:12:59 INFO - This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2020-04-25T11:12:59.290Z] 11:12:59 INFO - Allowed depth was 4
[task 2020-04-25T11:12:59.292Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, Gecko_StartBulkWriteString, nsstring::nsAString::start_bulk_write_impl
[task 2020-04-25T11:12:59.293Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-25T11:12:59.293Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_pod_arena_malloc, maybe_pod_arena_malloc, pod_arena_malloc
[task 2020-04-25T11:12:59.293Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.296Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::ProtoAndIfaceCache::PageTableCache::EntrySlotOrCreate, mozilla::dom::ProtoAndIfaceCache::EntrySlotOrCreate, mozilla::dom::BrowsingContext_Binding::CreateInterfaceObjects, mozilla::dom::GetPerInterfaceObjectHandle
[task 2020-04-25T11:12:59.297Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::dom::ProtoAndIfaceCache::PageTableCache::EntrySlotOrCreate matched a expected leak
[task 2020-04-25T11:12:59.297Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_pod_arena_malloc, maybe_pod_arena_malloc, pod_malloc
[task 2020-04-25T11:12:59.297Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.298Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::nsStandardURL::TemplatedMutator, mozilla::net::nsStandardURL::TemplatedMutator, std::function, operator
[task 2020-04-25T11:12:59.298Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::net::nsStandardURL::TemplatedMutator matched a expected leak
[task 2020-04-25T11:12:59.298Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_new, MakeUnique, MakeShapeSet
[task 2020-04-25T11:12:59.299Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.299Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at nsSupportsWeakReference::GetWeakReference, NS_GetWeakReference, do_GetWeakReference, nsMaybeWeakPtrArray
[task 2020-04-25T11:12:59.299Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame nsSupportsWeakReference::GetWeakReference matched a expected leak
[task 2020-04-25T11:12:59.300Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_calloc, js_pod_arena_calloc, maybe_pod_arena_calloc, pod_calloc
[task 2020-04-25T11:12:59.300Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_calloc matched a expected leak
[task 2020-04-25T11:12:59.300Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_pod_arena_malloc, maybe_pod_arena_malloc, js::AllocateObjectBuffer
[task 2020-04-25T11:12:59.300Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.301Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::Console::CreateInstance, mozilla::dom::console_Binding::createInstance, CallJSNative, js::InternalCallOrConstruct
[task 2020-04-25T11:12:59.302Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::dom::Console::CreateInstance matched a expected leak
[task 2020-04-25T11:12:59.302Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::detail::HashTable, mozilla::detail::HashTable, reserve, reserve
[task 2020-04-25T11:12:59.303Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame reserve matched a expected leak
[task 2020-04-25T11:12:59.305Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_pod_arena_malloc, maybe_pod_arena_malloc, js::detail::OrderedHashTable
[task 2020-04-25T11:12:59.305Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.306Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at AllocateProtoAndIfaceCache, mozilla::dom::CreateGlobal, mozilla::dom::Window_Binding::Wrap, CreateNativeGlobalForInner
[task 2020-04-25T11:12:59.306Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame AllocateProtoAndIfaceCache matched a expected leak
[task 2020-04-25T11:12:59.307Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::PerformanceMainThread::CreateNavigationTimingEntry, mozilla::dom::Performance::CreateForMainThread, nsPIDOMWindowInner::CreatePerformanceObjectIfNeeded, nsPIDOMWindowInner::GetPerformance
[task 2020-04-25T11:12:59.307Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::dom::PerformanceMainThread::CreateNavigationTimingEntry matched a expected leak
[task 2020-04-25T11:12:59.308Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::detail::HashTable, mozilla::detail::HashTable, putNew, putNew
[task 2020-04-25T11:12:59.308Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame putNew matched a expected leak
[task 2020-04-25T11:12:59.308Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at Malloc, nsTArray_base, nsTArray_Impl, InsertElementAt
[task 2020-04-25T11:12:59.308Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame Malloc matched a expected leak
[task 2020-04-25T11:12:59.309Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at nsGlobalWindowInner::GetApplicationCache, mozilla::dom::Window_Binding::get_applicationCache, mozilla::dom::binding_detail::GenericGetter, CallJSNative
[task 2020-04-25T11:12:59.309Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame nsGlobalWindowInner::GetApplicationCache matched a expected leak
[task 2020-04-25T11:12:59.309Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_realloc, js_pod_arena_realloc, maybe_pod_arena_realloc, pod_realloc
[task 2020-04-25T11:12:59.309Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_realloc matched a expected leak
[task 2020-04-25T11:12:59.312Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, nsTSubstring
[task 2020-04-25T11:12:59.313Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-25T11:12:59.313Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_calloc, js_pod_arena_calloc, maybe_pod_arena_calloc, maybe_pod_calloc
[task 2020-04-25T11:12:59.314Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_calloc matched a expected leak
[task 2020-04-25T11:12:59.314Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, mozilla::net::nsStandardURL::BuildNormalizedSpec
[task 2020-04-25T11:12:59.314Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-25T11:12:59.315Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::Performance::CreateForMainThread, nsPIDOMWindowInner::CreatePerformanceObjectIfNeeded, nsPIDOMWindowInner::GetPerformance, mozilla::dom::Window_Binding::get_performance
[task 2020-04-25T11:12:59.316Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::dom::Performance::CreateForMainThread matched a expected leak
[task 2020-04-25T11:12:59.316Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js::irregexp::BoyerMooreLookahead::EmitSkipInstructions, js::irregexp::ChoiceNode::Emit, js::irregexp::RegExpCompiler::Assemble
[task 2020-04-25T11:12:59.316Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.317Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::ProtoAndIfaceCache::PageTableCache::EntrySlotOrCreate, mozilla::dom::ProtoAndIfaceCache::EntrySlotOrCreate, mozilla::dom::WindowGlobalChild_Binding::CreateInterfaceObjects, mozilla::dom::GetPerInterfaceObjectHandle
[task 2020-04-25T11:12:59.317Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame mozilla::dom::ProtoAndIfaceCache::PageTableCache::EntrySlotOrCreate matched a expected leak
[task 2020-04-25T11:12:59.318Z] 11:12:59 INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at Realloc, nsTArray_base, nsTArray_Impl, AppendElement
[task 2020-04-25T11:12:59.320Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at std::sys::unix::alloc::_$LT$impl$u20$core..alloc..GlobalAlloc$u20$for$u20$std..alloc..System$GT$::alloc, __rdl_alloc, alloc::alloc::alloc, _$LT$alloc..alloc..Global$u20$as$u20$core..alloc..Alloc$GT$::alloc
[task 2020-04-25T11:12:59.321Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame alloc::alloc::alloc::h60447f72c515ace6 matched a expected leak
[task 2020-04-25T11:12:59.321Z] 11:12:59 INFO - TEST-FAIL | LeakSanitizer | leak at js_arena_malloc, js_pod_arena_malloc, maybe_pod_arena_malloc, make_pod_arena_array
[task 2020-04-25T11:12:59.321Z] 11:12:59 INFO - INFO | LeakSanitizer | Frame js_arena_malloc matched a expected leak
[task 2020-04-25T11:12:59.323Z] 11:12:59 INFO - Closing logging queue
[task 2020-04-25T11:12:59.323Z] 11:12:59 INFO - queue closed
[task 2020-04-25T11:12:59.330Z] 11:12:59 INFO - INFO | runtests.py | ASan using symbolizer at /builds/worker/workspace/build/application/firefox/llvm-symbolizer
[task 2020-04-25T11:12:59.330Z] 11:12:59 INFO - LSan enabled.
[task 2020-04-25T11:12:59.331Z] 11:12:59 INFO - INFO | runtests.py | ASan running in default memory configuration
[task 2020-04-25T11:12:59.331Z] 11:12:59 INFO - UBSan enabled.
[task 2020-04-25T11:12:59.332Z] 11:12:59 INFO - Application command: /builds/worker/workspace/build/application/firefox/firefox --marionette about:blank -profile /tmp/tmpF_S_Hl
[task 2020-04-25T11:12:59.333Z] 11:12:59 INFO - Starting runner
[task 2020-04-25T11:13:00.470Z] 11:13:00 INFO - ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[task 2020-04-25T11:13:00.471Z] 11:13:00 INFO - TEST-OK | /encoding/legacy-mb-japanese/euc-jp/eucjp-encode-form-cseucpkdfmtjapanese.html?2001-3000 | took 6344ms

Comment 1

[Intermittent Failures Robot]

5 failures in 5582 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5

Platform breakdown:

• linux1804-64-asan: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-04-20&endday=2020-04-26&tree=all

Comment 2

[Intermittent Failures Robot]

9 failures in 5423 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 8

Platform breakdown:

• linux1804-64-asan: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-04-27&endday=2020-05-03&tree=all

Comment 3

[Intermittent Failures Robot]

15 failures in 5276 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 7
• mozilla-beta: 7
• autoland: 1

Platform breakdown:

• linux1804-64-asan: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-05-04&endday=2020-05-10&tree=all

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This fails frequently in central-as-beta simulations since 2020-04-25, on beta since central merged to it in early May, and rarely on autoland. Push log for likely regression range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=f3686ddab4145c34d0663ff0a0cbecdde47ed38b&tochange=d8a8178627c4fab05f1eb46ee40f55db207c7c18

Henri, the leak reports gets printed during /encoding/ tests. Can you check if it's related to the handling of encoding or belongs elsewhere (e.g. JavaScript Engine)? Thank you in advance.

Comment 5

[Intermittent Failures Robot]

11 failures in 5488 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 6
• mozilla-beta: 4

Platform breakdown:

• linux1804-64-asan: 11

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 6

[Henri Sivonen (:hsivonen)]

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #4)

This fails frequently in central-as-beta simulations since 2020-04-25, on beta since central merged to it in early May, and rarely on autoland. Push log for likely regression range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=f3686ddab4145c34d0663ff0a0cbecdde47ed38b&tochange=d8a8178627c4fab05f1eb46ee40f55db207c7c18

Henri, the leak reports gets printed during /encoding/ tests. Can you check if it's related to the handling of encoding or belongs elsewhere (e.g. JavaScript Engine)? Thank you in advance.

I think this belongs elsewhere. The encoding tests themselves are not meant to exercise the things that leak in any particular way.

I expect this to be related to harness timeouts.

Comment 7

[Intermittent Failures Robot]

23 failures in 4727 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 10
• try: 4
• autoland: 9

Platform breakdown:

• linux1804-64-asan: 23

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-05-18&endday=2020-05-24&tree=all

Comment 8

[Julien Cristau [:jcristau]]

James, can you look at this this week? 78 merges to beta Monday next week.

Comment 9

[James Graham [:jgraham]]

OK, so the encoding thing is a red-herring, because wpt is now running with multiple processes for perf reasons it's a little harder to tell which things are actually causing the leak. In this case it's html/browsers/offline/introduction-4/ and I think that Bug 1637486 added these leaks to the allow list on central.

Comment 10

[Intermittent Failures Robot]

7 failures in 5458 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 3
• try: 1
• mozilla-release: 3

Platform breakdown:

• linux1804-64-asan: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 11

[Julien Cristau [:jcristau]]

Sebastian tells me this is no longer happening.

Comment 12

[Intermittent Failures Robot]

1 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• linux1804-64-asan: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633169&startday=2020-06-01&endday=2020-06-07&tree=all