Closed Bug 1634215 Opened 5 years ago Closed 4 years ago

Intermittent LeakSanitizer | leak at PLDHashTable::ChangeTable, SearchTable, PLDHashTable::Remove, RemoveEntry

Categories

(Core :: DOM: Service Workers, defect, P3)

defect

Tracking

()

RESOLVED INVALID
Tracking Status
firefox76 --- unaffected
firefox77 --- affected
firefox78 --- affected

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [retriggered][stockwell unknown])

Filed by: dvarga [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=300063927&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/QYWr3VxOSNOjIwsD9g5dEg/runs/0/artifacts/public/logs/live_backing.log


Last TEST-START | /service-workers/service-worker/xsl-base-url.https.html

[task 2020-04-29T21:40:13.928Z] 21:40:13     INFO - Browser exited with return code 0
[task 2020-04-29T21:40:13.928Z] 21:40:13     INFO - PROCESS LEAKS None
[task 2020-04-29T21:40:13.929Z] 21:40:13     INFO - ERROR | LeakSanitizer | SUMMARY: AddressSanitizer: 7298 byte(s) leaked in 44 allocation(s).
[task 2020-04-29T21:40:13.929Z] 21:40:13     INFO - LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2020-04-29T21:40:13.929Z] 21:40:13     INFO - This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2020-04-29T21:40:13.929Z] 21:40:13     INFO - Allowed depth was 4
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::ContentParent::CreateBrowser, nsFrameLoader::TryRemoteBrowserInternal, nsFrameLoader::TryRemoteBrowser, EnsureRemoteBrowser
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::dom::ContentParent::CreateBrowser matched a expected leak
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::BrowsingContext::CreateDetached, CreateBrowsingContext, nsFrameLoader::Create, mozilla::dom::XULFrameElement::LoadSrc
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame CreateBrowsingContext matched a expected leak
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Realloc, nsTArray_base, nsTArray_base, nsTArray_Impl
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Realloc matched a expected leak
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsNodeSupportsWeakRefTearoff::GetWeakReference, NS_GetWeakReference, do_GetWeakReference, mozilla::LoadContext::LoadContext
[task 2020-04-29T21:40:13.931Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsNodeSupportsWeakRefTearoff::GetWeakReference matched a expected leak
[task 2020-04-29T21:40:13.932Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::WeakPtr, SupportsWeakPtr, mozilla::dom::PBrowserParent::PBrowserParent, mozilla::dom::BrowserParent::BrowserParent
[task 2020-04-29T21:40:13.932Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::WeakPtr matched a expected leak
[task 2020-04-29T21:40:13.932Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, mozilla::net::nsStandardURL::BuildNormalizedSpec
[task 2020-04-29T21:40:13.932Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-29T21:40:13.934Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::WeakPtr, WeakPtr, mozilla::Maybe, mozilla::ContentPrincipal::AddonPolicy
[task 2020-04-29T21:40:13.934Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::WeakPtr matched a expected leak
[task 2020-04-29T21:40:13.934Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::HttpBaseChannel::CloneLoadInfoForRedirect, mozilla::net::nsHttpChannel::RedirectToInterceptedChannel, mozilla::net::nsHttpChannel::Connect, mozilla::net::nsHttpChannel::ContinueOnBeforeConnect
[task 2020-04-29T21:40:13.934Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::net::HttpBaseChannel::CloneLoadInfoForRedirect matched a expected leak
[task 2020-04-29T21:40:13.935Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at NewPage, mozilla::Queue, mozilla::detail::EventQueueInternal, mozilla::PrioritizedEventQueue::PutEvent
[task 2020-04-29T21:40:13.935Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame NewPage matched a expected leak
[task 2020-04-29T21:40:13.935Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Create, mozilla::ThrottledEventQueue::Create, mozilla::dom::BrowsingContextGroup::BrowsingContextGroup, operator
[task 2020-04-29T21:40:13.935Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Create matched a expected leak
[task 2020-04-29T21:40:13.935Z] 21:40:13     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at PLDHashTable::ChangeTable, SearchTable, PLDHashTable::Remove, RemoveEntry
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::nsStandardURL::TemplatedMutator, mozilla::net::nsStandardURL::TemplatedMutator, std::function, operator
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::net::nsStandardURL::TemplatedMutator matched a expected leak
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsHashPropertyBagBase::SetPropertyAsInterface, mozilla::extensions::ChannelWrapper::Get, mozilla::dom::ChannelWrapper_Binding::get
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsHashPropertyBagBase::SetPropertyAsInterface matched a expected leak
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at operator, OrInsert, mozilla::dom::BrowsingContextGroup::GetOrCreate, mozilla::dom::BrowsingContext::CreateDetached
[task 2020-04-29T21:40:13.937Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame operator matched a expected leak
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::extensions::ChannelWrapper::ChannelWrapper, mozilla::extensions::ChannelWrapper::Get, mozilla::dom::ChannelWrapper_Binding::get
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::extensions::ChannelWrapper::ChannelWrapper matched a expected leak
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::BasePrincipal::CreateContentPrincipal, mozilla::BasePrincipal::CreateContentPrincipal, nsScriptSecurityManager::GetChannelURIPrincipal, mozilla::net::HttpBaseChannel::GetURIPrincipal
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::BasePrincipal::CreateContentPrincipal matched a expected leak
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::ThrottledEventQueue::Create, mozilla::dom::BrowsingContextGroup::BrowsingContextGroup, operator, OrInsert
[task 2020-04-29T21:40:13.938Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::ThrottledEventQueue::Create matched a expected leak
[task 2020-04-29T21:40:13.940Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::BrowserParent::GetLoadContext, mozilla::dom::BrowserHost::GetLoadContext, nsFrameLoader::LoadContext, mozilla::dom::FrameLoader_Binding::get_loadContext
[task 2020-04-29T21:40:13.940Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::dom::BrowserParent::GetLoadContext matched a expected leak
[task 2020-04-29T21:40:13.940Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, nsTSubstring
[task 2020-04-29T21:40:13.940Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-29T21:40:13.940Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsStringBuffer::Alloc, nsAttrValue::GetStringBuffer, SetTo, nsAttrValue::ParseStringOrAtom
[task 2020-04-29T21:40:13.941Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsStringBuffer::Alloc matched a expected leak
[task 2020-04-29T21:40:13.943Z] 21:40:13     INFO - Closing logging queue
[task 2020-04-29T21:40:13.943Z] 21:40:13     INFO - queue closed
[task 2020-04-29T21:40:13.943Z] 21:40:13     INFO - PROCESS LEAKS None
[task 2020-04-29T21:40:13.943Z] 21:40:13     INFO - LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2020-04-29T21:40:13.943Z] 21:40:13     INFO - This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - Allowed depth was 4
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::ContentParent::CreateBrowser, nsFrameLoader::TryRemoteBrowserInternal, nsFrameLoader::TryRemoteBrowser, EnsureRemoteBrowser
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::dom::ContentParent::CreateBrowser matched a expected leak
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::BrowsingContext::CreateDetached, CreateBrowsingContext, nsFrameLoader::Create, mozilla::dom::XULFrameElement::LoadSrc
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame CreateBrowsingContext matched a expected leak
[task 2020-04-29T21:40:13.944Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Realloc, nsTArray_base, nsTArray_base, nsTArray_Impl
[task 2020-04-29T21:40:13.945Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Realloc matched a expected leak
[task 2020-04-29T21:40:13.946Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsNodeSupportsWeakRefTearoff::GetWeakReference, NS_GetWeakReference, do_GetWeakReference, mozilla::LoadContext::LoadContext
[task 2020-04-29T21:40:13.946Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsNodeSupportsWeakRefTearoff::GetWeakReference matched a expected leak
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::WeakPtr, SupportsWeakPtr, mozilla::dom::PBrowserParent::PBrowserParent, mozilla::dom::BrowserParent::BrowserParent
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::WeakPtr matched a expected leak
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, mozilla::net::nsStandardURL::BuildNormalizedSpec
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::WeakPtr, WeakPtr, mozilla::Maybe, mozilla::ContentPrincipal::AddonPolicy
[task 2020-04-29T21:40:13.947Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::WeakPtr matched a expected leak
[task 2020-04-29T21:40:13.948Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::HttpBaseChannel::CloneLoadInfoForRedirect, mozilla::net::nsHttpChannel::RedirectToInterceptedChannel, mozilla::net::nsHttpChannel::Connect, mozilla::net::nsHttpChannel::ContinueOnBeforeConnect
[task 2020-04-29T21:40:13.948Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::net::HttpBaseChannel::CloneLoadInfoForRedirect matched a expected leak
[task 2020-04-29T21:40:13.948Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at NewPage, mozilla::Queue, mozilla::detail::EventQueueInternal, mozilla::PrioritizedEventQueue::PutEvent
[task 2020-04-29T21:40:13.948Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame NewPage matched a expected leak
[task 2020-04-29T21:40:13.948Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Create, mozilla::ThrottledEventQueue::Create, mozilla::dom::BrowsingContextGroup::BrowsingContextGroup, operator
[task 2020-04-29T21:40:13.949Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Create matched a expected leak
[task 2020-04-29T21:40:13.950Z] 21:40:13     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at PLDHashTable::ChangeTable, SearchTable, PLDHashTable::Remove, RemoveEntry
[task 2020-04-29T21:40:13.950Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::nsStandardURL::TemplatedMutator, mozilla::net::nsStandardURL::TemplatedMutator, std::function, operator
[task 2020-04-29T21:40:13.950Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::net::nsStandardURL::TemplatedMutator matched a expected leak
[task 2020-04-29T21:40:13.950Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsHashPropertyBagBase::SetPropertyAsInterface, mozilla::extensions::ChannelWrapper::Get, mozilla::dom::ChannelWrapper_Binding::get
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsHashPropertyBagBase::SetPropertyAsInterface matched a expected leak
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at operator, OrInsert, mozilla::dom::BrowsingContextGroup::GetOrCreate, mozilla::dom::BrowsingContext::CreateDetached
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame operator matched a expected leak
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::extensions::ChannelWrapper::ChannelWrapper, mozilla::extensions::ChannelWrapper::Get, mozilla::dom::ChannelWrapper_Binding::get
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::extensions::ChannelWrapper::ChannelWrapper matched a expected leak
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::BasePrincipal::CreateContentPrincipal, mozilla::BasePrincipal::CreateContentPrincipal, nsScriptSecurityManager::GetChannelURIPrincipal, mozilla::net::HttpBaseChannel::GetURIPrincipal
[task 2020-04-29T21:40:13.951Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::BasePrincipal::CreateContentPrincipal matched a expected leak
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::ThrottledEventQueue::Create, mozilla::dom::BrowsingContextGroup::BrowsingContextGroup, operator, OrInsert
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::ThrottledEventQueue::Create matched a expected leak
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::dom::BrowserParent::GetLoadContext, mozilla::dom::BrowserHost::GetLoadContext, nsFrameLoader::LoadContext, mozilla::dom::FrameLoader_Binding::get_loadContext
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame mozilla::dom::BrowserParent::GetLoadContext matched a expected leak
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at Alloc, nsTSubstring, nsTSubstring, nsTSubstring
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame Alloc matched a expected leak
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - TEST-FAIL | LeakSanitizer | leak at nsStringBuffer::Alloc, nsAttrValue::GetStringBuffer, SetTo, nsAttrValue::ParseStringOrAtom
[task 2020-04-29T21:40:13.953Z] 21:40:13     INFO - INFO | LeakSanitizer | Frame nsStringBuffer::Alloc matched a expected leak
[task 2020-04-29T21:40:14.110Z] 21:40:14     INFO - Got 0 unexpected results
[task 2020-04-29T21:40:14.110Z] 21:40:14     INFO - SUITE-END | took 1761s
[task 2020-04-29T21:40:14.179Z] 21:40:14     INFO - Removed font: Ahem.ttf
[task 2020-04-29T21:40:14.194Z] 21:40:14     INFO - Closing logging queue
[task 2020-04-29T21:40:14.194Z] 21:40:14     INFO - queue closed
[task 2020-04-29T21:40:14.428Z] 21:40:14     INFO - Return code: 0

This is from Bug 1633204. Range of retriggers.
Nika, please have a look over this. Thank you

Flags: needinfo?(nika)
Regressed by: 1633204
Whiteboard: [retriggered]
Has Regression Range: --- → yes

This directory appears to already have a ton of lsan ignores, including PLDHashTable::Add (https://searchfox.org/mozilla-central/rev/54f965e51e4f77866bec42737978d40d4c1fdfc5/testing/web-platform/meta/service-workers/service-worker/__dir__.ini#2). I'm guessing that the changes I made changed the behaviour in this test enough that the hashtable decided to shrink it's allocation at some point, meaning that the leaked allocation's stack changed, and stopped being ignored.

The correct fix would be to fix the mountain of leaks which are apparently occurring here, but the easiest is probably to just add the new leak signature to the list and avert our eyes.

Flags: needinfo?(nika)

I have a feeling that we're probably going to (or are already) run into situations like this frequently when changing code in directories with mountains of lsan exemptions. Is there something better we can do than just add more overly-general exemptions like PLDHashTable::ChangeTable?

Flags: needinfo?(james)

It seems like this might be the same issue as bug 1631819, as this directory should perhaps be completely excluded from lsan leak checking.

See Also: → 1631819

(In reply to :Nika Layzell (ni? for response) from comment #5)

It seems like this might be the same issue as bug 1631819, as this directory should perhaps be completely excluded from lsan leak checking.

I assume that it is not a Service Worker problem then? Thanks for looking into it!

Yeah, I don't have a good suggestion in the short term other than to change the exclusion rules. I guess I can look at adding lsan-disable: true or something as an option in the ini files.

Flags: needinfo?(james)
Severity: normal → S3
Priority: -- → P3

Set release status flags based on info from the regressing bug 1633204

Hi Nika! As bug 1631819 has been resolved a while ago but we still see this here: any other suggestions?

Flags: needinfo?(nika)

This particular WPT from my quick search already has a ridiculous number of lsan suppressions set up on it: https://searchfox.org/mozilla-central/rev/ab81b8552f4aa9696a2524f97fdfeb59d4dc31c1/testing/web-platform/meta/html/semantics/embedded-content/__dir__.ini#1

This already includes suppressions in PLDHashTable such as PLDHashTable::Add which would normally suppress hashtable leaks like this one. This particular leak seems to be one caused by a slightly different hashtable relocation code path. I'm guessing non-determinism is causing the stack for the hashtable's buffer leak to change intermittently, meaning that this is the same issue as the existing suppressed leak.

The proper fix here is probably to figure out why we're leaking when running these tests, and fix that issue, but the easy one is to add another suppression to the megalist.

Flags: needinfo?(nika)

Plan is to disable the LSAN at a directory level based on pragmatic concerns and the general low data-structure churn of ServiceWorkers in steady-state that would mitigate any worst-case scenario. Needinfo to do that.

Flags: needinfo?(bugmail)

(In reply to Andrew Sutherland [:asuth] (he/him) from comment #24)

Plan is to disable the LSAN at a directory level based on pragmatic concerns and the general low data-structure churn of ServiceWorkers in steady-state that would mitigate any worst-case scenario. Needinfo to do that.

I assume, this is still the plan here. Yaron, moving the ni to you, but not for immediate action.

Flags: needinfo?(bugmail) → needinfo?(ytausky)
Flags: needinfo?(ytausky) → needinfo?(jstutte)

This is not happening since March.

Status: NEW → RESOLVED
Closed: 4 years ago
Flags: needinfo?(jstutte)
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.