1855305 - Memory leaks on new Ubuntu 22.04 Wayland test pool

Status: RESOLVED FIXED

(Core :: Widget: Gtk, defect, P3)

Description

[Andrew Halberstadt [:ahal]]

I'm standing up tests running against Wayland, and I noticed that most of the debug tasks have leakcheck failures.

Here's an example log:
https://treeherder.mozilla.org/logviewer?job_id=430268408&repo=try&lineNumber=6554

But clicking around this push, you can see leaks on most tasks:
https://treeherder.mozilla.org/jobs?repo=try&searchStr=debug&revision=b0c5e6e605d1de9a867dde8a21255f5cc700409f&selectedTaskRun=dsv2SB9ZRz6j7t0DUQ9gRw.0

I'm not sure where to put this as it's happening across so many different tests, so filing in Core :: General.

These tasks can be run on try via:

./mach try fuzzy --full -q "'wayland 'debug 'mochitest-plain"

(or similar)

Comment 1

[Andrew Halberstadt [:ahal]]

I'll also note that this is also the first time that we've run tasks against Ubuntu 22.04, so it's quite possible the leaks have nothing to do with Wayland, and everything to do with the Ubuntu upgrade.

Comment 2

[Andrew McCreight [:mccr8]]

It looks like the leaks are all Mutexes, so hopefully it is just one issue. If you have ASan builds working, you could try running a mochitest with that. If not, I could dig up the commands to get this leak checker to do more logging.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Gtk' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 4

[Martin Stránský [:stransky] (ni? me)]

Will look at it. Also Widget/Gtk is the best component for Wayland issues, I can forward it to different component after triage.

Comment 5

[Martin Stránský [:stransky] (ni? me)]

See some leaks but doesn't look related:

./mach test dom/tests/mochitest/bugs/

ERROR TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at wl_egl_window_create, mozilla::gl::GLContextEGL::CreateWaylandBufferSurface, mozilla::gl::GLContextEGL::CreateEGLSurfaceForCompositorWidget, mozilla::wr::RenderCompositorEGL::CreateEGLSurface
ERROR TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::gl::GLContextEGL::CreateWaylandBufferSurface, mozilla::gl::GLContextEGL::CreateEGLSurfaceForCompositorWidget, mozilla::wr::RenderCompositorEGL::CreateEGLSurface, mozilla::wr::RenderCompositorEGL::Resume

but will investigate it anyway.

Comment 6

[Emilio Cobos Álvarez [:emilio]]

LSan is different from leakcheck. I see the Mutex leaks.

Comment 7

[Emilio Cobos Álvarez [:emilio]]

Attachment: Allocation stack.

It seems to be the MozContainer lock.

This is easy to debug with:

XPCOM_MEM_LOG_CLASSES=Mutex ./mach mochitest dom/base/test/test_bug416317-1.html

Or so.

Comment 8

[Andrew McCreight [:mccr8]]

Yeah, I was just suggesting LSan because I couldn't remember the actual environment variables for regular leakchecking, so thanks for including that and the allocation stack.

Comment 9

[Emilio Cobos Álvarez [:emilio]]

With bug 1855457 and bug 1855461 I get no leakcheck failures locally. Andrew, can you give them a spin?

Comment 10

[Andrew Halberstadt [:ahal]]

Thanks for the quick investigation / fixes all!

I did end up scheduling a task on an asan build:
https://treeherder.mozilla.org/logviewer?job_id=430512469&repo=try&lineNumber=2588

But I guess we're passed that point now. I'll try a new push with those bugs applied shortly.

Comment 11

[Andrew Halberstadt [:ahal]]

Here's a try push with Emilio's patches:
https://treeherder.mozilla.org/jobs?repo=try&revision=cd1b481f0ca71e6641d107af4f13d0c9d1b1b2ac

Comment 12

[Emilio Cobos Álvarez [:emilio]]

Leakcheck failures are gone in comment 11, yay.

Comment 13

[Andrew Halberstadt [:ahal]]

Awesome, thanks Emilio!

I noticed one "YOU ARE LEAKING THE WORLD" message in this task:
https://treeherder.mozilla.org/logviewer?job_id=430549919&repo=try&lineNumber=12434

But this seems like a separate issue that I can triage later.