1671574 - ThreadSanitizer: thread leak [@ pthread_create] for StartupCache Thread

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Christian Holler (:decoder)]

The attached crash information was detected while running CI tests with ThreadSanitizer on try revision 1fdd4f52d08fd1fb01b6c77c4a86b03af275855d.

I saw this thread leak intermittently in mochitest-bc but nowhere else. It means we didn't join the StartupCache thread properly, is this expected? If so, that's fine and we can just suppress this. However, I was skeptical since it only happens intermittently and only in some tests.

Doug, maybe you can answer this?

General information about TSan reports

Why fix races?

Data races are undefined behavior and can cause crashes as well as correctness issues. Compiler optimizations can cause racy code to have unpredictable and hard-to-reproduce behavior.

Rating

If you think this race can cause crashes or correctness issues, it would be great to rate the bug appropriately as P1/P2 and/or indicating this in the bug. This makes it a lot easier for us to assess the actual impact that these reports make and if they are helpful to you.

False Positives / Benign Races

Typically, races reported by TSan are not false positives [1], but it is possible that the race is benign. Even in this case it would be nice to come up with a fix if it is easily doable and does not regress performance. Every race that we cannot fix will have to remain on the suppression list and slows down the overall TSan performance. Also note that seemingly benign races can possibly be harmful (also depending on the compiler, optimizations and the architecture) [2][3].

[1] One major exception is the involvement of uninstrumented code from third-party libraries.
[2] http://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
[3] How to miscompile programs with "benign" data races: https://www.usenix.org/legacy/events/hotpar11/tech/final_files/Boehm.pdf

Suppressing unfixable races

If the bug cannot be fixed, then a runtime suppression needs to be added in mozglue/build/TsanOptions.cpp. The suppressions match on the full stack, so it should be picked such that it is unique to this particular race. The bug number of this bug should also be included so we have some documentation on why this suppression was added.

Comment 1

[Christian Holler (:decoder)]

Attachment: Detailed Crash Information

Comment 2

[Christian Holler (:decoder)]

Attachment: Bug 1671574 - Add temporary suppression for StartupCache thread leak. r?gankro

Depends on D94286

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a39a3000939
Add temporary suppression for StartupCache thread leak. r=Gankro

Comment 4

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/7a39a3000939

Comment 5

[Ted Campbell [:tcampbell]]

The "StartupCache" thread races with main-thread and tries to touch pages of an mmap'd buffer in order to trigger OS-level paging. It has no effect on correctness and so is never joined with. Perhaps there is a better way to indicate this? Otherwise it may just be a justifiable suppression.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:decoder, maybe it's time to close this bug?

Comment 7

[Christian Holler (:decoder)]

Kris, can you mark this as suppression as permanent per comment 5 and close the bug? Thanks!

Comment 8

[Kris Wright :KrisWright]

(In reply to Christian Holler (:decoder) from comment #7)

Kris, can you mark this as suppression as permanent per comment 5 and close the bug? Thanks!

Works for me, I will have this updated shortly.

Comment 9

[Kris Wright :KrisWright]

Attachment: Bug 1671574 - Add perma TSan suppression to StartupCache

It looks like this suppression was removed downstream, but StartupCache's behavior still exists so it most likely stopped causing failures because of some other suppression interacting with it. I decided the best course of action was to put it back as a perma suppression in gecko in case we run into it again.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/304b42e6f3d8
Add perma TSan suppression to StartupCache r=decoder

Comment 11

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/304b42e6f3d8