1600567 - ThreadSanitizer: data race [@ operator!] vs. [@ assign_assuming_AddRef] in APZCTreeManager

Status: RESOLVED FIXED

(Core :: Graphics, defect, P3)

Description

[Christian Holler (:decoder)]

The attached crash information was detected while running CI tests with ThreadSanitizer on mozilla-central revision 66531295716a.

Assuming the assertion at

https://hg.mozilla.org/mozilla-central/file/66531295716a/gfx/layers/apz/src/APZCTreeManager.cpp#l807

holds, I guess this could be safe in terms of memory-safety (modulo any kind of unpredictable compiler behavior, e.g. making the assignment non-atomic in some way). But it would still cause random behavior here:

https://hg.mozilla.org/mozilla-central/file/66531295716a/gfx/layers/apz/src/APZCTreeManager.cpp#l3375

Hope this helps :)

General information about TSan reports

Why fix races?

Data races are undefined behavior and can cause crashes as well as correctness issues. Compiler optimizations can cause racy code to have unpredictable and hard-to-reproduce behavior.

Rating

If you think this race can cause crashes or correctness issues, it would be great to rate the bug appropriately as P1/P2 and/or indicating this in the bug. This makes it a lot easier for us to assess the actual impact that these reports make and if they are helpful to you.

False Positives / Benign Races

Typically, races reported by TSan are not false positives [1], but it is possible that the race is benign. Even in this case it would be nice to come up with a fix if it is easily doable and does not regress performance. Every race that we cannot fix will have to remain on the suppression list and slows down the overall TSan performance. Also note that seemingly benign races can possibly be harmful (also depending on the compiler, optimizations and the architecture) [2][3].

[1] One major exception is the involvement of uninstrumented code from third-party libraries.
[2] http://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
[3] How to miscompile programs with "benign" data races: https://www.usenix.org/legacy/events/hotpar11/tech/final_files/Boehm.pdf

Suppressing unfixable races

If the bug cannot be fixed, then a runtime suppression needs to be added in mozglue/build/TsanOptions.cpp. The suppressions match on the full stack, so it should be picked such that it is unique to this particular race. The bug number of this bug should also be included so we have some documentation on why this suppression was added.

Comment 1

[Christian Holler (:decoder)]

Attachment: Detailed Crash Information

Comment 2

[Botond Ballo [:botond]]

Indeed, every access to mRootNode should technically be guarded by mTreeLock. It should be easy to tweak SendSubtreeTransformsToChromeMainThread() so that this holds.

Comment 3

[Botond Ballo [:botond]]

This is a regression from bug 1592435.

Comment 4

[Botond Ballo [:botond]]

Attachment: Bug 1600567 - Avoid accessing APZCTreeManager::mRootNode without mTreeLock held in SendSubtreeTransformsToChromeMainThread(). r=tnikkel

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cfab514df88
Avoid accessing APZCTreeManager::mRootNode without mTreeLock held in SendSubtreeTransformsToChromeMainThread(). r=tnikkel

Comment 6

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/2cfab514df88

Comment 7

[Julien Cristau [:jcristau]]

Might be worth uplifting so we don't ship this?

Comment 8

[Botond Ballo [:botond]]

Comment on attachment 9112911 [details]
Bug 1600567 - Avoid accessing APZCTreeManager::mRootNode without mTreeLock held in SendSubtreeTransformsToChromeMainThread(). r=tnikkel

Beta/Release Uplift Approval Request

• User impact if declined: Data race while accessing APZCTreeManager::mRootNode. I believe it's benign (can't cause any user-noticeable effect), but it's better not to ship it since we know about it.
• Is this code covered by automated tests?: Unknown
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce: I'm not in a position to verify this, but Christian might be able to by re-running the TSan check on the new code.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Very simple fix, just moves the access into a code block below that is covered by a lock.
• String changes made/needed:

Comment 9

[Julien Cristau [:jcristau]]

Comment on attachment 9112911 [details]
Bug 1600567 - Avoid accessing APZCTreeManager::mRootNode without mTreeLock held in SendSubtreeTransformsToChromeMainThread(). r=tnikkel

move a check under appropriate lock, new regression in 72, approved for 72.0b5

Comment 10

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/releases/mozilla-beta/rev/e2829f8c04c1