1755267 - NSS-Try LeakSanitizer random test failures

Status: RESOLVED FIXED

(NSS :: Tools, defect, P3)

Description

[Leander Schwarz]

On NSS-Try random tests fail with LeakSanitizer errors. The same tests might or might not fail due to LeakSanitizer if run again.

LeakSanitizer mostly reports the following messages on failure:

==45==LeakSanitizer has encountered a fatal error.
==47==LeakSanitizer has encountered a fatal error.

See failed tests in example runs:
https://treeherder.mozilla.org/jobs?repo=nss-try&revision=d021832941f79b89365feba6c155e590bd12f3b6
https://treeherder.mozilla.org/jobs?repo=nss-try&revision=92a29c0e93b0d90b670b2a62b088c646164d0bd4

Comment 1

[John Schanck [:jschanck]]

Sample failure with LSAN_OPTIONS=verbosity=1:log_threads=1:
https://firefoxci.taskcluster-artifacts.net/hJLPjJn9SgWwQlUB_JKb2w/0/public/logs/live_backing.log

This gives a little more context:

==49==Could not attach to thread 47 (errno 13).
==49==Could not attach to thread 48 (errno 13).
==49==Failed suspending threads.
==47==LeakSanitizer has encountered a fatal error.

Which really makes this seem like an instance of https://github.com/google/sanitizers/issues/764.

We're already setting a taskcluster feature that should enable the SYS_PTRACE capability https://hg.mozilla.org/projects/nss/file/tip/automation/taskcluster/graph/src/extend.js#l680, so maybe this is a problem with the version of Docker that we're using?

mt, any thoughts?

Comment 2

[Martin Thomson [:mt:]]

Yeah, we had something like this happen before. I wasn't able to find the bug, but I believe that it was addressed by someone on the taskcluster team.

Comment 3

[Julien Cristau [:jcristau]]

Attachment: Bug 1755267 - run linux tests on nss-t/t-linux-xlarge-gcp.

NSS tasks using LSAN seem to run into frequent failures due to ptrace(2)
failing with EACCES (Permission Denied), apparently coming from the
apparmor profile for docker on the VM.

Until now Linux tests tasks were using the nss-{1,3}/linux-gcp pools,
which use the same base image as gecko builders. This switches them to
a new pool using the same base image as used by gecko's test tasks,
where ptrace appears to work reliably.

Comment 4

[John Schanck [:jschanck]]

https://hg.mozilla.org/projects/nss/rev/aeb46bf24a78b758a19e616b27c79eaaad33814e