Closed Bug 1560593 Opened 4 months ago Closed 2 months ago

Cleanup.sh script does not set error exit code for tests that "Failed with core"

Categories

(NSS :: Test, defect, P1, major)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kjacobs, Assigned: kjacobs)

Details

Attachments

(2 files, 1 obsolete file)

tests/common/cleanup.sh only counts tests non-crashing failures and ASAN errors when determining the exit code: https://searchfox.org/mozilla-central/source/security/nss/tests/common/cleanup.sh#56

This results in taskcluster reporting green for jobs that fail with a core dump. For example, Bogo fails on debug builds but is reported as passing:

#1  0x00007f413f29e801 in __GI_abort () at abort.c:79
#2  0x00007f413f9eb932 in PR_Assert (s=0x55a588471de3 "0", file=0x55a588471b20 "../../lib/ssl/ssl3con.c", ln=7432) at ../../../../pr/src/io/prlog.c:553
#3  0x000055a5883a1748 in ssl3_HandleCertificateRequest (ss=0x55a589ffe8d0, b=0x55a58a01029f 'N' <repeats 200 times>..., length=0) at ../../lib/ssl/ssl3con.c:7432
#4  0x000055a5883ab790 in ssl3_HandlePostHelloHandshakeMessage (ss=0x55a589ffe8d0, b=0x55a58a010294 "\002\001@", length=11) at ../../lib/ssl/ssl3con.c:11898
#5  0x000055a5883ab4a4 in ssl3_HandleHandshakeMessage (ss=0x55a589ffe8d0, b=0x55a58a010294 "\002\001@", length=11, endOfRecord=1) at ../../lib/ssl/ssl3con.c:11822
#6  0x000055a5883abb3a in ssl3_HandleHandshake (ss=0x55a589ffe8d0, origBuf=0x55a589ffeb80) at ../../lib/ssl/ssl3con.c:11996
#7  0x000055a5883acdf0 in ssl3_HandleNonApplicationData (ss=0x55a589ffe8d0, rType=ssl_ct_handshake, epoch=0, seqNum=3, databuf=0x55a589ffeb80)
    at ../../lib/ssl/ssl3con.c:12515
#8  0x000055a5883ad7e6 in ssl3_HandleRecord (ss=0x55a589ffe8d0, cText=0x7ffd35288b10) at ../../lib/ssl/ssl3con.c:12796
#9  0x000055a5883eb41b in ssl3_GatherCompleteHandshake (ss=0x55a589ffe8d0, flags=0) at ../../lib/ssl/ssl3gthr.c:512
#10 0x000055a5883ec067 in ssl_GatherRecord1stHandshake (ss=0x55a589ffe8d0) at ../../lib/ssl/sslcon.c:74
#11 0x000055a5883bde06 in ssl_Do1stHandshake (ss=0x55a589ffe8d0) at ../../lib/ssl/sslsecur.c:41
#12 0x000055a5883bef58 in SSL_ForceHandshake (fd=0x55a589ffe860) at ../../lib/ssl/sslsecur.c:384
#13 0x000055a58838a26a in TestAgent::Handshake (this=0x55a589ffe830) at ../../gtests/nss_bogo_shim/nss_bogo_shim.cc:421
#14 0x000055a58838a5c1 in TestAgent::DoExchange (this=0x55a589ffe830) at ../../gtests/nss_bogo_shim/nss_bogo_shim.cc:491
#15 0x000055a58838bd43 in RunCycle (cfg=std::unique_ptr<const Config> = {...}) at ../../gtests/nss_bogo_shim/nss_bogo_shim.cc:612
#16 0x000055a58838bf6d in main (argc=11, argv=0x7ffd35288fe8) at ../../gtests/nss_bogo_shim/nss_bogo_shim.cc:650```

Failed with core should be handled somehow other than just passing -- we also have the retrigger result for things that may be intermittent, and this might be appropriate here too.

Severity: normal → major
Priority: -- → P1
Assignee: nobody → kjacobs.bugzilla
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 3.46

This change introduced a new error:
/home/user/moz/nss/hg/nss/tests/common/cleanup.sh: line 63: [: -eq: unary operator expected

This happens if BUILD_OPT is undefined (empty).

In my build environment variable OBJDIR isn't set.

Oops. That seems like an easy fixup though.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #9087830 - Attachment is obsolete: true
Status: REOPENED → RESOLVED
Closed: 2 months ago2 months ago
Resolution: --- → FIXED

thanks!

You need to log in before you can comment on or make changes to this bug.