439123 - Assertion failure in libpkix at shutdown

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Christophe Ravel]

===============================================
Trust anchor: ROOT_7Root.der
-----------------------------------------------
vfychain -d EE1DB -pp -v CA1EE1cert.der BRIDGE_1_1BRIDGE_2_1cert.der BRIDGE_1_2BRIDGE_2_1cert.der BRIDGE_1_3BRIDGE_2_1cert.der BRIDGE_2_1CA1cert.der ROOT_1BRIDGE_1_1cert.der ROOT_2BRIDGE_1_1cert.der ROOT_3BRIDGE_1_1cert.der ROOT_4BRIDGE_1_2cert.der ROOT_5BRIDGE_1_2cert.der ROOT_6BRIDGE_1_2cert.der ROOT_7BRIDGE_1_3cert.der ROOT_8BRIDGE_1_3cert.der ROOT_9BRIDGE_1_3cert.der -t ROOT_7Root.der
Chain is good!
Root Certificate Subject:: "CN=ROOT_7 ROOT CA,O=ROOT_7,C=US"
Certificate 1 Subject: "CN=EE1 EE,O=CA1,C=US"
Certificate 2 Subject: "CN=CA1 INTERMEDIATE,O=CA1,C=US"
Certificate 3 Subject: "CN=BRIDGE_2_1 BRIDGE,O=BRIDGE_2_1,C=US"
Certificate 4 Subject: "CN=BRIDGE_1_3 BRIDGE,O=BRIDGE_1_3,C=US"
Assertion failure: objType < PKIX_NUMTYPES, at pkix_pl_object.c:136
Abort - core dumped
-----------------------------------------------
RESULT: FAIL
===============================================

Note: this is with a debug build of NSS.

Comment 1

[Christophe Ravel]

$ pstack core
core 'core' of 7681:    /home/chravel/security/securitytip/ws/mozilla/dist/SunOS5.10_DBG.OBJ/b
 fed469ec _lwp_kill (6, 0, fed72cb8, fed298e4, ffffffff, 6) + 8
 fecc1118 abort    (44, 1, fed1d070, ad318, fed71318, 0) + 110
 fef15a6c PR_Assert (ff238cbc, ff238cd4, 88, 5dac0, fed71840, 0) + 94
 ff1c8b28 pkix_pl_Object_GetHeader (9e3e4, ffbfec4c, 5dac0, 5dac0, 0, 96921) + 190
 ff1ca820 PKIX_PL_Object_DecRef (9e3e4, 5dac0, 5dac0, 5dac0, fed71840, 0) + 188
 ff166dfc pkix_List_Destroy (9f1bc, 5dac0, 1, 1, 0, 9e4d9) + 214
 ff1ca9b8 PKIX_PL_Object_DecRef (9f1bc, 5dac0, 5dac0, 5dac0, c, c) + 320
 ff166e64 pkix_List_Destroy (98364, 5dac0, 1, 1, 0, c) + 27c
 ff1ca9b8 PKIX_PL_Object_DecRef (98364, 5dac0, 5dac0, 5dac0, fed71840, 0) + 320
 ff166e64 pkix_List_Destroy (a0354, 5dac0, 1, 1, 0, 9f549) + 27c
 ff1ca9b8 PKIX_PL_Object_DecRef (a0354, 5dac0, 5dac0, 5dac0, 9, 9) + 320
 ff166dfc pkix_List_Destroy (9f5fc, 5dac0, 1, 1, 0, 9) + 214
 ff1ca9b8 PKIX_PL_Object_DecRef (9f5fc, 5dac0, 5dac0, 5dac0, fed71840, 0) + 320
 ff166e64 pkix_List_Destroy (9f564, 5dac0, 1, 1, 0, 9e4d9) + 27c
 ff1ca9b8 PKIX_PL_Object_DecRef (9f564, 5dac0, 5dac0, 5dac0, 5dac0, 0) + 320
 ff166e64 pkix_List_Destroy (9f40c, 5dac0, 1, 1, 0, 5ef19) + 27c
 ff1ca9b8 PKIX_PL_Object_DecRef (9f40c, 5dac0, 5dac0, 5dac0, fed71840, 0) + 320
 ff1c3e10 pkix_pl_HashTable_Destroy (5f07c, 5dac0, 1, 1, 0, 43259) + 2c8
 ff1ca9b8 PKIX_PL_Object_DecRef (5f07c, 5dac0, ffbff630, 0, ff3f42e8, 0) + 320
 ff13ff88 PKIX_Shutdown (5dac0, 2c, ffbff72c, 0, ff00, 80808080) + 4e8
 ff0309c4 NSS_Shutdown (658a8, 0, ffbff72c, 0, 3d508, 15) + a4
 0001635c main     (15, ffbff90c, ffbff964, 3dc00, ff390100, ff390140) + 1184
 000146a8 _start   (0, 0, 0, 0, 0, 0) + 108

Comment 2

[Christophe Ravel]

PKI configuration:

   ROOT_1  ROOT_2  ROOT_3   ROOT_4  ROOT_5 ROOT_6   ROOT_7  ROOT_8 ROOT_9
         \    |   /               \   |   /               \   |   /
         BRIDGE_1_1               BRIDGE_1_2              BRIDGE_1_3
                   \                  |                   /
                    \-----------------+------------------/
                                      |
                                   BRIDGE_2_1
                                      |
                                     CA1
                                      |
                                     EE1

Comment 3

[Christophe Ravel]

Attachment: Certs used for the PKI configuration

Root certs:
ROOT_1Root.der
ROOT_2Root.der
ROOT_3Root.der
ROOT_4Root.der
ROOT_5Root.der
ROOT_6Root.der
ROOT_7Root.der
ROOT_8Root.der
ROOT_9Root.der

BRIDGE_1_1 certs:
ROOT_1BRIDGE_1_1cert.der
ROOT_2BRIDGE_1_1cert.der
ROOT_3BRIDGE_1_1cert.der

BRIDGE_1_2 certs:
ROOT_4BRIDGE_1_2cert.der
ROOT_5BRIDGE_1_2cert.der
ROOT_6BRIDGE_1_2cert.der

BRIDGE_1_3 certs:
ROOT_7BRIDGE_1_3cert.der
ROOT_8BRIDGE_1_3cert.der
ROOT_9BRIDGE_1_3cert.der

BRIDGE_2_1 certs:
BRIDGE_1_1BRIDGE_2_1cert.der
BRIDGE_1_2BRIDGE_2_1cert.der
BRIDGE_1_3BRIDGE_2_1cert.der

CA1 cert:
BRIDGE_2_1CA1cert.der

EE1 cert:
CA1EE1cert.der

Note: cert are named with the following convention: IssuerEntity

Comment 4

[Christophe Ravel]

The optimized build doesn't crash with this test.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

There appear to be 7 or more levels of recursion in that stack.  
I count PKIX_PL_Object_DecRef 8 times in that stack.
I'm surprised there are any.  

Alexei, when a pkix_list is being destroyed, does it call itself recursively 
for every item in the list?  If so, that won't fly!

Comment 6

[Alexei Volkov]

(In reply to comment #5)
> There appear to be 7 or more levels of recursion in that stack.  
> I count PKIX_PL_Object_DecRef 8 times in that stack.
> I'm surprised there are any.
The stack of multiple calls of DecRef function is possible when destructing an object that has references to other objects. In this case a PKIX_List object get destroyed. "next" field of PKIX_List structure is the pointer to PKIX_List. In case of the crash fifth element of the list get destroyed. 
  
> 
> Alexei, when a pkix_list is being destroyed, does it call itself recursively 
> for every item in the list?  If so, that won't fly!
It does not call DecRef function a pointer to itself. It calls the destructor with a pointer to a next element in the list.  

Comment 7

[Alexei Volkov]

The reason of failure is the regression brought by patch of bug  438685. Regression fix is integrated. Closing this bug.

Comment 8

[Nelson Bolyard (seldom reads bugmail)]

Christophe, did this failure occur with a 3.12 release candiate? 
Or did it occur with an NSS trunk build that is newer than the release 
candidate?

Comment 9

[Christophe Ravel]

I ran the test with a NSS trunk build from last week (just after Alexei first fix for the bug within the bubble sort code).

Comment 10

[Nelson Bolyard (seldom reads bugmail)]

OK, so this was not a bug in 3.12, but rather was a bug in the unreleased
trunk.  That's an important distinction, IMO.