268526 - two subsequent C_DigestFinal calls are made on the same session during SSL

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Julien Pierre]

The following PKCS#11 trace was taken from selfserv running in single-threaded
mode. As you can see, C_DigestFinal gets called twice with the same hSession =
0x47 . The first call succeeds, but the second one fails with rv = 0x91,
CKR_OPERATION_NOT_INITIALIZED . I haven't looked yet where the failing call is
made from, but we should fix this sequence.

This invalid sequence is happening 11 times for each SSL full handshake in my
PKCS#11 server logs.

2[54da50]: C_DigestFinal
2[54da50]:   hSession = 0x47
2[54da50]:   pDigest = 0xfffffd7ffeb546d4
2[54da50]:   pulDigestLen = 0xfffffd7ffeb54300
2[54da50]:   rv = 0x0
2[54da50]: C_DigestUpdate
2[54da50]:   hSession = 0x48
2[54da50]:   pPart = 0xfffffd7ffeb54690
2[54da50]:   ulPartLen = 4
2[54da50]:   rv = 0x0
2[54da50]: C_DigestKey
2[54da50]:   hSession = 0x48
2[54da50]:   rv = 0x0
2[54da50]: C_DigestUpdate
2[54da50]:   hSession = 0x48
2[54da50]:   pPart = 0xfffffd7fff37ee98
2[54da50]:   ulPartLen = 40
2[54da50]:   rv = 0x0
2[54da50]: C_DigestFinal
2[54da50]:   hSession = 0x48
2[54da50]:   pDigest = 0xfffffd7ffeb54694
2[54da50]:   pulDigestLen = 0xfffffd7ffeb54300
2[54da50]:   rv = 0x0
2[54da50]: C_DigestFinal
2[54da50]:   hSession = 0x47
2[54da50]:   pDigest = 0x0
2[54da50]:   pulDigestLen = 0xfffffd7ffeb54138
2[54da50]:   rv = 0x91