240988 - Third party PKCS#11 module crashes during SSL/TLS handshake.

Status: RESOLVED WORKSFORME

(Core Graveyard :: Security: UI, defect)

Description

[Markus Heintel]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7b) Gecko/20040316
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7b) Gecko/20040316

We are in the development of a PKCS#11 library module for Siemens CardOS(R)
smart cards.

The following tests with Mozilla 1.7 on Mac OS X worked without any problem:
- Load PKCS#11 module.
- Log on to smart card.
- Import PKCS#12 file onto smart card.
- S/Mime Decryption/Encryption.
- S/Mime Signature/Verification.

Unfortunately we weren't that lucky with SSL/TLS Client Authentication. Pointing
the browser to a location that requires SSL/TLS client authentication crashes
Mozilla with the first call to the PKCS#11 library (either a C_Login() or a
C_FindObjectsInit() depending on whether I logged onto the token before).

The crash occurs somewhere within our library. So I don't want to blame you for
our bugs. However I've got really no idea what the problem could be. All works
fine (and really stable) in the scenarious listed above. I also tried Mozilla
version 1.6 - same results. A test application that sends the same calls to the
PKCS#11 library than Mozilla runs OK. Is it possible that the Mozilla crypto
engine messes something up with either stack or memory during SSL/TLS?

Everything works OK on Linux (SuSE 9.0, PCSCLite 1.2) as well as on Windows boxes.

It would be great if I could get any hints from you. Don't hesitate to contact
me in case any information is missing.

Thanks a lot,

Markus







Reproducible: Always
Steps to Reproduce:
1. Start Mozilla
2. Load third party PKCS#11 library module
3. Point to a location that requires client authentication

Actual Results:  
Mozilla crashes

Comment 1

[Greg K.]

Markus, even if the crash is your lib's fault, it still might be useful to
attach the stack trace generated by the crash to this bug.

Comment 2

[Markus Heintel]

Attachment: Stack dump

Stack dump as generated by the Apple Crash Report wizard.

Comment 3

[Greg K.]

Markus, you might also try posting a message referencing this bug to
<news://news.mozilla.org:119/netscape.public.mozilla.crypto>.

Comment 4

[Markus Heintel]

After reconfiguring and relinking the library in a different automake/autoconf 
environment (created on another Linux box) the problem is no longer 
reproducable. I suspect that there's been some misconfiguration in our build 
environment.