225626 - incorrect arena type in JSSL_JavaCertAuthCallback

Status: RESOLVED FIXED

(JSS Graveyard :: Library, defect)

Description

[Jamie Nicolson]

There is a hack in NSS whereby PLArenaPool* variables actually point at the data
type PORTArenaPool, which is a wrapper around a PLArenaPool. Since a PLArenaPool
is the first element in the PORTArenaPool struct, a PORTArenaPool* can be cast
to a PLArenaPool*. A PORTArenaPool is allocated, it is returned as a
PLArenaPool*, and then if the PORTArenaPool functionality is required, it is
cast back to a PORTArenaPool.

In JSSL_JavaCertAuthCallback we allocate an arena that is passed to
CERT_VerifyCert. The type we allocate is a PLArenaPool, which is in fact the
type indicated by the header files. However, NSS really expects it to be a
PORTArenaPool, and casts it to a PORTArenaPool* and tries to access memory in
the PORTArenaPool. At this point it is accessing memory that it shouldn't.

The solution is to allocate a PORTArenaPool instead of a PLArenaPool.

Comment 1

[Jamie Nicolson]

Attachment: proposed patch

This patch allocates a PORTArenaPool instead of a PLArenaPool.

Comment 2

[Jamie Nicolson]

Attachment: typo: DEFAULT_CHUNKSIZE should be DER_DEFAULT_CHUNKSIZE

Comment 3

[Wan-Teh Chang]

Attachment: Proposed patch v3

Changes from the previous patch:
1. Handle the failure of PORT_NewArena.
2. Fixed a memory leak on an error path.  Need to 'goto finish'
to execute the cleanup code before returning.

Jamie, please review.

Comment 4

[glen beasley]

Wan-Teh, I would like to check this in. I checked this fix it in to the
SUN_SECURITY_3_3_BRANCH early this year. 

Comment 5

[glen beasley]

confirmed Wan-teh checked in fix. target milestone set to 4.0 and closing bug. 

Comment 6

[Wan-Teh Chang]

This is fixed in JSS 3.5.  (There is a JSS 3.5 release.)