2nd argument in nsPSMComponent::HashEnd(..) should be an unsigned char**

VERIFIED FIXED

Status

()

Core
Security
P3
normal
VERIFIED FIXED
18 years ago
18 years ago

People

(Reporter: neeti, Assigned: Mitchell Stoltz (not reading bugmail))

Tracking

Trunk
x86
Windows NT
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

18 years ago
Data is getting lost in nsPSMComponent::HashEnd(PRUint32 id,char** hash, 
PRUint32* hashlen, PRUint32 maxLen) because HashEnd(...) calls 

if(CMT_HASH_End(controlConnection, id, (unsigned char*)*hash,
                        (CMUint32*)hashlen, maxLen) != CMTSuccess) in which 
the second argument "hash" is an unsigned char, whereas, we have "hash" as 
char** in HashEnd(...). 

This is causing the hashing algorithm to return incorrect values.i.e all values 
higher than 128 are treated as negative, hence we are getting the following 
results.

// results from psm-glue (incorrect)
ffffffdd 60 ffffff85 ffffffc7 fffffffb ffffffad 6b ffffffd5 67 61 01 ffffffc4 
ffffffce 7e 30 ffffffc7 

// results from 4.x hashing 
dd 60 85 c7 fb ad 6b d5 67 61 01 c4 ce 7e 30 c7 

This effects verification of netscape.cfg file by preferences.
(Reporter)

Comment 1

18 years ago
Reassigning to dougt.
(Reporter)

Updated

18 years ago
Summary: 2nd argument in nsPSMComponent::HashEnd(..) should be an unsigned char → 2nd argument in nsPSMComponent::HashEnd(..) should be an unsigned char**
(Reporter)

Comment 2

18 years ago
Reassigning
Assignee: norris → dougt

Comment 3

18 years ago
mstoltz
Assignee: dougt → mstoltz
(Assignee)

Comment 4

18 years ago
Looking at it. Please bear in mind I'm moving these functions out of 
nsIPSMComponent into a new interface, nsISignatureVerifier, because of bug 36853. 
The implementation will remain in nsPSMComponent. 
Status: NEW → ASSIGNED
(Assignee)

Updated

18 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → FIXED
(Assignee)

Comment 5

18 years ago
Fixed. This function is now in caps/idl/nsISignatureVerifier.idl, and it uses the 
native type |unsigned char*| as its out parameter. This makes it non-scriptable, 
but I don't think anyone was using it from JS anyway. 

Comment 6

18 years ago
Verified per Mitchell's comments.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.