Last Comment Bug 261333 - New PK11_PubEncryptPKCS1 function always fails
: New PK11_PubEncryptPKCS1 function always fails
Product: NSS
Classification: Components
Component: Libraries (show other bugs)
: 3.9.3
: All All
: P1 critical (vote)
: 3.9.3
Assigned To: Nelson Bolyard (seldom reads bugmail)
: Bishakha Banerjee
Depends on:
  Show dependency treegraph
Reported: 2004-09-23 21:23 PDT by Nelson Bolyard (seldom reads bugmail)
Modified: 2004-09-24 14:44 PDT (History)
3 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---

Patch v1 (978 bytes, patch)
2004-09-23 22:15 PDT, Nelson Bolyard (seldom reads bugmail)
julien.pierre: review+
Details | Diff | Splinter Review

Description Nelson Bolyard (seldom reads bugmail) 2004-09-23 21:23:01 PDT
In NSS 3.9.3, we added several new functions.  One of them is failing in tests.
This makes NSS 3.9.3 unusable by one internal "customer" for whom it is 
being released.  

The bug is in function pk11_PubEncryptRaw.  It initializes the variable "out",
which is the size of the output buffer, with the length of data in the input
buffer.  This was always wrong.  The output buffer length always should be 
the length of the modulus (exclusing leading zero bytes in the modulus).  
But it always worked because the SSL2 code always passed in an input buffer 
whose length matched the modulus length.  

So, this is an old bug, now surfacing.  I should have caught this in my 
testing, and didn't.  Mea Culpa.  

I'd like to fix this for NSS 3.9.3 (even though the release candidate was
built yesterday).
Comment 1 Nelson Bolyard (seldom reads bugmail) 2004-09-23 21:23:52 PDT
tentatively marking P1 for 3.9.3
Comment 2 Nelson Bolyard (seldom reads bugmail) 2004-09-23 22:15:22 PDT
Created attachment 159940 [details] [diff] [review]
Patch v1

fix incorrect output buffer length.  (fix tested with customer program).
Comment 3 Nelson Bolyard (seldom reads bugmail) 2004-09-24 14:41:22 PDT
Thanks for the quick review, Jullien.  Checked in on 3.9 branch.

Checking in pk11skey.c;
/cvsroot/mozilla/security/nss/lib/pk11wrap/pk11skey.c,v  <--  pk11skey.c
new revision:; previous revision:
Comment 4 Nelson Bolyard (seldom reads bugmail) 2004-09-24 14:44:56 PDT
Checked in on trunk.

Checking in pk11obj.c;
/cvsroot/mozilla/security/nss/lib/pk11wrap/pk11obj.c,v  <--  pk11obj.c
new revision: 1.3; previous revision: 1.2

Note You need to log in before you can comment on or make changes to this bug.