962777 - Move RSA_BlockType definition back to a header file

Status: RESOLVED WONTFIX

(NSS :: Libraries, defect)

Description

[Kai Engert [:KaiE:]]

Bug 836019 moved RSA_BlockType 
  from header file lib/softoken/softoknt.h
           to file lib/freebl/rsapkcs.c

This is inconvenient, because the nss-pem pkcs#11 module requires those constants.

Can we please move it back to a header file?

How about moving it to blapit.h?

Comment 1

[Kai Engert [:KaiE:]]

Attachment: p-962777

(this also includes the comment change suggested in bug 962762)

Comment 2

[Ryan Sleevi]

I really dislike exposing this internal detail of RSA, and shouldn't be part of the public API.

Why does the nss-pem require it? It should not be messing with raw RSA block types.

Comment 3

[Kai Engert [:KaiE:]]

(In reply to Ryan Sleevi from comment #2)
> Why does the nss-pem require it?

I didn't write the code, but here is the file that uses it:

https://git.fedorahosted.org/cgit/nss-pem.git/tree/nss/lib/ckfw/pem/rsawrapr.c

Comment 4

[Kai Engert [:KaiE:]]

FYI, we already remove a patch on top of the above file, pending to be applied,
which removes some use of the constants, and in order to fix the build, adds a copy of the type definition.

http://pkgs.fedoraproject.org/cgit/nss.git/tree/0039-Sync-up-with-nss-3.15.4-changes-in-freebl-and-softok.patch

Nevertheless, there are still functions like rsa_FormatOneBlock and pem_RSA_Sign which use the enum values.

Comment 5

[Kai Engert [:KaiE:]]

(In reply to Ryan Sleevi from comment #2)
> I really dislike exposing this internal detail of RSA, and shouldn't be part
> of the public API.

Since lib-pem is designed to be built as part of NSS (and we still intend to merge it eventually), it would be OK to move the type to one the other header files that are private.

Do you have a preference which header file would be good?

Comment 6

[Ryan Sleevi]

(In reply to Kai Engert (:kaie) from comment #5)
> (In reply to Ryan Sleevi from comment #2)
> > I really dislike exposing this internal detail of RSA, and shouldn't be part
> > of the public API.
> 
> Since lib-pem is designed to be built as part of NSS (and we still intend to
> merge it eventually), it would be OK to move the type to one the other
> header files that are private.
> 
> Do you have a preference which header file would be good?

Yeah, if that's the layering model you're after, use blapii.h

Although I'm not sure why PEM is implementing OAEP (certainly that's not FIPS-validated), if it could use blapi's internal bits.

Comment 7

[Rob Crittenden]

This comes largely from the softokn library. I needed to do RSA operations, as I recall Bob suggested copying this file while things have to be built outside of NSS. I wouldn't object to letting something else do the operations.

Comment 8

[Ryan Sleevi]

(In reply to Rob Crittenden from comment #7)
> This comes largely from the softokn library. I needed to do RSA operations,
> as I recall Bob suggested copying this file while things have to be built
> outside of NSS. I wouldn't object to letting something else do the
> operations.

I'm still a little confused about the layering - dependent versus parallel.

If you can depend on freebl, use blapii.h & blapi's RSA* functions (which now expose functions for OAEP, PSS, etc)

If you can only depend on softoken, have the RSA operations done via the softoken PKCS#11 interface (which I still need to flip the switch for OAEP, which I'll send a bug/review for today)

Comment 9

[Kai Engert [:KaiE:]]

Thanks to everyone for your comments in this bug, which were helpful as I'm making my first steps with the nss-pem code.

I'm realizing the nss-pem code that uses the enums are copies of functions rsa_FormatOneBlock and rsa_FormatBlock. Simply exposing the enums in a header wouldn't be sufficient.

Currently the nss-pem code calls rsa_FormatBlock (+rsa_FormatOneBlock), then RSA_PrivateKeyOpDoubleChecked.

I've filed a https://fedorahosted.org/nss-pem/ticket/3
to find a better solution, and I'm closing this one as wontfix.