rijndael_encryptECB, rijndael_encryptCBC, rijndael_decryptECB, and rijndael_decryptCBC do not set the *outputLen output argument. This violates the NSS convention. These functions are relying on their callers to set *outputLen on entry. Right now the only callers are AES_Encrypt and AES_Decrypt. They set *outputLen = inputLen before calling these functions.
Created attachment 8414292 [details] [diff] [review] bug791875_v1.patch
Comment on attachment 8414292 [details] [diff] [review] bug791875_v1.patch Cancelling review. After revisiting this, I'm no longer comfortable with making changes to what looks to me like rather important sensitive code.