190243 - cert7.db to cert8.db conversion loses personal certificates

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P1)

Description

[Kai Engert (:KaiE:)]

Please see bug 190241 for an effect of this bug.
Nelson helped to analzye the potential cause for this problem.

I have a cert7.db that contains personal certificates that do not have the "u"
flag set. However, NSS still treats that certificates as personal certificates.

When using a cert8.db, after the automatic conversion, those certificates are no
longer treated as personal certificates.

I think this is a regression.
One effect, as described in bug 190241, is, after upgrading to a newer version
of the Mozilla client, the user is no longer able to decrypt archived S/Mime
messages.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Adding some more info.  (Kai, please correct me if any of this is wrong.)

Kai's cert7.db file has numerous "user" certs (certs for which the private
keys are in the key DB) that have somehow lost the "u" trust flags.

Despite the absent u flags, versions of mozilla older than NSS 3.7 still treat 
these as user certs.  For example, using an older mozilla+NSS, Kai can decrypt 
and read email whose encryption cert is missing the u flag.  

Using a newer mozilla with NSS 3.7, that cert7 is converted to cert8.  The 
newer mozilla can NOT decrypt the same email message that the older one can.  

Also, in the newer mozilla, the encryption cert appears in the list of other
users' certs, not in the list of Kai's own certs.

We conducted an experiment.  Using a 1.5 year old certutil, we set the 
missing u flag on this cert in cert7.  Then the new mozilla converted this 
modified cert7 to a cert8.  The cert then appeaered among Kai's own certs in 
this cert8, and Kai was able to decrypt the email.  

This suggests to me tht the cert DB upgrade procedure should check for 
private keys even when the u flag is absent.  

Comment 2

[Wan-Teh Chang]

Bob, could you take a look at this?  Thanks.

Comment 3

[Kai Engert (:KaiE:)]

> Despite the absent u flags, versions of mozilla older than NSS 3.7 still treat 
> these as user certs.  For example, using an older mozilla+NSS, Kai can decrypt 
> and read email whose encryption cert is missing the u flag.  
> Using a newer mozilla with NSS 3.7, that cert7 is converted to cert8.  The 
> newer mozilla can NOT decrypt the same email message that the older one can.  
> Also, in the newer mozilla, the encryption cert appears in the list of other
> users' certs, not in the list of Kai's own certs.

correct


> We conducted an experiment.  Using a 1.5 year old certutil, we set the 
> missing u flag on this cert in cert7.  Then the new mozilla converted this 
> modified cert7 to a cert8.  The cert then appeaered among Kai's own certs in 
> this cert8, and Kai was able to decrypt the email.  

Not fully correct. Out of the list of 3 affected certs, certutil was able to set
the flag on one of the certs only. Certutil didn't modify all certs having the
same nickname, but only one. It was luck that it was able to modify one of the
affected certs, but for the two other certs, I was unable to add the 'u' flag
using certutil. Apparently the one cert I could "fix" was not the one necessary
to decrypt the archived test message. However, it appears to me it would work.


> This suggests to me tht the cert DB upgrade procedure should check for 
> private keys even when the u flag is absent.

This sounds reasonable to me. I have a backup of this cert db. If you could send
me a patch that disables this optimization, I could try it out.

Comment 4

[Robert Relyea]

The update procedure is indeed supposed to do exactly what Nelson suggested.
There is code in there to do this. Evidently it is not working correctly.

Comment 5

[Robert Relyea]

Attachment: Write out changes that we make to the trust when we verify the database.

OK, so the problem was we weren't writing the changes back to the database when
updated the trust for the cert (in fact we updated the trust for the cert in a
cert object which we immediately threw away afterwards).

I've tested this patch against the test cases described and wound out with
databases that have the user bits properly correct.

Comment 6

[Wan-Teh Chang]

Comment on attachment 112421 [details] [diff] [review]
Write out changes that we make to the trust when we verify the database.

r=wtc.	Nelson, could you review this patch, too?

Kai, could you test this patch on your cert7.db?

Comment 7

[Kai Engert (:KaiE:)]

This patch fixes the problem for me, thanks!

Comment 8

[Kai Engert (:KaiE:)]

I'm nominating this for inclusion in Mozilla 1.3 beta, to make sure this
regression will not hit our beta testers.

Comment 9

[Robert Relyea]

Patch is checked into the tip and the branch.

Comment 10

[Wan-Teh Chang]

Target 3.7.1.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

Comment on attachment 112421 [details] [diff] [review]
Write out changes that we make to the trust when we verify the database.

Looks good to me.  
I'm SO glad we can fix this before the mozilla release!
But we also need a solution for users who've already gone through the upgrade
process.  
I guess the only solution is to tell the users to export any new user certs and
then move aside cert8.db and redo the conversion, then import the certs again. 
Icky, but requires no new code, I think.

Comment 12

[Asa Dotzler [:asa]]

So this is fixed in the branch that mozilla uses and so will be fixed for 1.3beta?

Comment 13

[Asa Dotzler [:asa]]

a=asa (on behalf of drivers) for making this work in Mozilla 1.3b.

Comment 14

[Kai Engert (:KaiE:)]

Can we move NSS_CLIENT_TAG to a version that includes this fix?

Comment 15

[Wan-Teh Chang]

The fix is in NSS_3_7_1_BETA3 and NSS_CLIENT_TAG
(Mozilla 1.3b) now.  Marked the bug fixed.