Closed Bug 217305 Opened 21 years ago Closed 8 years ago

PKCS #12 backup fails for certs with ISO-8859-1 nicknames

Categories

(Core :: Security: PSM, defect)

Product:
Core
Component:
Security: PSM
Version:
Other Branch
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
major

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: hc, Unassigned)

References

(Depends on 1 open bug)

Details

(Whiteboard: [kerh-coz][psm-cert-manager])

Attachments

(1 file)

Patch for NSS
2.12 KB, patch
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6
Build Identifier: Mozilla 1.4 + RedHat Linux 8.0

I can install my digital certificate which cantains the following

CN = Hans Christian Støvring Studt

When I try to export my certificate

Preferences -> Certificates -> Manage Certificates ... -> Backup

I then enter
   master password
   certificate backup password (2 times)
   OK
resulting in the following alert

Failed to create the PKCS #12 backup file for unknown reasons.

Reproducible: Always

Steps to Reproduce:
1. Get a certificate with 'ø' in the name
2. Install certificate in Mozilla
3. Export certificate

Actual Results:  
alert
Failed to create the PKCS #12 backup file for unknown reasons.

Expected Results:  
exported signature in a PCKS#12 file.
and why should this be a security hole ?

-> PSM (for crypto)
Assignee: security-bugs → ssaux
Component: Security: General → Client Library
Product: Browser → PSM
QA Contact: carosendahl → bmartin
Version: Trunk → unspecified
*** Bug 208915 has been marked as a duplicate of this bug. ***
Depends on: 197009
Very likely an NSS issue, I htink.
Probably related to bug 53133.
Assignee: ssaux → kaie
*** Bug 210341 has been marked as a duplicate of this bug. ***
WFM MacOs 10.3.2 using both Mozilla 1.6 and a trunk build with trunk NSS.

Fixed for me with 2004020208 (Windows 2000)

But I used that same build to import the sample cert from 197009, and then
export it. It doesn't test if users that had a cert created with the older
builds will be able to export it. 
Also a more complete test with BMP/utf-8 encoded name in addition to
T61/ISO-8859-1 would be better, because it's unclear if the problem also touched
this certificates, in which cas the fix to bug 197009 didn't necessary solve it
fully.
I tested with Mozilla 1.7a Build 20040216 for Linux. Now the certicate name of
my certificate(containing the danish letter 'æ') is shown in the certicate
manager (it was not shown in Mozilla 1.6). But I still get the "Failed to create
the PKCS #12 backup file for unknown reasons"-error when I try to backup the
certificate.
*** Bug 228364 has been marked as a duplicate of this bug. ***
*** Bug 236572 has been marked as a duplicate of this bug. ***
I can't reproduce this, so someone else needs to diagnose where in the code (and
why) the error is being triggered.

Mathias, it's likely you still have the problem, because your certificate was
created before the bug was corrected.

Can you test again with a certificate you created with a corrected version ? 
If you could test with a sample certificate, that has no value, or that you can
revoke, this would be great because then you could attach the pkcs12 with the
bug, and it would make it easier to reproduce.

As I understand it, the bug is that he cannot create the .p12 file from his
cert DB.  He'd have to attach his cert8.db and key3.db files, and his password
for the key3.db file.
...which would let anyone who sees this bug use his certificates and stored
passwords.

I have now revoked my certificate and it should be "safe" to hand it over to you
for debugging. I'm still not happy about publishing it publicly by attaching it
to this bug(it has been the key to my life for the past six months ;-) ), but if
John G. Myers or Jean-Marc Desperrier(I trust you by intuition) would please
send a request directly to me, I would be happy to share.
Sorry, I *really* didn't mean to send a cert that you ever actually used for
anything significant. I meant a cert that you newly created for that and newer
would use for anything else.

You cant't really send your key3.db to anybody, the private keys for *all* your
certificates are inside that file. 

But if there something broken, it might be just inside the cert8.db, so that the
NSS guys can figure out from only this file.
No problem! I have removed all other certificates in the certificate-manager and
changed the master-password. I just need to know *who* would benefit from
getting a copy of my cert8.db/key3.db-files?

I had to revoke my certificate anyway, when I requested a new one to test if my
problems are caused by the fact that the certificate was installed in mozilla
before the bug-fix described in Comment #7 was added.
I will install the new certificate in Mozilla-1.7a when I receive it in a couple
of days, and then we will know if the problem is (partly)fixed.
I would however be nice if it was possible to backup certificates installed
before Mozilla-1.7!

Re comment 14:  
... which is why Jean-Marc asked for "a sample certificate, that has no value, "

Mathias, you can email the files directly to me, if you like.
I will test to see if the problem is in NSS by seeing if I can export the cert
using pk12util, an NSS test program.  I'll report results here.
Re: commenbt 17, the PKCS12 export code won't export without a private key.

Thanks, Mathias, for sending me your DB files.   Here's what I found.
Your cert DB's nickname for your cert is this string:

    Mathias Nicolajsen Kjærgaard's TDC ID

Where the 22nd character is the "ae" character, hex E6, an ISO-Latin-1
character, IINM.  That string is not UTF8-encoded.  It is not a valid
UTF8 string.

In a PKCS12 file, the "Friendly Name" is encoded as UCS2.  So, the PKCS12 
encoder passes this string to the UTF8-to-UCS2 converter, which John recently
rewrote.   The new converter function detects that the input is not valid
UTF8, and fails, causing the PKCS12 export ot fail.  The previous converter 
may have been (incorrectly) converting that string to UCS2, and appearing to
succeed.  

I would not say that John's rewrite caused a regression, but rather that it
exposed a bug that may have been masked by the previous implementation of 
the converter, namely that non-UTF8 strings were being passed to the 
UTF8-to-UCS2 converter.  

The problem is that we have non-UTF8 strings stored in places where we 
believe they are UTF8 strings, and is broader than just pkcs12 exporting.
So, I will file a separate bug against NSS about this, and make that bug
block this one.  Stay tuned.
I also discovered that when windows' fprintf function prints mathias's 
nickname to a "command console" (shell) window, e.g. to stdout, it 
coverts that "ae" character to another character, from 0xE6 to 0xB5.

That means it is not possible to take the output of, say, certutil -L 
and copy-n-paste the displayed nickname into another command line and get
usable results.  The nickname is already corrupted by the time it is 
displayed.  :(  
I have the very same problem with Mozilla 1.7b running on WinXP. But my
Certificate has a German ä (ä) in the Certificate Name.

Mozilla V1.6 did show a blank line in the Certificate Manager, V1.7b does show a
Éa instead of the ä. But I can not backup the Certificate in either version.

I get the very same error message: "Failed to create the PKCS #12 backup file
for unknown reason".

For other certificates without ä it does work so.

A way out of this would be a patch to keyutil.c to allow to dump all keys, and
to be able to dump them inside a pkcs#8 file. 
You then just would need to get a copy of the cert to easily recreate a p12 with
openssl. If the cert is usable within Mozilla, then just extract it from a
signed mail.

The first point should be very easy for anyone interested, but the second point
might be the hard part. And the default display format of keyutil for the
private key is hardly usable as an input form for any tool.
same to me with a certificate with ñ inside (spanish)
This is also a problem with "Æ" 
 
I have just confirmed that the problem still exists in the nightly build: 
ftp://ftp.scarlet.be/pub/mozilla.org/mozilla/nightly/2004-05-10-10-1.7 
*** Bug 243795 has been marked as a duplicate of this bug. ***
*** Bug 268288 has been marked as a duplicate of this bug. ***
Summary: PKCS #12 backup fails with danish names containg the letter 'ø' → PKCS #12 backup fails with danish names containg the letter '�'
*** Bug 262073 has been marked as a duplicate of this bug. ***
Blocks: 243738
Assignee: kaie → nobody
Product: PSM → Core
Why is nobody picking this one up? It is really important to get this fixed as
the use of certificates is increasing rapidly in countries where we have these
character issues.
This only applies to certs issued by CAs (one particular CA) that is
noncompliant with relevant standards.  It is not generally applicable to
internationalized certs, it is just applicable to certs which incorrectly encode
iso8859-1 using a T61String.

This also only applies to certs that were imported before bug 53133 was fixed.
"This also only applies to certs that were imported before bug 53133 was fixed."

Bug 53133 seems to be for the Windows platform, so how are the situation with
regard to Linux which is the platform originally used when reporting this issue.
As I just noted in  bug 53133, that bug fix was for all platforms, 
even though the bug was reported only against windows.
Ok, but how do the people who have this problem resolve it? Do we delete our
certificates and apply for a new one to be issued or do we sit back and wait
till somebody fix the problem in the Mozilla code or ....?
People who have *this* problem should if possible revoke the affected cert, and
ask for a new one that they will be able to save.
It might be best to delete the affected cert just before getting the new one,
just in case.

Now, I see duplicates where the reporter has mozilla 1.7 which should not be
affected, and I wonder if there's not another issue in addition to the one
corrected in this bug. If you can see the problem happening with a cert *issued*
from mozilla 1.7, it's not the same bug, and it needs to be corrected separately.

I plead guilty for one thing. When I reproduced in comment 7 and said the bug
was fixed, I tested by importin/exporting a pkcs#12.
I now realize that maybe the bug is still there when you enroll from Mozilla.

About the people who are affected by this bug, and *need* to save that
certificate somehow, the solution I think would be a patch for the NSS tools to
be able to dump the key database including entries where the nickname is
invalid, and then intructions to recreate a pkcs#12 from that.

Nobody has stepped up to do that, and the core NSS team has other priorities. 
I believe it wouldn't be too difficult to do for a competent programmer, even
without prior NSS experience. You just need to enumerate and dump all keys in
pkcs#8 format, from that I can give instructions about how to recreate the
pkcs#12 using only openssl in command line, so no additional programming required.

It has been asserted above that this problem is fixed, but I'd like to gather 
more info to determine if it really is or not.  So I'm asking the people who 
are having this problem (who are cc'ed on this bug) to add comments telling 
us the following information.  Please be concise.

1. When (on what date) did you "import" your cert (and private key) 
   into your cert DB?
2. What product and version did you use to import it?  
   mozilla 1.3? FF 1.0? TB 0.5? :)
3. How did you get the cert?  did you:
   - import it from a PKCS12 file?
   - visit a CA's web site and get it directly from the CA's web site?
     if so, what CA? (please cite URL)
   - other? (please explain)
4. When will your cert expire? (the one that you're trying to export)

If indeed it is true the current versions of mozilla (seamonkey) and firefox
no longer create/import certs with troublesome nicknames, and that only 
people who imported certs using older browsers have this problem, then it
may be best to try to fix this using pk12util, the command line tool for
exporting certs/privKeys from cert and key DB files.  

As Jean-Marc noted above, that's not going to be a priority for NSS 
devlopers any time soon, but it might be simple enough for someone to 
undertake (much simpler than trying to undertake a chage to the browser,
IMO).
(In reply to comment #35)

> 1. When (on what date) did you "import" your cert (and private key) 
>    into your cert DB?
12-12-2003.

> 2. What product and version did you use to import it?  
>    mozilla 1.3? FF 1.0? TB 0.5? :)
Not sure about this. If I have to choose between the above it would be Mozilla
1.3 but could it have been Mozilla 1.1 or Netscape 7.1?

> 3. How did you get the cert?  did you:
>    - import it from a PKCS12 file?
>    - visit a CA's web site and get it directly from the CA's web site?
>      if so, what CA? (please cite URL)
>    - other? (please explain)
Visited a CA's web site:
https://bestilling.certifikat.tdc.dk/pocesapply/jsp/index.jsp

> 4. When will your cert expire? (the one that you're trying to export)
12-12-2005.
(In reply to comment #35)
> 1. When (on what date) did you "import" your cert (and private key) 
>    into your cert DB?
Apr 6 2005

> 2. What product and version did you use to import it?  
>    mozilla 1.3? FF 1.0? TB 0.5? :)
Both TB 1.0.2 and FF 1.0 on Windows XP Home Ed. SP2

> 3. How did you get the cert?  did you:
>    - import it from a PKCS12 file?
>    - visit a CA's web site and get it directly from the CA's web site?
>      if so, what CA? (please cite URL)
>    - other? (please explain)
Import from PKCS12

> 4. When will your cert expire? (the one that you're trying to export)
Dec 5 2006

My cert has the danish character æ (ae):
-----BEGIN CERTIFICATE-----
MIIFPjCCBCagAwIBAgIEP7tbnjANBgkqhkiG9w0BAQUFADAxMQswCQYDVQQGEwJE
SzEMMAoGA1UEChMDVERDMRQwEgYDVQQDEwtUREMgT0NFUyBDQTAeFw0wMzEyMDUx
NDU4NDFaFw0wNTEyMDUxNTI4NDFaMHoxCzAJBgNVBAYTAkRLMSkwJwYDVQQKEyBJ
bmdlbiBvcmdhbmlzYXRvcmlzayB0aWxrbnl0bmluZzFAMBkGA1UEAxQSUGV0ZXIg
TGluZCBEYW1rauZyMCMGA1UEBRMcUElEOjk4MDItMjAwMi0yLTU4NDcwNzcyNjA1
MDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAsYq/tpmClXMqbqcPnuaTeTLq
w40lqEuOsmmtim9PG226NbDLoHS17XIM4N0y81RiFp6wyO8DO5LeB5yPJa8QKas4
9mYd5kvkz0bDTXMwyY584wrVD6sTYjViKJSDw0t2PWjXu6vKp3IGYuqG7qi6tLRp
IJ/QBWw9YY3leKnAXjsCAwEAAaOCApcwggKTMA4GA1UdDwEB/wQEAwID+DArBgNV
HRAEJDAigA8yMDAzMTIwNTE0NTg0MVqBDzIwMDUxMjA1MTUyODQxWjCCATcGA1Ud
IASCAS4wggEqMIIBJgYKKoFQgSkBAQEBATCCARYwLwYIKwYBBQUHAgEWI2h0dHA6
Ly93d3cuY2VydGlmaWthdC5kay9yZXBvc2l0b3J5MIHiBggrBgEFBQcCAjCB1TAK
FgNUREMwAwIBARqBxkZvciBhbnZlbmRlbHNlIGFmIGNlcnRpZmlrYXRldCBn5mxk
ZXIgT0NFUyB2aWxr5XIsIENQUyBvZyBPQ0VTIENQLCBkZXIga2FuIGhlbnRlcyBm
cmEgd3d3LmNlcnRpZmlrYXQuZGsvcmVwb3NpdG9yeS4gQmVt5nJrLCBhdCBUREMg
ZWZ0ZXIgdmlsa+VyZW5lIGhhciBldCBiZWdy5m5zZXQgYW5zdmFyIGlmdC4gcHJv
ZmVzc2lvbmVsbGUgcGFydGVyLjAhBgNVHREEGjAYgRZwZXRlckBsaW5kLWRhbWtq
YWVyLmRrMIGPBgNVHR8EgYcwgYQwSaBHoEWkQzBBMQswCQYDVQQGEwJESzEMMAoG
A1UEChMDVERDMRQwEgYDVQQDEwtUREMgT0NFUyBDQTEOMAwGA1UEAxMFQ1JMODcw
N6A1oDOGMWh0dHA6Ly9jcmwub2Nlcy5jZXJ0aWZpa2F0LmRrL29jZXMvMTA2OTI0
MzI5NC5jcmwwHwYDVR0jBBgwFoAUYLWF7FZkfhIZJ2cdUBVLc647+RIwHQYDVR0O
BBYEFG2ImN52chIlyXZ1qeR3g5Dghez0MAkGA1UdEwQCMAAwGQYJKoZIhvZ9B0EA
BAwwChsEVjYuMAMCA6gwDQYJKoZIhvcNAQEFBQADggEBAIhzCB26r/pQLcAiAlDS
loQ0SwPfwGUXbBZ3DcepSyOTkWUbThv1Bx2OAehKAYJ1YNe76GnTZVht/aD18LNE
EA9tpm2UPNcjImzBWzWhmeYeuCYOGBtoafpyuFkqrF19vxIs2vna5EgajJ+zd+cj
c7xdf5lELr+K/VqIHK0t8szVrokbrhBHNWd+tspCrtJS+oRq/FdMpTYjVImhLmBd
5FuHQD5SUaUPhLAowU205yYKaKHk2CKfgey0LciZMEL5itwoAGwgIuHfC/79gbey
t5gP29UCqcS03B7lJwD4BlvfpQLdudSfhX03Xbo22kCxg1ainObVy78EUoTPym35
+w0=
-----END CERTIFICATE-----

(In reply to comment #37)
Argghhhh, forgot to mention that everything worked out fine for me!

/Peter
(In reply to comment #36)
My certificate has ø (oe) in it.
Nelson's list of question is a bit extensive. To sort out the most interesting
cases :
- when first receiving the cert, if you used a mozilla version inferior to 1.7
fional, you are expected to meet the problem, and it's not so interesting to
report it
- if you first received the cert using a later version, but by importing it from
a pkcs#12, it is expected to work (confirmed in comment #37). If it did not,
that's interesting.
- if you first received the cert using a later version by an on-line process
from a CA's web site, no pkcs#12 involved, your feedback is really useful both
if it worked OK or if it did not.

(In reply to comment #40)
> a pkcs#12, it is expected to work (confirmed in comment #37). If it did not,
> that's interesting.
> - if you first received the cert using a later version by an on-line process
> from a CA's web site, no pkcs#12 involved, your feedback is really useful both
> if it worked OK or if it did not.

This is the case that apply to my situation and it does not work now, nor
before. I use Firefox 1.0.1.


Per,  After re-reading your comments above, I think it is the case that you
originally got a certificate with a version of mozilla older than 1.7, and
have since renewed the certificate with a version 1.7 or higher.  
Is that correct?

I think you got the bad nickname string with the older software, and now 
that nickname string is being preserved as you renew the cert.  
Is that correct?

If so, please email me a copy of your cert8.db file, the one that contains 
your cert with the special character that you cannot export.  Also, if you 
still have your cert7.db file (used by the older browsers), please send me 
a copy of that also.  Please do NOT send me a copy of your key3.db file, 
because I don't want your private key.  Please email it to my email address
shown in this bug report.  I will investigate devising a method to correct
the nickname (converting to UTF8, or changing to plain ASCII).

Emailing a copy of your cert8.db file is a little tricky.  You must exit 
your browser and/or email program (mozilla/FF/TB), and then make a copy of
cert8.db by another name or in another directory.  Then restart the email
program and email the copy you made, not the original.  If you have cert7.db
please do likewise for that file.  Thanks.
Component: Security: UI → Security: PSM
No longer blocks: 243738
Whiteboard: [kerh-coz]
I could manage to backup my cert (it hasn't got any non-ascii character but the name of the CA has) by deleting the certificate of the CA, backing up the cert, and re-importing the CA cert.
When using the latest Mozilla client software and a CA that can issue certificates with international characters in fields like the common name, e.g. https://digitalid.verisign.com/client/class1Netscape.htm
everything seems to be working fine. I was able to request a certificate for common name "Hänschen Müller", retrieve the certificate and export to a p12 file correctly.
 
I am now able to reproduce the bug.

I used the old Mozilla version 1.4.1 with a fresh certificate database and obtained a certificate for common Name "Hänschen Müller" from https://digitalid.verisign.com/client/class1Netscape.htm

When using this certificate database with latest Mozilla software, an attempt to export the certificate to a p12 backup file produces the reported error message.

Nelson, I will send you all *.db files by private mail.

#0  sec_port_ucs2_utf8_conversion_function (toUnicode=1, inBuf=0x14051c0 "H�schen 4 Mller's VeriSign, Inc. ID", inBufLen=37,
    outBuf=0x15fb508 "", maxOutBufLen=113, outBufLen=0x7ffffff333f0) at utf8.c:319
#1  0x00002aaaaedded81 in PORT_UCS2_UTF8Conversion (toUnicode=1, inBuf=0x14051c0 "H�schen 4 Mller's VeriSign, Inc. ID",
    inBufLen=37, outBuf=0x15fb508 "", maxOutBufLen=113, outBufLen=0x7ffffff333f0) at secport.c:539
#2  0x00002aaaaeb02405 in sec_pkcs12_convert_item_to_unicode (arena=0x15f50c0, dest=0x7ffffff333e0, src=0x7ffffff334b0,
    zeroTerm=0, asciiConvert=0, toUnicode=1) at p12local.c:939
#3  0x00002aaaaeb052cc in sec_PKCS12AddAttributeToBag (p12ctxt=0x161fd50, safeBag=0x15fb440,
    attrType=SEC_OID_PKCS9_FRIENDLY_NAME, attrData=0x7ffffff334b0) at p12e.c:950
#4  0x00002aaaaeb0585a in SEC_PKCS12AddCert (p12ctxt=0x161fd50, safe=0x161feb8, nestedDest=0x0, cert=0x1337540, certDb=0x11529f0,
    keyId=0x15f6fb8, includeCertChain=1) at p12e.c:1146
#5  0x00002aaaaeb06144 in SEC_PKCS12AddCertAndKey (p12ctxt=0x161fd50, certSafe=0x161feb8, certNestedDest=0x0, cert=0x1337540,
    certDb=0x11529f0, keySafe=0x161fe18, keyNestedDest=0x0, shroudKey=1, pwitem=0x7ffffff336d0,
    algorithm=SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC) at p12e.c:1549
#6  0x00002aaaae97a932 in nsPKCS12Blob::ExportToFile (this=0x7ffffff337a0, file=0x95da10, certs=0x15b71c0, numCerts=1)
    at /home/kaie/moz/head/mozilla/security/manager/ssl/src/nsPKCS12Blob.cpp:449



(gdb) print inBuf
$25 = (unsigned char *) 0x14051c0 "H�schen 4 Mller's VeriSign, Inc. ID"
(gdb) x/40x inBuf
0x14051c0:      0x48    0xe4    0x6e    0x73    0x63    0x68    0x65    0x6e
0x14051c8:      0x20    0x34    0x20    0x4d    0xfc    0x6c    0x6c    0x65
0x14051d0:      0x72    0x27    0x73    0x20    0x56    0x65    0x72    0x69
0x14051d8:      0x53    0x69    0x67    0x6e    0x2c    0x20    0x49    0x6e
0x14051e0:      0x63    0x2e    0x20    0x49    0x44    0x00    0x00    0x00
(gdb) x/40c inBuf
0x14051c0:      72 'H'  -28 '�  110 'n' 115 's' 99 'c'  104 'h' 101 'e' 110 'n'
0x14051c8:      32 ' '  52 '4'  32 ' '  77 'M'  -4 ''   108 'l' 108 'l' 101 'e'
0x14051d0:      114 'r' 39 '\'' 115 's' 32 ' '  86 'V'  101 'e' 114 'r' 105 'i'
0x14051d8:      83 'S'  105 'i' 103 'g' 110 'n' 44 ','  32 ' '  73 'I'  110 'n'
0x14051e0:      99 'c'  46 '.'  32 ' '  73 'I'  68 'D'  0 '\0'  0 '\0'  0 '\0'


The string contains non UTF-8 chars at positions 2 and 13. The routine detects an invalid UTF-8 byte at position 13 and returns with an error.


My quick look at the two sets of DBs show me that one of them has the nickname
encoded with UTF-8, and the other with some other encoding.\

Here's the non-UTF-8 version

H   S   n   s   c   h   e   n       4       M   n   l   l   e
48 E4   6E 73   63 68   65 6E   20 34   20 4D   FC 6C    656C
r   '   s       V   e   r   i   S   i   g   n   ,       I   n
72 27   73 20   56 65   72 69   53 69   67 6E   2C 20    6E49
c   .       I   D
63 2E   20 49   44 20   20 20   20 20   20 20   20 20    2020

Here's the UTF-8 version.

H   +   ñ   n   s   c   h   e
48 C3   A4 6E   73 63    6568
n       5       M   +   +   l   l   e   r   '   s       V   e
6E 20   35 20   4D C3   BC 6C   6C 65   72 27   73 20    6556
r   i   S   i   g   n   ,       I   n   c   .       I   D
72 69   53 69   67 6E   2C 20   49 6E   63 2E   20 49    2044

So I guess the remaining question is: is there any way that we can export
a cert from a cert DB that has non-UTF8 characters in nicknames?

Seems like the answer might be: if the UTF8->UCS2 conversion fails, then
try again with an ISO-Latin-1->UCS2 conversion function.  The resultant
nickname might not be perfect (charaacter miscoding) but the result 
should be quite exportable and importable.  

Does this help?
Magic string for fixing the output of "od -cx" on little endian machines:

:.,$s/ \([0-9A-F][0-9A-F]\)\([0-9A-F][0-9A-F]\) /\2 \1 /g
> if the UTF8->UCS2 conversion fails, then try again 

I agree, if first conversion attempt fails, we should try a fallback strategy.


> with an ISO-Latin-1->UCS2 conversion function.  The resultant
> nickname might not be perfect (charaacter miscoding) but the result 
> should be quite exportable and importable.  

I think we should not fall back to Iso Latin 1.
We don't know whether Latin 1, 2, or some other encoding has been used.

My proposal is, convert all chars >= 128 to some generic replacement char like a dot. That way we won't do any incorrect conversion to unexpected characters.


Does it make sense to implement the fallback mechanism within NSS?

What would happen when you subsequently restore?  Would you get the cert entered twice, once with each encoding?

The question in comment 42 still hasn't been answered.  What happens when the cert with the badly encoded nickname is renewed? 

This seems a lot of effort to address the fallout of a bug that was fixed two years ago.

In response to comment 49: we know, as described in bug 237077, that the encoding is either UTF-8 or iso-8859-1.
The nickname is associated with the DER Subject. Any cert with the same DER Subject will have the same nickname. When you import a new certificate for which one already exists with that Subject, the imported nickname is ignored and the old nickname is used. A single cert will never have more than one nickname.

bob
> This seems a lot of effort to address the fallout of a bug that was fixed two
> years ago.

But there might be an unknown number of end users who now own such a broken cert in their database. And the cert might be important for them. And in that case they are forced to stick with their original cert database.

In my opinion, if the fix is simple, and if having the same cert twice with different nicknames doesn't hurt, we should go for it.

I believe the proper fix would be for NSS to repair the corrupted databases.  On startup, NSS should scan the database for nicknames that aren't syntactically valid UTF-8.  Such nicknames should be converted from ISO-8859-1 to UTF-8 and the UTF-8 form stored back in the database.

Kai,  In answer to your question in comment 49:

> Does it make sense to implement the fallback mechanism within NSS?

Your stack trace from comment 46 reduces to this:

#0  sec_port_ucs2_utf8_conversion_function ()
#1  PORT_UCS2_UTF8Conversion ()
#2  sec_pkcs12_convert_item_to_unicode ()
#3  sec_PKCS12AddAttributeToBag ()
#4  SEC_PKCS12AddCert ()
#5  SEC_PKCS12AddCertAndKey ()
#6  nsPKCS12Blob::ExportToFile ()

NSS allows the calling application to replace the call to sec_port_ucs2_utf8_conversion_function (in function PORT_UCS2_UTF8Conversion) with a call to a function of the application's own choosing.  To register an alternative conversion function, the application calls PORT_SetUCS2_UTF8ConversionFunction().  Thereafter, PORT_UCS2_UTF8Conversion will call the function supplied by the application.

If mozilla wants NSS's PKCS12 export code to use a conversion function that tries to do UTF8->UCS2 conversion, and failing that, does ISO-8859-1->UCS2 
conversion, then mozilla should register such a function using that technique.  

Having said that, I think it might be reasonable to ask NSS to provide a
function that does that first-UTF8-then-ISO8859 approach, so that mozilla
merely needs to tell NSS to use that function.  


In reply to John's comment 53 :
I'm not sure we want NSS to take a full pass on the cert DB every time it 
is opened, but it does seem reasonable to have a way to repair such a DB.

NSS has a stand alone database diagnosis and repair program, named dbck.  
(Right now the diagnosis part works, but the repair part is in disrepair. :)
Adding recovery of nicknames to that program should be straightforward.
(Perhaps it already does something like that.  IIRC, it constructs new 
nicknames for certs that lack them.)
(In reply to comment #42)
This is what I wrote to Nelson Bolyard on 06-04-2005:

Hi Nelson.

First of all thank you very much for dealing with this problem.

About you two questions:
> Per,  After re-reading your comments above, I think it is the case that you
> originally got a certificate with a version of mozilla older than 1.7, and
> have since renewed the certificate with a version 1.7 or higher. 
> Is that correct?
I must admit I can't remember if I have renewed the certificate, but yes, that could very well be the case.

> I think you got the bad nickname string with the older software, and now
> that nickname string is being preserved as you renew the cert. 
> Is that correct?
The nickname looks fine in Firefox, but that is not what you mean, right? It is very difficult for me to figure out where the problem is. My understanding of the problem is merely at the level where I can see that the backup process fails.

I am afraid I am not of much help.

The cert files I have attached, both the cert8 file and several cert7 files lying around on my system, since I am not sure if any of them contains a certificate.

Hope you can solve the problem.
---------------------------------------------------------------------------
And this i what he answered, with my reply:

> Content-type: text/plain; charset=ISO-8859-1; format=flowed
> X-MIME-Autoconverted: from 8bit to quoted-printable by brmea-mail-4.sun.com id j3H4Jti9025978
>
> Per Mørkegaard Hansen wrote:
>> Hi Nelson.
>> First of all thank you very much for dealing with this problem.
>
> You're welcome.  I have something for you to try, and some more questions.
>
>> I must admit I can't remember if I have renewed the certificate, but yes, that could very well be the case.
>
> It appears that you obtained a brand new certificate and sent it to me
> for this test.  I hope that you did not have to pay much money just for
> this test!  I need to know if this cert8.db file that you sent me is
> one that was created new for this test, or if it is an old one that
> you've had for a long time through many months of mozilla testing.
> The cert8.db file doesn't seem to contain many certificates.  So,
> please answer these few questions for me.
>
> Did you create a new cert8.db file (perhaps in a new mozilla profile) just
> to send to me?

No, it was the one I have used all the time.


> Did you delete other certs from your cert8.db file before sending it to me?

No.


> Do you normally have this few certificates in your cert DB?

Yes. I only use certificates for one thing, namely accessing government sites here in Denmark and there I always use the same certificate, that is one of the clever things about it, I guess.


>> I am afraid I am not of much help.
>
> You are helping!  I appreciate your help.
>
> And The answers to my questions above will also help.
>
> Using the cert8.db file you sent me, I extracted your cert from it,
> and created a new cert8.db file with that cert in it.  I then put the
> new cert8.db file into a zip file, and attached the zip file to this
> email.  I would like you to do some tests with it.  I will explain how.
>
> The cert DB contains a copy of your certificatet, and it also contains
> a copy of a "nickname" for your certificate.  The nickname is only used
> in mozilla's certificate selection dialogs.  It is never transmitted
> to anyone.
>
> I am testing the hypotheis that the problem is not caused by European
> characters in the certificate itself, but only in the nickname.  If we
> can confirm this nypothesis, then I will have a good idea how to proceed
> to fix this problem.  You can help me confirm or disprove this hypothesis
> by testing the cert8.db file that is attached.
>
> In the cert8.db file you sent me, the nickname of your cert contained
> two "special" characters in the middle name.
>
> In the new cert8.db file that I am sending you, the certificate is
> exactly the same as before, but the nickname has no European characters
> and has been changed to Per Morkegard Hanseen's TDC ID .
> Please forgive my misspelling.  This should only be temporary.
>
> To conduct this test, please do the following steps:
> 1. save the attached zip file to your disk.
> 2. exit your mozilla browser/email program.
>    (Perhaps you should print out a copy of these instructions first :)
> 3. make a backup copy of the old cert8.db file so that you do not
>    lose it.
> 4. unzip the cert8.db file in the attached ZIP file, and replace the
>    old cert8.db file with the new one.
> 5. Restart your mozilla program.
> 6. Go into the certificate Manager window, and try again to "backup"
>    your certificate into a PKCS12 file.
>
> Please let me know if your test succeeds or fails.

I did as you explained and what happened was that there was no longer a certificate in my certificate list, but it had moved to the list of "Other people's" certificates, form where I can't create back up's. I used Firefox 1.0.0 for this.
QA Contact: bmartin → psm
Attached patch Patch for NSS — — Splinter Review 
(In reply to comment #54)
> Having said that, I think it might be reasonable to ask NSS to provide a
> function that does that first-UTF8-then-ISO8859 approach, so that mozilla
> merely needs to tell NSS to use that function.  

Like this?
> NSS allows the calling application to replace the call to
> sec_port_ucs2_utf8_conversion_function (in function PORT_UCS2_UTF8Conversion)
> with a call to a function of the application's own choosing.  To register an
> alternative conversion function, the application calls
> PORT_SetUCS2_UTF8ConversionFunction().  Thereafter, PORT_UCS2_UTF8Conversion
> will call the function supplied by the application.
> 
> If mozilla wants NSS's PKCS12 export code to use a conversion function that
> tries to do UTF8->UCS2 conversion, and failing that, does ISO-8859-1->UCS2 
> conversion, then mozilla should register such a function using that technique. 
> 
> Having said that, I think it might be reasonable to ask NSS to provide a
> function that does that first-UTF8-then-ISO8859 approach, so that mozilla
> merely needs to tell NSS to use that function.  


Nelson, we should reuse the existing functions that NSS provides, and only use a wrapper function around them, as I've proposed with the attached patch.

Currently functions sec_port_ucs2_utf8_conversion_function
and sec_port_iso88591_utf8_conversion_function are not being exported.

But a replacement function at the PSM level registered with a call to PORT_SetUCS2_UTF8ConversionFunction, would require that they are exported.

But even if we add the proposed function to the NSS sources,
we must at least export the new 
sec_port_ucs2_utf8_fallback_iso88591_conversion_function.

Which of the following do you prefer?

a)
export both 
  sec_port_ucs2_utf8_conversion_function
  sec_port_iso88591_utf8_conversion_function
and have PSM implement the wrapper function.

b)
add
  sec_port_ucs2_utf8_fallback_iso88591_conversion_function
to NSS and export
all three functions
  sec_port_ucs2_utf8_conversion_function
  sec_port_iso88591_utf8_conversion_function
  sec_port_ucs2_utf8_fallback_iso88591_conversion_function

c)
add
  sec_port_ucs2_utf8_fallback_iso88591_conversion_function
to NSS and export only the new function.


d)
Introduce a new function and two enum values like this:

enum builtin_ucs2_conversion_functions {
  builtin_ucs2_utf8_conversion_function,
  builtin_ucs2_utf8_fallback_iso88591_conversion_function
}

SECStatus PORT_SetBuiltinUCS2_UTF8ConversionFunction(
                  builtin_ucs2_conversion_functions bucf);

I just discovered that functions 
sec_port_ucs2_utf8_conversion_function
sec_port_ucs4_utf8_conversion_function
ARE exported, even though they do not appear in nss.def, because their
definitions are surrounded by PR_IMPLEMENT() which evidently forces the 
symbols to be exported.  

RATS!  I wonder how many other functions that we have been thinking were 
private are actually public.  8-( 

Your proposals a or b are both OK with me.  But let's find a shorter
name than sec_port_ucs2_utf8_fallback_iso88591_conversion_function.
(In reply to comment #58)
>
> Your proposals a or b are both OK with me.  But let's find a shorter
> name than sec_port_ucs2_utf8_fallback_iso88591_conversion_function.

My preference is b)

Where can we save chars?
Do you want to keep the _conversion_function suffix, or can we shorten it?
Should we replace _fallback_ with _or_?

Proposals:
- sec_port_ucs2_utf8_or_iso88591_conv_func
- sec_port_ucs2_utf8_or_iso88591_conversion_function

Pick one, or feel free to come up with another proposal, and I'll write a new patch.


Arg... Well PR_IMPLEMENT means the would be exported on some platforms, they aren't universally exported, so if we do a) we would still have to add them nss.def.

Is this still an issue? I don't have Danish letters in my name, so I can't test.
The problems caused by NSS's use of PR_IMPLEMENT were fixed in NSS 3.12.2
with the patch for Bug 450845.  None of the various 
sec_port_*_conversion_function functions are exported on Windows any more.

hansen,
I think the problem reported in this bug report is still unfixed, but I'm 
not sure it is an issue any more.  

Here is a summary.  Back in the days of the old Mozilla 1.4 browser (and
before) there was one (or more) bugs that caused certs to be stored in 
the cert DB with "friendly names" (a.k.a. "nicknames") that contained 
characters encoded in ISO-8859-1 rather than in UTF8 (as they should have
been).  This was not a problem with the certificate itself, but with the 
"nickname" that is stored along with the cert in the DB, and in pkcs12 files.
This caused several problems, one of which was that a cert in the cert DB
with an incorrectly encoded "nickname" could not be "exported" or "backed up"
into a PKCS#12 file.  That problem is the subject of this bug.

The original bug which allowed incorrectly encoded nicknames into the cert DB was fixed in Mozilla 1.7, so that new databases created after that time did 
not have this problem because we were being more careful to only put UTF8 characters into nicknames in cert DBs.  So users whose cert DBs were created
after that date had no problems backing up their certs into PKCS12 files 
caused by incorrectly encoded nicknames.  But users whose cert DBs were old,
having been created by Mozilla browsers older than mozilla 1.7 still could
not export the certs from those old DBs.  That was the state when this bug
was filed.

AFAIK, that is still the case today.  Even today, users with cert DBs with
incorrectly encoded nicknames still cannot export the affected certs into 
PKCS12 files.  However, enough time has passed that very few, if any, of 
the DBs with those old certs are still in use (probably), and/or the certs 
have expired long ago.  So, perhaps this issue is no longer worthy of being
fixed.  It's been a long time since anyone complained about this, I believe.

There was/is a workaround that is somewhat tedious and may be beyond most 
user's capabilities.  There is a command line tool named "certutil" that 
performs operations on cert DBs.  It is possible with that tool to export
the affected certs into PEM files (not PKCS12 files).  Unlike PKCS12 files,
PEM files have no nicknames.  By carefully exporting all the affected certs
into PEM files, then deleting them all from the cert DB, then re-importing
them with correctly encoded nicknames (also using certutil), it is possible
to effectively "fix" the encoding of those certs, which in turn makes it 
possible to export the certs and their private keys into PKCS12 files.
I rather doubt that anyone other than NSS developers have ever accomplished
that, and perhaps even no NSS developers have done so.  

Today, I'm not certain that we could readily create a set of cert and key 
DBs that have this problem, with which to reproduce this bug and test a fix.  
I'm not sure it would be worth the effort to try to do so.

Maybe we should try to work Kai's proposed fix into the NSS function 
PORT_UCS2_UTF8Conversion.  Or Maybe we should resolve this bug WONTFIX.
Summary: PKCS #12 backup fails with danish names containg the letter '�' → PKCS #12 backup fails for certs with ISO-8859-1 nicknames
I tried to import an S/MIME certificate from both StartCom and InstantSSL for secure email and it comes up with "The PKCS #12 operation failed for unknown reasons."

This is on Thunderbird 3.1.7
Daniel, your problem is not the one described in this bug report.
You have PKCS12 files that have been incorrectly encoded with "Friendly Names" 
that are encoded in some character set other than Unicode (e.g. UTF8).
Firefox will not import them.  You need to re-make them with proper UTF8 
Friendly Names.
Whiteboard: [kerh-coz] → [kerh-coz][psm-cert-manager]
I'm not able to backup my startssl.com login cert, may be because of this bug. If so, this should be fixed very soon.
If I'm reading the history of this bug correctly, the failure occurs for certificates that were imported with a very old version of Firefox but won't happen for certificates imported in more recent versions. If that's the case, I don't think there's anything further to do here (since those certificates should have long since been replaced). For other failures to backup certificates, please file a new bug in this component.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: