1415912 - Document a recommended mechanism to perform explicit NSS database migration from dbm to sql using existing NSS tools

Status: RESOLVED FIXED

(NSS :: Tools, enhancement)

Description

[Kai Engert (:KaiE:)]

cheimes asked me about an good approach to perform an explicit migration of an existing dbm database to an sql database. He suggested to use "certutil -N".

I suggest to use this bug to investigate if "certutil -N" is an appropriate mechanism, or if something else should be used. When we have that, we should document it somewhere in our docs, for example in the certutil help or manual page.

Comment 1

[Kai Engert (:KaiE:)]

I see that using certutil -N will require multiple prompts for passwords, even if the databases uses the empty string as the password. It asks for the password of the old file, then for the new database. A good migration mechanism should require the password at most once.

Comment 2

[Christian Heimes]

I assume that

"certutil -N -d sql:/path/to/old/nssdb -f /path/to/old/nssdb/pwdfile.txt -@ /path/to/old/nssdb/pwdfile.txt" does not prompt for a password. AFAIK -f is the password file for the new DB in SQL format, -@ refers to the password of the old, existing DB in DBM format.

$ mkdir /tmp/nssdb
$ cd /tmp/nssdb
$ pwgen 16 1 > pwdfile.txt
$ certutil -N -d dbm:/tmp/nssdb -f /tmp/nssdb/pwdfile.txt 
$ ls
cert8.db  key3.db  pwdfile.txt  secmod.db


$ certutil -N -d sql:/tmp/nssdb -f /tmp/nssdb/pwdfile.txt -@ /tmp/nssdb/pwdfile.txt 
Password changed successfully.
$ ls
cert8.db  cert9.db  key3.db  key4.db  pkcs11.txt  pwdfile.txt  secmod.db
$ stat cert9.db 
  File: cert9.db
  Size: 28672           Blocks: 56         IO Block: 4096   regular file
Device: 2bh/43d Inode: 5422833     Links: 1
Access: (0600/-rw-------)  Uid: ( 1000/  heimes)   Gid: ( 1000/  heimes)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2017-11-09 17:34:31.854114697 +0100
Modify: 2017-11-09 17:34:25.431195758 +0100
Change: 2017-11-09 17:34:25.431195758 +0100
 Birth: -
$ stat key4.db 
  File: key4.db
  Size: 36864           Blocks: 72         IO Block: 4096   regular file
Device: 2bh/43d Inode: 5422835     Links: 1
Access: (0600/-rw-------)  Uid: ( 1000/  heimes)   Gid: ( 1000/  heimes)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2017-11-09 17:35:56.314052950 +0100
Modify: 2017-11-09 17:35:56.314052950 +0100
Change: 2017-11-09 17:35:56.314052950 +0100
 Birth: -


$ certutil -N -d sql:/tmp/nssdb -f /tmp/nssdb/pwdfile.txt -@ /tmp/nssdb/pwdfile.txt
$ stat cert9.db 
  File: cert9.db
  Size: 28672           Blocks: 56         IO Block: 4096   regular file
Device: 2bh/43d Inode: 5422833     Links: 1
Access: (0600/-rw-------)  Uid: ( 1000/  heimes)   Gid: ( 1000/  heimes)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2017-11-09 17:35:56.314052950 +0100
Modify: 2017-11-09 17:34:25.431195758 +0100
Change: 2017-11-09 17:34:25.431195758 +0100
 Birth: -
$ stat key4.db 
  File: key4.db
  Size: 36864           Blocks: 72         IO Block: 4096   regular file
Device: 2bh/43d Inode: 5422835     Links: 1
Access: (0600/-rw-------)  Uid: ( 1000/  heimes)   Gid: ( 1000/  heimes)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2017-11-09 17:35:56.314052950 +0100
Modify: 2017-11-09 17:35:56.314052950 +0100
Change: 2017-11-09 17:35:56.314052950 +0100
 Birth: -


A second certutil -N does not change the inode, so it does not create a new file. It might merge keys from old to new DB, though.

Comment 3

[Robert Relyea]

certutil -N will only upgrade a database. Once the new database is created, there is not attempt to 'sync' with the old database.

The second certutil -N will act the same as certutil -W (changing the password). Even if you change the password to the same name password, a new password entry will be created with new salt, meaning the underlying key used to encrypt the key database will change and all the private records in the key database will be reencrypted (and the authentication data on the trust records will be resigned).

If you already have a sql: database and you want to merge in from another database, certutil --merge is the correct command. For most instances (you have a dbm database and you want to create an sql database), certutil -N will work just fine. (actually certutil -L -X would work as well without requiring 2 passwords).

bob

Comment 4

[Robert Relyea]

FYI, I believe the NSS tests use certutil -G, but it has the side effect of creating an orphaned key in the new database.

Comment 5

[Kai Engert (:KaiE:)]

(In reply to Robert Relyea from comment #3)
> actually certutil -L -X would work as well without requiring 2 passwords

This sounds like a good approach.
However, it doesn't prompt for the password, even if there's a private key in the dbm database.
Is it possible that this triggers only a partial migration?

How about using -K (list private keys), which asks for the password?

  certutil -K -X -d sql:/path/to/db-to-migrate

When testing this with an input dbm database that contains a private key, 
and comparing the results of "certutil -L -X" and "certutil -K -X",
the resulting key4.db created by -K is bigger, which seems to support my theory that "-L -X" does a partial migration, only.

I'd prefer that we recommend a command that performs a complete migration (not partial migration).

(This could also help with selinux environments, where a daemon is often restricted to read-only, and where an automatic migration is undesirable. In such an environment, a one-time conversion with more privileges can be used to avoid the automatic migration.)

Christian, how does the -K -X command work for you?

Comment 6

[Robert Relyea]

You are right -K -X is the correct way to do this. We should try changing the update tests. nss/tests/dbupgrade/dbupgrade.sh

Change the certutil -G -g  and the certutil -S -g -n  to certutil -K -X

If the tests pass, then certutil -K -X should be fine. (The dbupgrade tests take dbm created databases and upgrade them to sql databases then runs all the tests against the resulting sql databases).

Comment 7

[Christian Heimes]

The command

    certutil -K -X -d sql:/path/to/db-to-migrate

fails with error code 255 when NSSDB does not contain any private key. According to my tests, -L -X does not fail with am empty DB and still migrates all certs, keys, and security modules.

I also noticed that certutil does not retain retain permission and ownership. I can work around the issue.

Comment 8

[Christian Heimes]

Correction, '-L -X' does *NOT* migrate the database. I had a bug in my test case.

For a NSSDB with at least one private key, I can successfully migrate the DB with:

    certutil -K -X -d sql:/path/to/db-to-migrate -f /path/to/pwdfile

Without any private keys I have to use:

    certutil -N -d sql:/path/to/db-to-migrate -f /path/to/pwdfile -@ /path/to/pwdfile

With '-K' argument, certutils fails with error code 255. With '-N', certutils migrates public certs from old DB format to new DB format.

Comment 9

[Kai Engert (:KaiE:)]

Christian, thanks for your analysis.

We really need a command that works for all scenarios, for any database state.
I think you have shown that none of Bob's or my suggestions are sufficient.

It seems the command you're suggesting is the move universal solution.

Looking at the certutil code, I don't see any explicit action performed by the -N command, it seems to rely on the standard NSS init code path, and rely on NSS doing migration automatically.

It's unfortunate that the password parameter must be given twice, that's not very intuitive.

We could potentially introduce a new command that's easier and more discoverable, but on the other hand, if we do, people might try to migrate with older versions that don't have the new command yet. This would be confusing.

It seems that recommending -N -f -@ might be the best approach.

Comment 10

[Kai Engert (:KaiE:)]

(In reply to Kai Engert (:kaie:) from comment #9)
> It seems the command you're suggesting is the move universal solution.

"most" universal

Comment 11

[Kai Engert (:KaiE:)]

I have added documentation on the following Fedora wiki page to section "Upgrade/compatibility impact":
  https://fedoraproject.org/wiki/Changes/NSSDefaultFileFormatSql#Upgrade.2Fcompatibility_impact