1586192 - implicit nssdb migration (DBM=>SQL) and NSS_INIT_NOMODDB

Status: NEW

(NSS :: Libraries, enhancement, P2)

Description

[Stanislav Levin]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

There is a nice project named Certmonger (https://pagure.io/certmonger/).

The goal is to have certmonger do what you need it to do based on what you've told it you need. If you already have a certificate, it will be happy to just check on it periodically and warn you when it's about to expire.

For example, one of the tracked certs is placed on NSS db (DBM):

LANG=C ls -la /var/lib/pki/pki-tomcat/alias/

-rw------- 1 pkiuser pkiuser 65536 Aug 13 09:12 cert8.db
-rw------- 1 pkiuser pkiuser 24576 Aug 13 09:12 key3.db
-rw------- 1 pkiuser pkiuser 16384 Jul 16 16:30 secmod.db

Then we upgrade a distro branch where previously, the default NSS DB type was DBM to the one - SQL.
After that procedure, Certmonger gets restarted and then, as usual, opens NSS DB with NSS_INIT_NOMODDB.

https://pagure.io/certmonger/blob/master/f/src/certread-n.c#_103

ctx = NSS_InitContext(entry->cm_cert_storage_location,
		      NULL, NULL, NULL, NULL,
		      (readwrite ? 0 : NSS_INIT_READONLY) |
		      NSS_INIT_NOROOTINIT |
		      NSS_INIT_NOMODDB);

This causes the implicit NSS DB migration (DBM => SQL).

Note: the simple reproducer in Pytest is attached.
To run it: python3 -m pytest repr.py -v

Actual results:

Due to that flag (NSS_INIT_NOMODDB)
secmod.db is not migrated to pkcs11.txt.

LANG=C ls -la /var/lib/pki/pki-tomcat/alias/

-rw------- 1 pkiuser pkiuser 65536 Aug 13 09:12 cert8.db
-rw------- 1 root root 28672 Aug 13 09:12 cert9.db
-rw------- 1 pkiuser pkiuser 24576 Aug 13 09:12 key3.db
-rw------- 1 root root 28672 Aug 13 09:12 key4.db
-rw------- 1 pkiuser pkiuser 16384 Jul 16 16:30 secmod.db

This results in a half-migrated state.
The owner of this DB (Dogtag PKI) fails to run a self-check because of a missing pkcs11.txt.

Expected results:

Proposal: the migration should not be triggered at all in the case of secmod.db and NSS_INIT_NOMODDB.

Comment 1

[Stanislav Levin]

Attachment: repr.py

Comment 2

[Kevin Jacobs [:kjacobs]]

Thanks for the report. While this is a bit of an unexpected result, the legacy DB will be going away eventually, so ideally we don't stop upgrades in this case.

Would it be possible for the application to detect this failure, attempt opening with NSS_INIT_NOCERTDB (which I presume will upgrade secmod.db) or no flags at all, then retry?

I'm open to other suggestions, but it's hard to see a solution that doesn't involve some change to the consuming application.

Comment 3

[Alexander Scheel]

Kevin,

One of the Dogtag maintainers here. Context from this request is that Certmonger and Dogtag are most typically deployed together as part of IPA. In this scenario, IPA sets up a Dogtag CA, and uses Certmonger to manage its certificates for renewal (for example, the CA subsystem SSL certificate).

On a system upgrade that Stanislav described (from an OS release with DBM as NSS default to one with SQL as NSS default), Certmonger will run, opening the NSS DB as above. Even though it is supposed to be read-only, it still triggers (at least a partial) NSS DB migration. This is unwanted behavior.

When Dogtag later starts post-upgrade, it will open the NSS DB read-write and attempt to do a proper migration. This fails (on our part) because a partial migration has already occurred -- and we're unwilling to do a new migration since a partial one already exists. What happens if the user added certs to either partial DB? What happens if a user removed certs? &c.

I'd propose NSS DB with suitable flags (perhaps NSS_INIT_READONLY?) simply not trigger a migration at all. Then we can handle the migration afterwards when Dogtag starts.

Otherwise, care needs to be taken (IMO, inside NSS) that if a migration has been started, it is complete (pending fatal errors), and thus generates a pkcs11.txt file.

So yes, at some point, consuming applications will need to be modified to do the migration, but in this particular case, that already exists. It just needs to be thought of in a context broader than just the certmonger application.

Some of our discussions about this topic can be found below:

• https://pagure.io/dogtagpki/issue/3104
• https://github.com/dogtagpki/pki/pull/239
• https://pagure.io/certmonger/issue/133

My 2c.

Comment 4

[Stanislav Levin]

What is about new NSS_INIT_NOAUTOUPGRADE flag (to disable auto migration)?

Comment 5

[Kevin Jacobs [:kjacobs]]

Apologies for the slow reply and thanks for the additional context.

I can reproduce this locally (just using certutil, since I don't have the python dependencies configured) when calling NSS_InitContext with flags NSS_INIT_NOROOTINIT | NSS_INIT_NOMODDB:

../../dist/Debug/bin/certutil -G -d . -h internal -f pw.txt -z pw.txt
Generating key.  This may take a few moments...
ls
total 328
-rw-------  1 kjacobs  staff  65536 Nov 20 10:51 cert8.db
-rw-------  1 kjacobs  staff  28672 Nov 20 10:51 cert9.db
-rw-------  1 kjacobs  staff  16384 Nov 20 10:51 key3.db
-rw-------  1 kjacobs  staff  36864 Nov 20 10:51 key4.db
-rw-r--r--  1 kjacobs  staff      5 Nov 20 10:37 pw.txt
-rw-------  1 kjacobs  staff  16384 Nov 20 10:46 secmod.db

However, when using NSS_INIT_READONLY | NSS_INIT_NOROOTINIT | NSS_INIT_NOMODDB:

../../dist/Debug/bin/certutil -G -d . -h internal -f pw.txt -z pw.txt
Generating key.  This may take a few moments...
certutil: unable to generate key(s)
: SEC_ERROR_READ_ONLY: security library: read-only database.
ls
total 200
-rw-------  1 kjacobs  staff  65536 Nov 20 10:47 cert8.db
-rw-------  1 kjacobs  staff  16384 Nov 20 10:47 key3.db
-rw-r--r--  1 kjacobs  staff      5 Nov 20 10:37 pw.txt
-rw-------  1 kjacobs  staff  16384 Nov 20 10:46 secmod.db

So, I assume the initial NSS_InitContext call from Certmonger is made with readwrite==0.

It seems like what you suggest is already supported through the NSS_INIT_READONLY flag. Have I missed something?

Comment 6

[Rob Crittenden]

certmonger maintainer...

certmonger opens the database with NSS_INIT_NOROOTINIT | NSS_INIT_NOMODDB specifically to see if it has the correct perms. If it fails w/o errno == EACCES or EPERM it opens it again with NSS_INIT_NOROOTINIT | NSS_INIT_NOMODDB | NSS_INIT_READONLY and if successful that means that it lacks write perms. Upon success it would then close the database and re-opens it w/o the NSS_INIT_NOMODDB and proceed to do what it needs.

The reason being that SEC_ERROR_BAD_DATABASE is generally returned when a database fails to open and it is a rather broad and not particularly helpful error, just that something went wrong. It can mean any number of different things. It is the return value for 26 different types of errors in softoken/legacydb/pcertdb.c alone. I know, legacy, consistent API and it dates back to the late 90's. The original author of certmonger did this open and re-open scheme as a way of working around some of that.