1360900 - Floating Point Exception in __hash_open (hash.c:229)

Status: RESOLVED WONTFIX

(NSS :: Tools, defect, P3)

Description

[Brian Carpenter [:geeknik]]

Attachment: cert8.db (SIGFPE)

Triggered with changeset 13315:769f9ae07b10. Built with afl-clang-fast on Debian 8 x64 (CC=afl-clang-fast CXX=afl-clang-fast++ AFL_USE_ASAN=1 USE_64=1 make nss_build_all).

run ./certutil -U -d . with the attached cert8.db.

Program received signal SIGFPE, Arithmetic exception.
0x00007ffff1367bd7 in __hash_open (file=<optimized out>, flags=<optimized out>, mode=<optimized out>, info=<optimized out>, dflags=<optimized out>) at hash.c:229
229             nsegs = (hashp->MAX_BUCKET + 1 + hashp->SGSIZE - 1) /
(gdb) bt
#0  0x00007ffff1367bd7 in __hash_open (file=<optimized out>, flags=<optimized out>, mode=<optimized out>, info=<optimized out>, dflags=<optimized out>) at hash.c:229
#1  0x00007ffff13659ff in dbopen (fname=0x60200000e890 "./cert8.db", flags=<optimized out>, mode=384, type=<optimized out>, openinfo=0x7ffff1380500 <dbs_hashInfo>) at db.c:103
#2  0x00007ffff12f998f in dbsopen (dbname=0x60200000e890 "./cert8.db", flags=-134322312, mode=0, type=DB_HASH, userData=<optimized out>) at dbmshim.c:515
#3  0x00007ffff1348580 in nsslowcert_OpenPermCertDB (handle=<optimized out>, readOnly=<optimized out>, appName=<optimized out>, prefix=<optimized out>, namecb=<optimized out>,
    cbarg=<optimized out>) at pcertdb.c:4086
#4  nsslowcert_OpenCertDB (handle=<optimized out>, readOnly=<optimized out>, appName=<optimized out>, prefix=<optimized out>, namecb=<optimized out>, cbarg=<optimized out>,
    openVolatile=<optimized out>) at pcertdb.c:4587
#5  0x00007ffff132feb7 in lg_OpenCertDB (configdir=<optimized out>, prefix=<optimized out>, readOnly=<optimized out>, certdbPtr=<optimized out>) at lginit.c:365
#6  legacy_Open (configdir=<optimized out>, certPrefix=<optimized out>, keyPrefix=<optimized out>, certVersion=<optimized out>, keyVersion=<optimized out>, flags=1, certDB=<optimized out>,
    keyDB=<optimized out>) at lginit.c:609
#7  0x00007ffff1d2a4c4 in sftkdbCall_open (dir=<optimized out>, certPrefix=<optimized out>, keyPrefix=<optimized out>, certVersion=<optimized out>, keyVersion=<optimized out>,
    flags=<optimized out>, certDB=<optimized out>, keyDB=<optimized out>) at lgglue.c:306
#8  0x00007ffff1dc2a2f in sftk_DBInit (configdir=<optimized out>, certPrefix=<optimized out>, keyPrefix=<optimized out>, updatedir=<optimized out>, updCertPrefix=<optimized out>,
    updKeyPrefix=<optimized out>, updateID=<optimized out>, readOnly=<optimized out>, noCertDB=<optimized out>, noKeyDB=<optimized out>, forceOpen=<optimized out>, isFIPS=<optimized out>,
    certDB=<optimized out>, keyDB=<optimized out>) at sftkdb.c:2584
#9  0x00007ffff1d40991 in SFTK_SlotReInit (slot=0x61200000b5c0, configdir=<optimized out>, updatedir=<optimized out>, updateID=<optimized out>, params=0x6110000097f8, moduleIndex=<optimized out>)
    at pkcs11.c:2484
#10 0x00007ffff1d4214d in SFTK_SlotInit (configdir=<optimized out>, updatedir=<optimized out>, updateID=<optimized out>, params=0x6110000097f8, moduleIndex=<optimized out>) at pkcs11.c:2600
#11 0x00007ffff1d45738 in nsc_CommonInitialize (pReserved=<optimized out>, isFIPS=<optimized out>) at pkcs11.c:3052
#12 0x00007ffff1d461c9 in NSC_Initialize (pReserved=0x7fffffffdcf0) at pkcs11.c:3115
#13 0x00007ffff67bec1b in secmod_ModuleInit (mod=<optimized out>, reload=<optimized out>, alreadyLoaded=<optimized out>) at pk11load.c:245
#14 0x00007ffff67c0a62 in secmod_LoadPKCS11Module (mod=<optimized out>, oldModule=0x7fffffffe0e0) at pk11load.c:504
#15 0x00007ffff67f49df in SECMOD_LoadModule (
    modulespec=0x61400000fe40 "library= name=\"NSS Internal PKCS #11 Module\" parameters=\"configdir='.' certPrefix='' keyPrefix='' secmod='secmod.db' flags=readOnly updatedir='' updateCertPrefix='' updateKeyPrefix='' updateid='' upda"..., parent=<optimized out>, recurse=<optimized out>) at pk11pars.c:1672
#16 0x00007ffff67f4d48 in SECMOD_LoadModule (
    modulespec=0x61200000bbc0 "name=\"NSS Internal Module\" parameters=\"configdir='.' certPrefix='' keyPrefix='' secmod='secmod.db' flags=readOnly updatedir='' updateCertPrefix='' updateKeyPrefix='' updateid='' updateTokenDescription"..., parent=<optimized out>, recurse=<optimized out>) at pk11pars.c:1707
#17 0x00007ffff6711f01 in nss_InitModules (configdir=0x13a6040 <SECU_ConfigDirectory.buf> ".", certPrefix=<optimized out>, keyPrefix=<optimized out>, secmodName=<optimized out>,
    updateDir=<optimized out>, updCertPrefix=<optimized out>, updKeyPrefix=<optimized out>, updateID=<optimized out>, updateName=<optimized out>, configName=<optimized out>,
    configStrings=<optimized out>, pwRequired=<optimized out>, readOnly=<optimized out>, noCertDB=<optimized out>, noModDB=<optimized out>, forceOpen=<optimized out>,
    optimizeSpace=<optimized out>, isContextInit=<optimized out>) at nssinit.c:464
#18 nss_Init (configdir=<optimized out>, certPrefix=<optimized out>, keyPrefix=<optimized out>, secmodName=<optimized out>, updateDir=<optimized out>, updCertPrefix=<optimized out>,
    updKeyPrefix=<optimized out>, updateID=<optimized out>, updateName=<optimized out>, initContextPtr=<optimized out>, initParams=<optimized out>, readOnly=<optimized out>,
    noCertDB=<optimized out>, noModDB=<optimized out>, forceOpen=<optimized out>, noRootInit=<optimized out>, optimizeSpace=<optimized out>, noSingleThreadedModules=<optimized out>,
    allowAlreadyInitializedModules=<optimized out>, dontFinalizeModules=<optimized out>) at nssinit.c:689
#19 0x00007ffff67130a2 in NSS_Initialize (configdir=0x61700000f914 "", certPrefix=0x7ffff7fe6778 "\313y", keyPrefix=0x0, secmodName=0x0, flags=<optimized out>) at nssinit.c:889
#20 0x00000000004ddd8e in certutil_main (argc=<optimized out>, argv=<optimized out>, initialize=<optimized out>) at certutil.c:2986
#21 0x00000000004db7b4 in main (argc=63764, argv=0x7ffff7fe6778) at certutil.c:3703

Comment 1

[Daniel Veditz [:dveditz]]

Unless you can cause the creation of a corrupted cert8.db from remote content this isn't really a vulnerability in Firefox. Leaving hidden because maybe it is for other NSS applications.

Comment 2

[Brian Carpenter [:geeknik]]

Assigned CVE-2017-11697.

Comment 3

[Al Billings [:abillings - ex-MoCo]]

(In reply to Brian Carpenter [:geeknik] from comment #2)
> Assigned CVE-2017-11697.


Assigned by whom?

Comment 4

[Kai Engert [:KaiE:]]

How did you create the attached cert8.db ?

Comment 5

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Bulk change per wleung.

Comment 6

[J.C. Jones [:jcj] (he/they)]

I’m sorry this bug didn’t get suitable, timely attention, nor follow-up. This CVE was not tracked in Mozilla’s lists (since the CVE wasn’t allocated by us), and both age and turnover in the NSS team led to it being dropped.

This bug and its peers from the 9 Aug 2017 disclosure [0] are all in libnssdbm, which has been replaced by a SQLite datastore, starting in NSS 3.12 in 2008 [1]. In 2018, Firefox 60 and NSS 3.35 made SQLite the default [2], and in Bug 1594931 (Firefox 73) and Bug 1594933 (NSS 3.49) we will stop building this legacy database by default [3][4].

These bugs are real and easily demonstrated, but require local modification of the profile directory, and thus are difficult to exploit widely. The underlying causes are deep within DBM, which was legacy ndbm code even back unto the first commits of NSS in Netscape. Fixing these issues is effectively fixing structual problems with the serialization layer of ndbm from the early 1990s. Unfortunately, these bugs are not shallow. The solution is to move to the SQLite format and leave this deprecated, legacy code until we can remove it entirely in the early 2020s.

For that reason, I am closing this and its peer bugs as WONTFIX. As [0] already disclosed the bugs, I am going to open them up as well to explain this publicly.

[0] https://seclists.org/fulldisclosure/2017/Aug/17
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.12_release_notes.html
[2] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.35_release_notes
[3] https://groups.google.com/d/msg/mozilla.dev.security/n5VNRpGwRIQ/Kr6_S34ZAQAJ
[4] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.48_release_notes