It's possible under some circumstances to ask Firefox to enable FIPS mode and have that succeed, even if NSS is using non-FIPS ciphersuites and settings (See Bug 106604, and I've specifically seen this with ChaCha20/Poly1394 being used while in FIPS mode). Since Bug 106604 isn't readily solvable, this bug's purpose is to fail to enter FIPS mode unless it has a high likelihood of truly complying with the FIPS 140-2 guidelines. The first option that occurs to me is before we attempt to set the flag on NSS and restart it, to enumerate the available cryptographic functions and compare them to a whitelist. If there are functions observed which are not on the whitelist, fail. Another option (#2) would be to introduce a compile-time flag on Firefox that, unless set, would force the FIPS button to fail immediately. Anyone wishing to use Firefox with the FIPS toggle button functioning would need to set that compile-time flag. Mozilla would not set this flag. Option #3 would be to solve Bug 106604.
You need to log in before you can comment on or make changes to this bug.