200775 - FIPS mode not working after upgrade from 1.3-final to 1.4-alpha

Status: RESOLVED FIXED

(Core Graveyard :: Security: UI, defect)

Description

[ptashek]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.4a) Gecko/20030401
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.4a) Gecko/20030401

After an upgrade from 1.3-final to 1.4-alpha I'm unable to access any data
maintained by the PSM when in FIPS mode (mainly stored passwords).
Navigator/Mail claims it's unable to initialize the secuirity component.

Reproducible: Always

Steps to Reproduce:
1. Install 1.3-final and create a new user profile.
2. Enable FIPS for the newly created profile.
3. Make PSM store some passwords while in FIPS mode.
4. Upgrade to 1.4-alpha.
5. Try to access any site/mail account which has a password maintained by PSM.

Actual Results:  
I've received an alert stating "Could not initialize the browser's security
component"

Expected Results:  
The expected resuklt was Mozilla to ask for FIPS master password and, if
correct, retrieve requested data from the maintained database (in this case
e-mail account passwords).

System config: Windows 2000 SP3 + all critical fixes released up-to-date

Comment 1

[ptashek]

Attachment: Alert screenshot

Here's a screenshot of the alert I get from Mozilla while trying to access
stored passwords in FIPS mode.

Comment 2

[ptashek]

To make things clear, there is enough disk space (4.05GB) and the NTFS access
rights and other attributes are allowing full read/write access.

Comment 3

[Stephane Saux]

If you log in manually to the FIPS module it should work.

Comment 4

[ptashek]

Manual FIPS-logon also doesn't work (Mozilla shows FIPS as disabled and does not
react when the "Enable FIPS" button is clicked).

I can confirm that also for Windows NT 4.0 SP6a

Comment 5

[Brant Gurganus]

I too have been able to reproduce this in the 1.4 builds.

The Enable FIPS button gives the following JavaScript Exception:

Error: uncaught exception: [Exception... "Component returned failure code:
0x80004005 (NS_ERROR_FAILURE) [nsIPKCS11ModuleDB.toggleFIPSMode]"  nsresult:
"0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame ::
chrome://pippki/content/device_manager.js :: toggleFIPS :: line 431"  data: no]

Recent changes to the mentioned JS file were done by [e-mail] from bug
189205.  Last change to device_manager.xul was by [e-mail] for bug 149841.

Comment 6

[Asa Dotzler [:asa]]

This is probably used (and required) by a lot of users. It's a recent regression
that we should try to track down for 1.4.

Comment 7

[Asa Dotzler [:asa]]

Kaie, can you look into this regression and let us know what can be done? 

Comment 8

[Kai Engert (:KaiE:)]

I can reproduce the problem on my Linux machine.

I see the problem with both 1.4a and 1.4b. If the security database is set to be
in FIPS mode, any attempt use crypto fails. If using a database that is not yet
in FIPS mode, crypto works, but it's not possible to switch to FIPS mode.

Unfortunately, I only can reproduce using downloaded builds. My own build works
perfectly!

This looks like some kind of packaging problem, possible having to do with NSS'
new feature to do some kind of self integrity checksum/signature verification tests?

I suspect that, because when I try to use FIPS crypto with 1.4a, I see the
following on the console:
File /home/kaie/test-profiles/../releases/mozilla-1.4a/libsoftokn3.so: 445152 bytes
  hash: 20 bytes
    9f b5 96 9b a8 a3 9a 1c af 05
    ae 65 80 55 7a 75 29 e1 9a 3a
  signature: 40 bytes
    17 dd 62 51 97 20 8e be e8 b8
    05 16 b8 5d 3d df 46 54 a3 8c
    74 93 9d e9 5a 35 71 22 ef fe
    43 0f 95 e7 44 9e c4 2f 39 67
Verified : FALSE

Comment 9

[Kai Engert (:KaiE:)]

I can reproduce the problem with my own build, if I create a .tar.gz package
from my own build, extract the distribution package, and try to run that.
Something must be missing. Investigating what is missing.

Comment 10

[Kai Engert (:KaiE:)]

I'm giving updates as I have them.

SECMOD_CanDeleteInternalModule() returns "1"

but

SECMOD_DeleteInternalModule("NSS Internal PKCS #11 Module") 
returns -1

Comment 11

[Kai Engert (:KaiE:)]

Ok, found the problem, at least on Linux.
Since the 1.4 development phase, NSS is now checking a checksum on its nssckbi
shared library.

The checksum gets created on the original version of the library, but the
distribution package contains a "stripped" version (from debugging symbols) of
that library, and is therefore different.

I seem to remember we already discussed this in the past already. Either we have
not fixed the problem at all yet, or we made a mistake. I'm trying to find the
other bug.

I'll also check whether the same thing happens on Windows.

Comment 12

[Wan-Teh Chang]

This is a known problem.  There is already a report of
this bug in Bugzilla.  The Windows build has the same
problem but may have a different cause.

The fix is to regenerate the softokn3 checksum file
whenever the softokn3 shared library/DLL is post-processed
in any way (for example, if it is stripped).  The
shlibsign tool should be used to regenerate the checksum
file.

Comment 13

[Kai Engert (:KaiE:)]

*** Bug 200757 has been marked as a duplicate of this bug. ***

Comment 14

[Kai Engert (:KaiE:)]

Asa, I think we need more help, maybe you can help us find the right people.

On all platforms, we have now a strict dependency between shared library file
*softokn3* and its related checksum/signature file *.chk

It seems common that our packaging processes are doing post processing of shared
libraries.
Any post processing makes the checksum file to become incorrect, and the
software will refrain to work.

It's very likely that on Linux, the only post processing is the "strip" tool
that removes debugging information from the library.

However, I think on Windows there is no such strip tool. Because the same
failure has been reported on Windows, too, there might be some other post
processing that breaks it - Wan-Teh suspects possibly talkbalk could cause that?


What we have to do is:
- investigate all our different packaging procedures
- add an additional step to the packaging procedure, that happens
  after any shared library post processing,
  executing a command line like (Linux example):
    shlibsign -v -i libsoftokn3.so
  that produces an updated chk file

Do you know who knows our packaging processes well enough to be able to help?

I might be able to help, too, if we find people that can point me to the right
places where the final signature generation should happen.

Comment 15

[Asa Dotzler [:asa]]

leaf, bryner, cls, can one of you all help us with this blocker?

Comment 16

[Kai Engert (:KaiE:)]

Discussed with Bryner and Leaf, and Leaf is now working on a patch.

Comment 17

[Daniel (Leaf) Nunes]

working on a fix, need to update my tree and test. ETA: sometime this afternoon.

Comment 18

[Jon Granrose]

adding sgehani, we need this in 1.4f

Comment 19

[Daniel (Leaf) Nunes]

assigning to me to corral the necessary build/installer/packaging fixes for each
platform in.

I've tested my windows patch on a release system, the binaries can be found in:
http://ftp.mozilla.org/pub/mozilla/nightly/2003-05-16-08-trunk/

Might need sgehani's fix for the linux installer bits. 

Comment 20

[Wan-Teh Chang]

Please attach your patches and Bob Relyea or I will
review them.  Thank you.

Comment 21

[Daniel (Leaf) Nunes]

Attachment: windows build system change for signing during a release build

Comment 22

[Robert Relyea]

Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

This patch looks good assuming all the NSS shared libraries are in $(DIST)/bin
or in the path (seems like a reasonable assumption since softokn3.dll is as
well).

It will be painfull obvious if they are not (the build would fail).

Comment 23

[Brian Ryner (not reading)]

leaf, are there also changes needed to xpinstall/packager to make sure we sign
the NSS libraries after doing stripping?

Comment 24

[Wan-Teh Chang]

Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

r=wtc.

Same comments as Relyea's.

You need to make sure that $(DIST)/bin/softokn3.dll and
$(DIST)/bin/softokn3.chk are the copies of those files
that you will include in the installer, because NSS itself
generates those files under $(DIST)/lib.

Comment 25

[Daniel (Leaf) Nunes]

bryner: if there's a strip pass in the windows installer creation, then yes, one
will need to be added there. that *should* take care of unix installer builds as
well, yes?

relyea: the build target will fail, we'll have to add something to the windows
build automation that tests the make return status (currently, we don't check
it, so the signing can fail, and the rest of the release packaging would carry
on; we should at least warn about it).

wtc: thanks, i'll get this checked in.

I'm not likely going to be able to work on this (the unix installer, and windows
installer if there's a strip pass after the splitsym run has happened) this
weekend, and if i can't and/or it doesn't get fixed by monday, it will be top
priority then.

Comment 26

[Brian Ryner (not reading)]

Yeah, I mainly meant the strip pass that's done for unix.

Comment 27

[Wan-Teh Chang]

I verified that I can enable FIPS mode with Leaf's win32
build (2003051608) in comment 19.

Comment 28

[Asa Dotzler [:asa]]

Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

a=asa (on behalf of drivers) for checkin to 1.4

Comment 29

[Brant Gurganus]

I was able to enable FIPS mode in 2003051708 on WinXP SP1 Home.  However, if
FIPS mode is disabled, the Security Devices Manager needs to be reopened before
it can be reenabled.  Is that by design or a bug?

Comment 30

[Kai Engert (:KaiE:)]

> I was able to enable FIPS mode in 2003051708 on WinXP SP1 Home.  However, if
> FIPS mode is disabled, the Security Devices Manager needs to be reopened before
> it can be reenabled.  Is that by design or a bug?

That's a consequence of the design. Before we can change again, all open handles
must get closed, however, the open window still has handles allocated in the
JavaScript context. So, after the cleanup on Windows close, it works again.

Comment 31

[Daniel (Leaf) Nunes]

Attachment: after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

Comment 32

[Daniel (Leaf) Nunes]

Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

looking for review. i know this is a hack, but i don't have time to implement
and test a better fix before 1.4 final ships.

Comment 33

[Daniel (Leaf) Nunes]

Attachment: er, use the right options to shlibsign

same as the last patch, but with "-v -i" to the shlibsign call.

Comment 34

[Wan-Teh Chang]

Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

This patch is probably no good.  The most serious problem is
that the hardcoded .so suffix means it won't work on HP-UX.

I suggest that we simply not strip the softokn3 shared library.
If I understand Mozilla's build system correctly, stripping is
done here:
http://lxr.mozilla.org/seamonkey/source/xpinstall/packager/Makefile.in#184

My suggested workaround is to add this line:

    ! -name "*softokn3.*" \

to the 'find' command.

Comment 35

[Kai Engert (:KaiE:)]

What about Unix packages that don't get created with the deliver.pl script?
Will the .tar.gz packages work, too?

Comment 36

[Wan-Teh Chang]

Comment on attachment 123734 [details] [diff] [review]
er, use the right options to shlibsign

Another problem with this patch is that you are not
setting the platform-dependent shared library search
path environment variable (which is LD_LBRARY_PATH on
most Unix variants) to point to the NSS and NSPR shared
libraries required by 'shlibsign'.  If it works on Linux,
it's because the NSS and NSPR shared libraries are
available in /usr/lib in recent Red Hat Linux releases.

Comment 37

[Daniel (Leaf) Nunes]

wtc: what should we set LD_LIBRARY_PATH to (relative to mozilla/) for shlibsign
to work?

Comment 38

[Daniel Veditz [:dveditz]]

Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

clearing sr request until there's a reviewed patch. the nss guys don't seem to
like this one.

Comment 39

[Wan-Teh Chang]

Leaf, see http://lxr.mozilla.org/seamonkey/source/security/nss/cmd/shlibsign/sign.sh
for an example of how LD_LIBRARY_PATH should be set.  Note that the case
for OpenVMS is very complicated and may require a shell script wrapper.
Also note that OS/2 uses the file
http://lxr.mozilla.org/seamonkey/source/security/nss/cmd/shlibsign/sign.cmd
instead, but if Mozilla copies the DLLs to $(DIST)/bin, we may not need
to use 'sign.cmd' on OS/2 (just like Windows).

Comment 40

[Daniel (Leaf) Nunes]

Attachment: put nss signing step in xpinstall/packager/Makefile.in

this patch re-signs the lib once it's in the package-creation staging area and
has been stripped. The signature should remain valid if the lib is stripped
again.

Again, this is somewhat of a hack just to get us through 1.4, a better fix
incorporates some kind of an export "need to be signed" list from nss so we
don't have to worry about updating the version number when NSS revs.

Comment 41

[Daniel Veditz [:dveditz]]

Comment on attachment 123809 [details] [diff] [review]
put nss signing step in xpinstall/packager/Makefile.in

Dunno how you got two of everything... clearing extras (hope the multi-submit
bug hasn't come back)

sr=dveditz if it's OK with wtc

Comment 42

[Daniel (Leaf) Nunes]

Attachment: nss signing step in xpinstall/packager/Makefile.in

After talking with wtc, i think i fixed the way the ld_library_paths are
referenced, and added a couple of other files that might need to get signed on
other unix platforms.

Comment 43

[Daniel (Leaf) Nunes]

Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

seeking review. i hope i can get this in before tomorrow.

Comment 44

[Wan-Teh Chang]

Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

r=wtc.	Make sure you test this on a non-Linux machine.

Some suggested changes.

1. Use line continuation (\) for the long SIGN_ENV line.

2. "cd $(DIST)/$(MOZ_PKG_APPNAME)" and the "$(DIST)/$(MOZ_PKG_APPNAME)"
in SOFTOKN, FREEBL_PURE, and FREEBL_HYBRID are redundant.  Use one.
If you choose the cd command, you only need to cd once.

3. $(SOFTOKN) exists on all platforms, so you can omit
the existence test for it.

Comment 45

[Daniel (Leaf) Nunes]

suggested changes made, tested on linux. trying to find another machine to test on.

Comment 46

[Daniel Veditz [:dveditz]]

Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

sr=dveditz, a=dveditz

Comment 47

[Daniel (Leaf) Nunes]

fix with wtc's suggestions implemented, tested on solaris 2.6 and linux, checked in.

Comment 48

[Wan-Teh Chang]

Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

Leaf, could you delete the redundant "cd $(DIST)/$(MOZ_PKG_APPNAME) &&"
in SIGN_CMD?  Thanks.

Comment 49

[Wan-Teh Chang]

Attachment: Remove the redundant cd command from SIGN_CMD

Leaf, this is the change I was talking about.

Comment 50

[Kai Engert (:KaiE:)]

*** Bug 205183 has been marked as a duplicate of this bug. ***

Comment 51

[Nelson Bolyard (seldom reads bugmail)]

*** Bug 200215 has been marked as a duplicate of this bug. ***