Closed Bug 200775 Opened 18 years ago Closed 18 years ago

FIPS mode not working after upgrade from 1.3-final to 1.4-alpha

Categories

(Core Graveyard :: Security: UI, defect)

Other Branch
defect
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED
Future

People

(Reporter: ptashek, Assigned: leaf)

References

(Blocks 1 open bug)

Details

(Keywords: regression, Whiteboard: adt1)

Attachments

(4 files, 3 obsolete files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.4a) Gecko/20030401
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; pl-PL; rv:1.4a) Gecko/20030401

After an upgrade from 1.3-final to 1.4-alpha I'm unable to access any data
maintained by the PSM when in FIPS mode (mainly stored passwords).
Navigator/Mail claims it's unable to initialize the secuirity component.

Reproducible: Always

Steps to Reproduce:
1. Install 1.3-final and create a new user profile.
2. Enable FIPS for the newly created profile.
3. Make PSM store some passwords while in FIPS mode.
4. Upgrade to 1.4-alpha.
5. Try to access any site/mail account which has a password maintained by PSM.

Actual Results:  
I've received an alert stating "Could not initialize the browser's security
component"

Expected Results:  
The expected resuklt was Mozilla to ask for FIPS master password and, if
correct, retrieve requested data from the maintained database (in this case
e-mail account passwords).

System config: Windows 2000 SP3 + all critical fixes released up-to-date
Attached image Alert screenshot
Here's a screenshot of the alert I get from Mozilla while trying to access
stored passwords in FIPS mode.
To make things clear, there is enough disk space (4.05GB) and the NTFS access
rights and other attributes are allowing full read/write access.
Summary: Unable to use PSM when in FIPS mode after upgrade from 1.3-final to 1.4-alpha → Unable to use PSM when in FIPS mode after upgrade from 1.3-final to 1.4-alpha
If you log in manually to the FIPS module it should work.
Assignee: ssaux → kaie
Severity: blocker → normal
Target Milestone: --- → Future
Manual FIPS-logon also doesn't work (Mozilla shows FIPS as disabled and does not
react when the "Enable FIPS" button is clicked).

I can confirm that also for Windows NT 4.0 SP6a
I too have been able to reproduce this in the 1.4 builds.

The Enable FIPS button gives the following JavaScript Exception:

Error: uncaught exception: [Exception... "Component returned failure code:
0x80004005 (NS_ERROR_FAILURE) [nsIPKCS11ModuleDB.toggleFIPSMode]"  nsresult:
"0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame ::
chrome://pippki/content/device_manager.js :: toggleFIPS :: line 431"  data: no]

Recent changes to the mentioned JS file were done by kaie@netscape.com from bug
189205.  Last change to device_manager.xul was by robin.lu@sun.com for bug 149841.
Blocks: fips
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking1.4?
OS: Windows 2000 → All
Hardware: PC → All
Summary: Unable to use PSM when in FIPS mode after upgrade from 1.3-final to 1.4-alpha → FIPS mode not working after upgrade from 1.3-final to 1.4-alpha
This is probably used (and required) by a lot of users. It's a recent regression
that we should try to track down for 1.4.
Flags: blocking1.4? → blocking1.4+
Keywords: regression
Kaie, can you look into this regression and let us know what can be done? 
I can reproduce the problem on my Linux machine.

I see the problem with both 1.4a and 1.4b. If the security database is set to be
in FIPS mode, any attempt use crypto fails. If using a database that is not yet
in FIPS mode, crypto works, but it's not possible to switch to FIPS mode.

Unfortunately, I only can reproduce using downloaded builds. My own build works
perfectly!

This looks like some kind of packaging problem, possible having to do with NSS'
new feature to do some kind of self integrity checksum/signature verification tests?

I suspect that, because when I try to use FIPS crypto with 1.4a, I see the
following on the console:
File /home/kaie/test-profiles/../releases/mozilla-1.4a/libsoftokn3.so: 445152 bytes
  hash: 20 bytes
    9f b5 96 9b a8 a3 9a 1c af 05
    ae 65 80 55 7a 75 29 e1 9a 3a
  signature: 40 bytes
    17 dd 62 51 97 20 8e be e8 b8
    05 16 b8 5d 3d df 46 54 a3 8c
    74 93 9d e9 5a 35 71 22 ef fe
    43 0f 95 e7 44 9e c4 2f 39 67
Verified : FALSE
I can reproduce the problem with my own build, if I create a .tar.gz package
from my own build, extract the distribution package, and try to run that.
Something must be missing. Investigating what is missing.
I'm giving updates as I have them.

SECMOD_CanDeleteInternalModule() returns "1"

but

SECMOD_DeleteInternalModule("NSS Internal PKCS #11 Module") 
returns -1
Ok, found the problem, at least on Linux.
Since the 1.4 development phase, NSS is now checking a checksum on its nssckbi
shared library.

The checksum gets created on the original version of the library, but the
distribution package contains a "stripped" version (from debugging symbols) of
that library, and is therefore different.

I seem to remember we already discussed this in the past already. Either we have
not fixed the problem at all yet, or we made a mistake. I'm trying to find the
other bug.

I'll also check whether the same thing happens on Windows.
This is a known problem.  There is already a report of
this bug in Bugzilla.  The Windows build has the same
problem but may have a different cause.

The fix is to regenerate the softokn3 checksum file
whenever the softokn3 shared library/DLL is post-processed
in any way (for example, if it is stripped).  The
shlibsign tool should be used to regenerate the checksum
file.
*** Bug 200757 has been marked as a duplicate of this bug. ***
Asa, I think we need more help, maybe you can help us find the right people.

On all platforms, we have now a strict dependency between shared library file
*softokn3* and its related checksum/signature file *.chk

It seems common that our packaging processes are doing post processing of shared
libraries.
Any post processing makes the checksum file to become incorrect, and the
software will refrain to work.

It's very likely that on Linux, the only post processing is the "strip" tool
that removes debugging information from the library.

However, I think on Windows there is no such strip tool. Because the same
failure has been reported on Windows, too, there might be some other post
processing that breaks it - Wan-Teh suspects possibly talkbalk could cause that?


What we have to do is:
- investigate all our different packaging procedures
- add an additional step to the packaging procedure, that happens
  after any shared library post processing,
  executing a command line like (Linux example):
    shlibsign -v -i libsoftokn3.so
  that produces an updated chk file

Do you know who knows our packaging processes well enough to be able to help?

I might be able to help, too, if we find people that can point me to the right
places where the final signature generation should happen.
leaf, bryner, cls, can one of you all help us with this blocker?
Discussed with Bryner and Leaf, and Leaf is now working on a patch.
working on a fix, need to update my tree and test. ETA: sometime this afternoon.
adding sgehani, we need this in 1.4f
Whiteboard: adt1
assigning to me to corral the necessary build/installer/packaging fixes for each
platform in.

I've tested my windows patch on a release system, the binaries can be found in:
http://ftp.mozilla.org/pub/mozilla/nightly/2003-05-16-08-trunk/

Might need sgehani's fix for the linux installer bits. 
Assignee: kaie → leaf
Please attach your patches and Bob Relyea or I will
review them.  Thank you.
Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

This patch looks good assuming all the NSS shared libraries are in $(DIST)/bin
or in the path (seems like a reasonable assumption since softokn3.dll is as
well).

It will be painfull obvious if they are not (the build would fail).
Attachment #123551 - Flags: review+
leaf, are there also changes needed to xpinstall/packager to make sure we sign
the NSS libraries after doing stripping?
Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

r=wtc.

Same comments as Relyea's.

You need to make sure that $(DIST)/bin/softokn3.dll and
$(DIST)/bin/softokn3.chk are the copies of those files
that you will include in the installer, because NSS itself
generates those files under $(DIST)/lib.
bryner: if there's a strip pass in the windows installer creation, then yes, one
will need to be added there. that *should* take care of unix installer builds as
well, yes?

relyea: the build target will fail, we'll have to add something to the windows
build automation that tests the make return status (currently, we don't check
it, so the signing can fail, and the rest of the release packaging would carry
on; we should at least warn about it).

wtc: thanks, i'll get this checked in.

I'm not likely going to be able to work on this (the unix installer, and windows
installer if there's a strip pass after the splitsym run has happened) this
weekend, and if i can't and/or it doesn't get fixed by monday, it will be top
priority then.
Status: NEW → ASSIGNED
Yeah, I mainly meant the strip pass that's done for unix.
I verified that I can enable FIPS mode with Leaf's win32
build (2003051608) in comment 19.
Comment on attachment 123551 [details] [diff] [review]
windows build system change for signing during a release build

a=asa (on behalf of drivers) for checkin to 1.4
Attachment #123551 - Flags: approval1.4+
I was able to enable FIPS mode in 2003051708 on WinXP SP1 Home.  However, if
FIPS mode is disabled, the Security Devices Manager needs to be reopened before
it can be reenabled.  Is that by design or a bug?
> I was able to enable FIPS mode in 2003051708 on WinXP SP1 Home.  However, if
> FIPS mode is disabled, the Security Devices Manager needs to be reopened before
> it can be reenabled.  Is that by design or a bug?

That's a consequence of the design. Before we can change again, all open handles
must get closed, however, the open window still has handles allocated in the
JavaScript context. So, after the cleanup on Windows close, it works again.
Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

looking for review. i know this is a hack, but i don't have time to implement
and test a better fix before 1.4 final ships.
Attachment #123733 - Flags: superreview?(dveditz)
Attachment #123733 - Flags: review?(sgehani)
Attachment #123733 - Flags: approval1.4?
same as the last patch, but with "-v -i" to the shlibsign call.
Attachment #123733 - Attachment is obsolete: true
Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

This patch is probably no good.  The most serious problem is
that the hardcoded .so suffix means it won't work on HP-UX.

I suggest that we simply not strip the softokn3 shared library.
If I understand Mozilla's build system correctly, stripping is
done here:
http://lxr.mozilla.org/seamonkey/source/xpinstall/packager/Makefile.in#184

My suggested workaround is to add this line:

    ! -name "*softokn3.*" \

to the 'find' command.
Attachment #123733 - Attachment is obsolete: false
What about Unix packages that don't get created with the deliver.pl script?
Will the .tar.gz packages work, too?
Comment on attachment 123734 [details] [diff] [review]
er, use the right options to shlibsign

Another problem with this patch is that you are not
setting the platform-dependent shared library search
path environment variable (which is LD_LBRARY_PATH on
most Unix variants) to point to the NSS and NSPR shared
libraries required by 'shlibsign'.  If it works on Linux,
it's because the NSS and NSPR shared libraries are
available in /usr/lib in recent Red Hat Linux releases.
Attachment #123734 - Flags: review-
wtc: what should we set LD_LIBRARY_PATH to (relative to mozilla/) for shlibsign
to work?
Attachment #123734 - Attachment is obsolete: true
Attachment #123733 - Attachment is obsolete: true
Comment on attachment 123733 [details] [diff] [review]
after we make the zip files, but before we make the installers, strip and sign libsoftokn3.so

clearing sr request until there's a reviewed patch. the nss guys don't seem to
like this one.
Attachment #123733 - Flags: superreview?(dveditz)
Attachment #123733 - Flags: review?(sgehani)
Attachment #123733 - Flags: approval1.4?
Leaf, see http://lxr.mozilla.org/seamonkey/source/security/nss/cmd/shlibsign/sign.sh
for an example of how LD_LIBRARY_PATH should be set.  Note that the case
for OpenVMS is very complicated and may require a shell script wrapper.
Also note that OS/2 uses the file
http://lxr.mozilla.org/seamonkey/source/security/nss/cmd/shlibsign/sign.cmd
instead, but if Mozilla copies the DLLs to $(DIST)/bin, we may not need
to use 'sign.cmd' on OS/2 (just like Windows).
this patch re-signs the lib once it's in the package-creation staging area and
has been stripped. The signature should remain valid if the lib is stripped
again.

Again, this is somewhat of a hack just to get us through 1.4, a better fix
incorporates some kind of an export "need to be signed" list from nss so we
don't have to worry about updating the version number when NSS revs.
Attachment #123809 - Flags: superreview?(dveditz)
Attachment #123809 - Flags: review?(wtc)
Attachment #123809 - Flags: approval1.4+
Attachment #123809 - Flags: approval1.4?
Comment on attachment 123809 [details] [diff] [review]
put nss signing step in xpinstall/packager/Makefile.in

Dunno how you got two of everything... clearing extras (hope the multi-submit
bug hasn't come back)

sr=dveditz if it's OK with wtc
Attachment #123809 - Flags: approval1.4?
Attachment #123809 - Flags: superreview?(dveditz) → superreview+
Attachment #123809 - Flags: approval1.4+ → approval1.4?
After talking with wtc, i think i fixed the way the ld_library_paths are
referenced, and added a couple of other files that might need to get signed on
other unix platforms.
Attachment #123809 - Attachment is obsolete: true
Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

seeking review. i hope i can get this in before tomorrow.
Attachment #123830 - Flags: superreview?(dveditz)
Attachment #123830 - Flags: review?(wtc)
Attachment #123830 - Flags: approval1.4?
Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

r=wtc.	Make sure you test this on a non-Linux machine.

Some suggested changes.

1. Use line continuation (\) for the long SIGN_ENV line.

2. "cd $(DIST)/$(MOZ_PKG_APPNAME)" and the "$(DIST)/$(MOZ_PKG_APPNAME)"
in SOFTOKN, FREEBL_PURE, and FREEBL_HYBRID are redundant.  Use one.
If you choose the cd command, you only need to cd once.

3. $(SOFTOKN) exists on all platforms, so you can omit
the existence test for it.
Attachment #123830 - Flags: review?(wtc) → review+
suggested changes made, tested on linux. trying to find another machine to test on.
Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

sr=dveditz, a=dveditz
Attachment #123830 - Flags: superreview?(dveditz)
Attachment #123830 - Flags: superreview+
Attachment #123830 - Flags: approval1.4?
Attachment #123830 - Flags: approval1.4+
fix with wtc's suggestions implemented, tested on solaris 2.6 and linux, checked in.
Status: ASSIGNED → RESOLVED
Closed: 18 years ago
Resolution: --- → FIXED
Attachment #123809 - Flags: approval1.4?
Comment on attachment 123830 [details] [diff] [review]
nss signing step in xpinstall/packager/Makefile.in

Leaf, could you delete the redundant "cd $(DIST)/$(MOZ_PKG_APPNAME) &&"
in SIGN_CMD?  Thanks.
Leaf, this is the change I was talking about.
*** Bug 205183 has been marked as a duplicate of this bug. ***
*** Bug 200215 has been marked as a duplicate of this bug. ***
Attachment #123809 - Flags: review?(wchang0222)
Product: PSM → Core
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.