522462 - 64-bit libfreebl wants executable stack on SELinux

Status: RESOLVED DUPLICATE of bug 486537

(NSS :: Libraries, defect)

Description

[peter_jonson]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.19) Gecko/20081216 Fedora/2.0.0.19-1.fc8 Firefox/2.0.0.19
Build Identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3

When compiling, I get 'SELinux is preventing shlibsign from making the program stack executable. The shlibsign application attempted to make its stack executable. This is a potential security problem. This should never ever be necessary.'.

When running the main linux build (from http://www.mozilla.com/en-US/firefox/upgrade.html?from=getfirefox), on startup it says 'SELinux is preventing firefox-bin from changing a writable memory segment executable. The firefox-bin application attempted to change the access protection of memory (e.g., allocated using malloc). This is a potential security problem. Applications should not be doing this.'.

Reproducible: Always

Comment 1

[Johnathan Nightingale [:johnath]]

There may be two bugs here (one in shlibsign, one in firefox code) or zero bugs here if SELinux is just confused, but let's track down the shlibsign one first and see where that takes us?

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

I'm sure that no code in NSS makes any system calls with the intention of
changing stack or heap memory to be executable. Off hand, I don't even know
what system call exists for this purpose. Possible explanations for this 
warning (error?) that occur to me include:

1) NSS makes some system call that has this effect has an unintended 
consequence (side effect).  What call might that be?

2) NSS calls some library function in some non-NSS library that, in turn,
makes some system call that has this effect.  

3) Some shared library in NSS has a component that is unintentionally marked 
as requiring an executable stack (is there a similar flag that marks a 
library as requiring an executable heap?) and when that library is 
dynamically loaded, this warning is issued.  

4) This is a false warning/error.

Is there any way to get a stack trace of the system call that triggers this 
warning?

Comment 3

[peter_jonson]

The relevant part of the compile process is (in effect):
cd mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ
sh <path>/mozilla-1.9.1/security/nss/cmd/shlibsign/./sign.sh <path>/mozilla-1.9.1/dist \
    <path>/mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ Linux \
    <path>/mozilla-1.9.1/dist/lib <path>/mozilla-1.9.1/dist/lib/libsoftokn3.so

I added ldd to the sign.sh script, and got:
        linux-vdso.so.1 =>  (0x00007fff353ff000)
        libplc4.so => <path>/mozilla-1.9.1/dist/lib/libplc4.so (0x0000000000110000)
        libplds4.so => <path>/mozilla-1.9.1/dist/lib/libplds4.so (0x0000000000606000)
        libnspr4.so => <path>/mozilla-1.9.1/dist/lib/libnspr4.so (0x0000000000809000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000039be000000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000039bdc00000)
        libc.so.6 => /lib64/libc.so.6 (0x00000039bd400000)
        /lib64/ld-linux-x86-64.so.2 (0x00000039bc200000)
<path>/mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/shlibsign -v -i <path>/mozilla-1.9.1/dist/lib/libsoftokn3.so
moduleSpec configdir='' certPrefix='' keyPrefix='' secmod='' flags=noCertDB, noModDB
C_Initialize failed: 0x00000030, CKR_DEVICE_ERROR
NSPR error code: -5977: Failure to load dynamic library
Initiailzing softoken failed: 0x00000030, CKR_DEVICE_ERROR
NSPR error code: -5977: Failure to load dynamic library

At the same time, this log message appears in the system log:'setroubleshoot: SELinux is preventing shlibsign from making the program stack executable.'

If I do 'setenforce 0' and repeat, it seems to work.

If I just do '<path>/mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/shlibsign -v -i <path>/mozilla-1.9.1/dist/lib/libsoftokn3.so' directly, it seems to work:
moduleSpec configdir='' certPrefix='' keyPrefix='' secmod='' flags=noCertDB, noModDB
Generate a DSA key pair ...
Library File: ../../security/nss/lib/freebl/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/libfreebl3.so 427192 bytes
Check File: ../../security/nss/lib/freebl/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/libfreebl3.chk
Link: libfreebl3.chk
  hash: 20 bytes
    (hex dump)
  signature: 40 bytes
	(hex dump)

'ldd <path>/mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/shlibsign' produces:
        linux-vdso.so.1 =>  (0x00007fffc57ff000)
        libplc4.so => /usr/lib64/libplc4.so (0x0000003bcb400000)
        libplds4.so => /usr/lib64/libplds4.so (0x0000003bcb000000)
        libnspr4.so => /usr/lib64/libnspr4.so (0x0000003bcac00000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000039be000000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000039bdc00000)
        libc.so.6 => /lib64/libc.so.6 (0x00000039bd400000)
        /lib64/ld-linux-x86-64.so.2 (0x00000039bc200000)

I tried modifying sign.sh to call 'gdb -e "${2}/shlibsign"'. Then I had to type 'run' manually:
GNU gdb Red Hat Linux (6.6-45.fc8rh)
Copyright...
This GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) run -v -i <path>/mozilla-1.9.1/dist/lib/libsoftokn3.so
Starting program: <path>/mozilla-1.9.1/security/nss/cmd/shlibsign/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/shlibsign -v -i <path>/mozilla-1.9.1/dist/lib/libsoftokn3.so
Using host libthread_db library "/lib64/libthread_db.so.1".
[Thread debugging using libthread_db enabled]
[New Thread 140468822320880 (LWP 8320)]
moduleSpec configdir='' certPrefix='' keyPrefix='' secmod='' flags=noCertDB, noModDB
C_Initialize failed: 0x00000030, CKR_DEVICE_ERROR
NSPR error code: -5977: Failure to load dynamic library
Initiailzing softoken failed: 0x00000030, CKR_DEVICE_ERROR
NSPR error code: -5977: Failure to load dynamic library

Program exited normally.
(gdb) bt
No stack.

Comment 4

[Nelson Bolyard (seldom reads bugmail)]

Comment 0 said "The firefox-bin application attempted to change the access
protection of memory (e.g., allocated using *malloc*)."  Memory allocated 
using malloc is not stack memory.  It is heap memory.  Comment 0 is a 
complaint about making the HEAP executable.  But comment 3 clearly says:
SELinux is preventing shlibsign from making the program *stack* executable.

Now, we've run into problems before where gcc/gas wants to mark .o files built from assembly code as saying that they require an executable stack, and we 
have to take steps to prevent that.  I think it's likely that some .o file 
that is a component of libfreebl3.so is marked in that way.  Perhaps you can
search through your build and find it for us.  That should be easy to fix, 
once we find it (although I expect there will be FIPS boundary issues :(.

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

(oops, hit commit when I meant to scroll down)
... but I have no idea about executable heap.  I suspect that the warning 
about executable heap was mistaken, and that if/when we find and correct an
executable stack problem, the issues will just vanish.  But if there is also
really an executable heap issue, I think you will have to find it via strace.

Comment 6

[Nelson Bolyard (seldom reads bugmail)]

Oh, please report the exact version of NSS you're using.

Comment 7

[peter_jonson]

I ran mozilla-1.9.1/security/nss/cmd/shlibsign/./sign.sh modified to call shlibsign with strace, and part of the output was:
open("<path>/mozilla-1.9.1/dist/lib/libfreebl3.so", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`8\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=427192, ...}) = 0
mmap(NULL, 2508288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7964000
mprotect(0x79c3000, 2097152, PROT_NONE) = 0
mmap(0x7bc3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5f000) = 0x7bc3000
mmap(0x7bc5000, 13824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7bc5000
mprotect(0x39bc41a000, 3492, PROT_READ|PROT_WRITE) = 0
mprotect(0x39bc41a000, 3492, PROT_READ) = 0
mprotect(0x7fff4932d000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC|PROT_GROWSDOWN) = -1 EACCES (Permission denied)
close(3)                                = 0
readlink("<path>/mozilla-1.9.1/dist/lib/libsoftokn3.so", "../../security/nss/lib/softoken/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/libsoftokn3.so", 1023) = 83
readlink("../../security/nss/lib/softoken/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/libsoftokn3.so", 0x1ac7e00, 1023) = -1 ENOENT (No such file or directory)
open("../../security/nss/lib/softoken/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/libfreebl3.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("<path>/mozilla-1.9.1/dist/lib/libfreebl3.so", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`8\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=427192, ...}) = 0
mmap(NULL, 2508288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x5c9f000
mprotect(0x5cfe000, 2097152, PROT_NONE) = 0
mmap(0x5efe000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5f000) = 0x5efe000
mmap(0x5f00000, 13824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x5f00000
mprotect(0x39bc41a000, 3492, PROT_READ|PROT_WRITE) = 0
mprotect(0x39bc41a000, 3492, PROT_READ) = 0
mprotect(0x7fff4932d000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC|PROT_GROWSDOWN) = -1 EACCES (Permission denied)
close(3)                                = 0
write(2, "C_Initialize failed: 0x00000030,"..., 70C_Initialize failed: 0x00000030, CKR_DEVICE_ERROR) = 70
write(2, "NSPR error code: -5977: Failure "..., 56NSPR error code: -5977: Failure to load dynamic library) = 56
write(2, "Initiailzing softoken failed: 0x"..., 79Initiailzing softoken failed: 0x00000030, CKR_DEVICE_ERROR) = 79
write(2, "NSPR error code: -5977: Failure "..., 56NSPR error code: -5977: Failure to load dynamic library) = 56


I don't know how to test .o files to see if they require executable stack.


I modified run-mozilla.sh to call firefox-bin using strace, and during startup at about the time of the selinux error, I got:
mmap2(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EACCES (Permission denied)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EACCES (Permission denied)
mmap2(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EACCES (Permission denied)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EACCES (Permission denied)
open("/dev/urandom", O_RDONLY)          = 19
read(19, "\37%C}", 4)                   = 4
gettimeofday({1256031218, 209564}, NULL) = 0
stat64(0xf79776c8, 0xf7bf570c)          = 0
open("/usr/local/firefox/components/browser.xpt", O_RDONLY|O_LARGEFILE) = 20
read(20, "XPCOM\nTypeLib\r\n\32\1\2\5\266\0\5\204\34\0\0\0\"\0\0\240\t"..., 361500) = 361500
mmap2(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffffffff5ffe000
munmap(0xf5ffe000, 1048576)             = 0


The version of nss installed is 3.12.2.0.

Comment 8

[Nelson Bolyard (seldom reads bugmail)]

The strace output shows us the Linux dlopen library function attempting to 
load libfreebl3.so, and then trying to give some part of it execute access,
probably the stack.    So, the next step is to find the part of libfreebl3.so
that wants an executable stack.  I'm not sure of the exact command to use to
query a file for that attribute, but I'll bet it's one of the many elfdump 
options.

Comment 9

[Nelson Bolyard (seldom reads bugmail)]

suggestion:
cd into the directory where the .o files are built for freebl, and try 
a shell command like

for a in *.a *.o; do echo === $a ==; elfdump $a |grep 'sh_.*EXEC';done >foo.txt

then if it's not too big, attach foo.txt to this bug as an attachment.  thanks.

Comment 10

[peter_jonson]

'elfdump' is not included in Fedora, and it is not clear where is a sensible place to get it. I tried compiling version 0.3, but got:
gcc -c symbols.c -o symbols.o -Wall -O2
symbols.c: In function ‘display_symbols’:
symbols.c:55: error: invalid lvalue in assignment

Fedora does include 'objdump' and 'readelf', but I don't know how to use them to get the information you want.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

See http://bashcurescancer.com/man/cmd/execstack for the man page on the 
execstack command, which can query a shared library to see if it needs 
an executable stack, and can also mark it as needing, or NOT neeting, an
executable stack.  

I'd like you to query all your NSS and NSPR shared libraries with that 
tool, and let us know here what it says about their needs for executable 
stacks.  Then, use it to mark all the NSS shared libraries as NOT needing
executable stacks, and try again.  
  e.g. 
    execstack -q *3.so
    execstack -c *3.so


I'm told that others who've built NSS on SELinux have gotten binaries that 
execstack says do NOT require executable stacks.  We don't know why the 
files you built would be different than theirs.

Comment 12

[peter_jonson]

The only .so files I've got that with that flag set are:
X dist/lib/libfreebl3.so
X security/nss/lib/freebl/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/libfreebl3.so

The first is just a link to the second.

I then did 'execstack -c security/nss/lib/freebl/Linux2.6_x86_64_glibc_PTH_64_OPT.OBJ/Linux_SINGLE_SHLIB/libfreebl3.so' and it fixed the problem.

It's still not clear what caused it, as execstack only works for '.so' files, not '.o'.

Comment 13

[Nelson Bolyard (seldom reads bugmail)]

Several, but not all, of the .s (assembly language) files in libfreebl
have two lines of code on the end that read as follows:

# Magic indicating no need for an executable stack
.section .note.GNU-stack, "", @progbits

I'd like you to conduct an experiment.  I'd like you first determine 
exactly which .s files are being built into .o files in your builds, then
do a series of repeat builds of NSS, adding those two magic lines to those
.s files that get built on your system that do not already have those lines.
I'd like you to do them one at a time.  Add those lines to one .s file, 
then rebuild libfreebl entirely from scratch, then retest (or just rerun 
execstack to see if the flag has been set or not).  Eventually you will 
find exactly which .s file or files need this on your system.  Please tell us
what you find.

Comment 14

[peter_jonson]

I don't know how to compile NSS by itself. I tried
cd security/nss
make
but I got errors.

Comment 15

[Nelson Bolyard (seldom reads bugmail)]

Peter, there's something strange about your system. 
I've had other people confirm that they can build and run NSS on SELinux 
(Fedora) without any problems.  They've tried to reproduce the problem you
see, but it works OK for them.  So, something about your system is unusual.
Maybe it's your version of gcc, or the exact set of revisions of Linux 
you're using.  I dunno.  

We could spend a lot of time trying to figure out all the specific details,
but I think you've got a solution now, with the execstack program, and your
problem is solved.  Agreed?

Comment 16

[peter_jonson]

Yes, thanks, I've got a workaround for my problem with building. It's strange, I don't know what's causing it. It's unfortunate I don't know how to use any mainstream tools to check the executable memory flag in the .o files.

The problem of the pre-built distribution changing a writable memory
segment executable is still outstanding. Of course it is an x86 version.

Comment 17

[Wan-Teh Chang]

This is a known bug, introduced in NSS 3.12.3 (the Intel AES
assembly code) and fixed in NSS 3.12.4.

All Linux distributions apply the patch in bug 486537 to their
NSS 3.12.3.1 packages.