Closed Bug 4464 Opened 25 years ago Closed 25 years ago

PR_SetThreadPriority should cast enum PRThreadPriority to a signed int before doing comparisions

Categories

(NSPR :: NSPR, defect, P3)

Product:
NSPR
Component:
NSPR
Platform:
All
FreeBSD
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: wtc, Assigned: wtc)

Details

This bug is discovered by Jeremy Lea <reg@shale.csir.co.za>
when running the 'priotest' test.

In PR_SetThreadPriority, we compare the new priority with
PR_PRIORITY_FIRST and PR_PRIORITY_LAST to make sure that it
is in the legal range of the enum PRThreadPriority.

Since all the legal values of the enum PRThreadPriority
are 0 or positive numbers, the enum is implemented as an
unsigned int under some compilers, such as gcc and egcs.
So the value of -1 for the enum is treated as 0xffffffff
and is greater than 0, which is absurd.

We can force the compiler to do a signed comparison
by casting the enum to PRIntn (a synonym for int) before
doing the comparisons.

PR_SetThreadPriority in the pthreads version (ptthread.c)
already has the PRIntn casts.  We just need to fix
PR_SetThreadPriority in classic NSPR (pruthr.c).
Status: NEW → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
To reproduce the bug, run the 'priotest' test
with the -d option on FreeBSD 3.1, classic NSPR.
You will get:
% priotest -d
PR_SetThreadPriority(-1) failed

Adding the PRIntn casts fixes the problem.
This fix is checked into the tip.
/cvsroot/mozilla/nsprpub/pr/src/threads/combined/pruthr.c, revision 3.8.

Also checked into the internal CVS repository.
/m/src/ns/nspr20/pr/src/threads/combined/pruthr.c, revision 2.25.
Jeremy, can you verify my fix please?
Then choose "Mark bug as VERIFIED"
and commit.  Thanks.
Status: RESOLVED → VERIFIED
NSPR now has its own Bugzilla product.  Moving this bug to the NSPR product.
You need to log in before you can comment on or make changes to this bug.