PR_SetThreadPriority should cast enum PRThreadPriority to a signed int before doing comparisions

VERIFIED FIXED

Status

NSPR
NSPR
P3
normal
VERIFIED FIXED
20 years ago
20 years ago

People

(Reporter: Wan-Teh Chang, Assigned: Wan-Teh Chang)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

20 years ago
This bug is discovered by Jeremy Lea <reg@shale.csir.co.za>
when running the 'priotest' test.

In PR_SetThreadPriority, we compare the new priority with
PR_PRIORITY_FIRST and PR_PRIORITY_LAST to make sure that it
is in the legal range of the enum PRThreadPriority.

Since all the legal values of the enum PRThreadPriority
are 0 or positive numbers, the enum is implemented as an
unsigned int under some compilers, such as gcc and egcs.
So the value of -1 for the enum is treated as 0xffffffff
and is greater than 0, which is absurd.

We can force the compiler to do a signed comparison
by casting the enum to PRIntn (a synonym for int) before
doing the comparisons.

PR_SetThreadPriority in the pthreads version (ptthread.c)
already has the PRIntn casts.  We just need to fix
PR_SetThreadPriority in classic NSPR (pruthr.c).
(Assignee)

Updated

20 years ago
Status: NEW → RESOLVED
Last Resolved: 20 years ago
Resolution: --- → FIXED
(Assignee)

Comment 1

20 years ago
To reproduce the bug, run the 'priotest' test
with the -d option on FreeBSD 3.1, classic NSPR.
You will get:
% priotest -d
PR_SetThreadPriority(-1) failed

Adding the PRIntn casts fixes the problem.
This fix is checked into the tip.
/cvsroot/mozilla/nsprpub/pr/src/threads/combined/pruthr.c, revision 3.8.

Also checked into the internal CVS repository.
/m/src/ns/nspr20/pr/src/threads/combined/pruthr.c, revision 2.25.
(Assignee)

Comment 2

20 years ago
Jeremy, can you verify my fix please?
Then choose "Mark bug as VERIFIED"
and commit.  Thanks.

Updated

20 years ago
Status: RESOLVED → VERIFIED

Comment 3

20 years ago
NSPR now has its own Bugzilla product.  Moving this bug to the NSPR product.
You need to log in before you can comment on or make changes to this bug.