504976 - stack limit calculated incorrectly when stack size is unlimited

Status: VERIFIED FIXED

(Tamarin Graveyard :: Virtual Machine, defect, P2)

Description

[Chris Dearman]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009060309 Ubuntu/9.04 (jaunty) Firefox/3.0.11
Build Identifier: 

UnixPlatform::getMainThreadStackLimit uses getrlimit to determine the available stack size. If the stack size is set to unlimited, r.rlim_cur is set to RLIM_INFINITY (0xffffffff) which is then used in the stack limit calculation generating an incorrect result.


Reproducible: Always

Steps to Reproduce:
This problem shows up when invoking avmshell from a makefile, but it is easier to do it by hand:

1. Remove limit on stack size ("ulimit -s none" in bash, "unlimit stack" in tcsh). You may need to be running as a priviliged user to do this.
2. Run "avmshell -repl"

Actual Results:  
root@vm-chris:/opt/tamarin/tamarin-redux# ulimit -s unlimited
# ulimit -s
unlimited
# objdir-x86/shell/avmshell -repl
Segmentation fault


Expected Results:  
# ulimit -s 8192
# ulimit -s
8192
# objdir-x86/shell/avmshell -repl
avmplus interactive shell
Type '?' for help

>



The same code is also used in MacPlatform::getMainThreadStackLimit()

Comment 1

[Chris Dearman]

Attachment: Cope with getrlimit reporting unlimited stack size

I'm not sure this is the best fix as the kStackSizeFallbackValue will be used as the available stack size even though the system considers the stacksize to be unlimited.

I did try setting stackheight to stackbase but that causes 2 of the acceptance tests to fail, and there isn't an obvious intermediate value to use. The tests that failed were:
as3/Definitions/FunctionAccessors/InfRec.as
ecma3/Exceptions/bug127913.as

I think a similar patch is also needed for shell/avmshellMac.cpp

Comment 2

[Lars T Hansen]

Attachment: Unix and MacOS both

Comment 3

[Lars T Hansen]

redux changeset:   2201:409cf685cd2a