243305 - Memory leak with compiled scripts

Status: RESOLVED INVALID

(Rhino Graveyard :: Compiler, defect)

Description

[Morten Moeller]

We've had problems with JavaScript out in the field where seemingly random 
javascripts fails with NPE in the constructor.

java.lang.ExceptionInInitializerError

java.lang.ExceptionInInitializerError: 

java.lang.NullPointerException

            at org.mozilla.javascript.gen.c225020.<clinit>(Script 408354)

…

If anyone has any idea to help us debug this. It only happens in very rare 
occations (once every 100.000 compile maybe).

Comment 1

[Igor Bukanov]

Could you try Rhino 1.5R5? Also what is your JVM?

Comment 2

[Morten Moeller]

We're only seeing this at a deployed customer site, and I'm not able to repro 
locally. I'm trying to get them to send me the exact script that is causing 
it..

The JVM is IBM JDK 1.3.1 running under a weblogic application server.

Updating the Rhino engine isn't possible as this happens on a huge production 
system, and changing engine sometimes changes behavior, so it would take 
months. But I'll try to get more info about this shortly.

Comment 3

[Morten Moeller]

Attachment: Failing Javascript

This is the script that normally breaks. Note that it doesn't always fail just
sometimes. 

It does use a lot of functions and I see the function compile is improved for
R5, so that might be of help. 

But if anyone have any idea why there can be a NPE in the class constructor,
that would be really great help :)

Comment 4

[Igor Bukanov]

(In reply to comment #3)
> Created an attachment (id=150713)
> Failing Javascript
> 
> This is the script that normally breaks. Note that it doesn't always fail just
> sometimes. 
> 
> It does use a lot of functions and I see the function compile is improved for
> R5, so that might be of help. 
> 
> But if anyone have any idea why there can be a NPE in the class constructor,
> that would be really great help :)

Rhino 1.5R5 contains a few fixes for code generation including fixes to race
conditions that I think could lead to NPE. Since some fixes were in fact
significant chnages to code generation, it would not make sence to backport them
since it will only produce just another untested Rhino version. 

If upgrade is not possible, you may try to work arround them in your Java code
that uses Rhino. But that is non-trivial and requires details about your Rhino
embedding.
> 

Comment 5

[Morten Moeller]

Well after them investigating more it seems like we found the actual issue. 

They were doing an eval() inside a for loop creating a -lot- of classes. Now 
they could in this case rewrite around the eval() it wasn't needed, so I think 
our issue is gone.

Will R5 be more forgiving about eval() and creating classes in general? One of 
our biggest issues with Rhino/optimization has been that it creates a lot of 
classes that tends, at least on JDK older than 1.4, to cause the PermMem to 
fill up rapidly (and never really clean up) causing random crashes and OOM 
exceptions. This seems to be a jdk issue and not how Rhino does it though 
(same problems occurs in weblogic with a lot of jsp reloads for example), but 
I was just wondering if the "new way" of doing things in R5+ might help on 
that issue.

Anyways, I guess there is no fix for this eval() issue, so feel free to close 
this issue unless anyone got a solution around it ;)

Comment 6

[Igor Bukanov]

Rhino uses a pure interpretation mode to implement JavaScript eval and no
additional classes should be generated for the usage you described. 

On the other hand if Java code repeatedly recompiles the same script then using
R5 may help as it generates one class per script compared with previous releases
that produced additional class for each and every function. But if JVM does not
really collect generated classes and AFAIK IBM JDK 1.3.1 has this bug, then you
better stick with interpretation mode for everything. 

The bug would also be visible if the code repeatedly initialize Rhino runtime
and define additional host objects that extend ScriptableObject for each
Context.initStandardObjects call. In this case Rhino also generates additional
helper classes to avoid reflection overhead.

Even if JVM does not have the bug, I would recommend to use the interpretation
mode in a server environment unless you use 1.5R5. 

Comment 7

[Morten Moeller]

Well in production systems we cache all compiled scripts (very carefully) which
is why this is the only customer ever seeing this problem. If we don't cache,
servers aren't up more than maybe 30 minutes :). In production these scripts
never really changes (and if they do, normally a reboot is done).


However, before we started cacheing, I tried at some point to turn off
optimization mode (off hand I think it was context.setOptimizationLevel(-1) or
something similar), but I swear this still generated classes and therefore
didn't help us. I found out why at some point I'll go back and look and see if I
find it.

Comment 8

[Igor Bukanov]

(In reply to comment #7)
> However, before we started cacheing, I tried at some point to turn off
> optimization mode (off hand I think it was context.setOptimizationLevel(-1) or
> something similar), but I swear this still generated classes and therefore
> didn't help us. I found out why at some point I'll go back and look and see if I
> find it.

Do you use JavaAdapter in your scripts to implement/extend Java interfaces and
classes? If so then it should also lead to class generation. If not then with
context.setOptimizationLevel(-1) turned on Rhino should never generate any classes. 

You may try to remove org/mozilla/classfile and org/mozilla/javascript/optimizer
firectories from your Rhino jar to remove class generation code for sure and see
if the memory leak is caused by classes or not. 

Comment 9

[Igor Bukanov]

Attachment: Test case to check for memory leaks in JVM related to class loading

To check if JVM leaks memory, compile and run the above class like in:

javac -classpath js.jar Test.java
java -classpath js.jar:. Test

Under Sun JDK 1.4.2 for Linux it eventually run out of memory.

Comment 10

[Igor Bukanov]

Marking as invalid: this is JVM bug with well defined work around which is using
interpreted mode.