856969 - Too much recursion error with Java LiveConnect Applet

Status: RESOLVED WONTFIX

(Core Graveyard :: Plug-ins, enhancement, P5)

Description

[Richard Maher]

Attachment: simple.html

User Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET4.0C; Tablet PC 2.0)

Steps to reproduce:

FireFox 19.0.2
Windows 7 Ultimate 32 bit SP1 
Java 7.0.17

I don't currently have a public facing web-site but the problem is extremely easy to reproduce locally.

1. Compile attached Java source file
javac -cp ".;C:\Program Files\Java\jre7\lib\plugin.jar" Tier3TABiPlexer.java

2. Build the Applet JAR file
jar -cf poc.jar Tier3TABiPlexer.class

Copy the "poc.jar" and "simple.html" files to your Web-Server's root directory

Open http://127.0.0.1/simple.html in FireFox and you should see two input fields and may be prompted to "Always allow Applets from this site"; say "yes". The Applet will load and it's a good idea to have the Java console on as well as "Web Console" (Tools Web Developer) and Error Console.


Actual results:

After ~86 iterations (of setTimer(myfunc) and not mfunc calling itself) the error "Too Much recursion" with be returned, normally to both the Web Console and Error Console. 

This only happens if I am running more than on "Tester" object in the Javascript. If I comment out Tester2 in the following code: -
        var tester1 = new tester(1,10);
 //       var tester2 = new tester(2,20);

Then no error occurs although if multiple tabs of simple.html are added then FF only allows two concurrent tabs to continue to run without error.

If I make the call to my Applet sen()/GetNum() method Asynchronous by adding another thread to perform the JSObject callback then it appears to work.

If I remove the 300ms delay (simulated workload) from the Applet then it appears to work. If the delay is varied then so too appears the number of iterations before fail.

Laundering the JSObject callback through a top level window function (function x(JSObject){ return JSObject;} had no effect.


Expected results:

The field values should have just continued to click over ad infinitum as is the case on other browsers: -

Windows Chrome 26.0
Windows Opera 12.14
OS/X 10.8.3 Safari 6.0.3 

Internet Explorer exhibits similar behaviour in that, after ~600 iterations, it returns "stack overflow". Although, with only one Tester JS object IE lets me add any number of concurrent tabs.

Please let me know if there is any other information that I can provide!

Cheers Richard Maher

Comment 1

[Richard Maher]

Attachment: Small Java Applet to reproduce the problem

Comment 2

[Richard Maher]

Attachment: Error Console snapshot

Comment 3

[Loic]

Is it possible to put online testcase without having the need to compile java file?
Or provide directly poc.jar, please.

Comment 4

[Richard Maher]

Attachment: The JAR file for the Applet as requested

Comment 5

[Loic]

I put simple.html and poc.jar in the same folder, but the Java applet doesn't load.
What's wrong?

Comment 6

[Richard Maher]

(In reply to Loic from comment #5)
> I put simple.html and poc.jar in the same folder, but the Java applet
> doesn't load.
> What's wrong?

Is it Windows and you've placed both files in wwwroot. Or the root of whatever web server you're using? Anything in the error console or Web console? Are you using FireFox 19? Do you have Java enabled? Do you have Java JRE 7+ installed?

You're not giving too much away :-)

Comment 7

[Loic]

FF19 on Win 7.
I downloaded and put both files simple.html and poc.jar in a folder in My Documents and I opened simple.html directly in Firefox 19.

Comment 8

[Richard Maher]

(In reply to Loic from comment #7)
> FF19 on Win 7.
> I downloaded and put both files simple.html and poc.jar in a folder in My
> Documents and I opened simple.html directly in Firefox 19.

Hi Loic,

Please read and follow the instructions if you wish to help. Not "My Documents" and not with a file URL. As was explicitly instructed.

Do you have anything to do with FF or are you just site seeing?

Cheers Richard Maher

Comment 9

[Matthias Versen [:Matti]]

Richard: 
Loic is one of the few people from the community who are doing bug triage and without people like him your bug will go nowhere.
FYI: Not everyone has a local http server installed....


I can see the too much recursion error in the error console with Seamonkey trunk on windows7

Comment 10

[(mostly gone) XtC4UaLL [:xtc4uall]]

Error: [Exception... "'InternalError: too much recursion' when calling method: [nsIObserver::observe]"  nsresult: "0x8057001c (NS_ERROR_XPC_JS_THREW_JS_OBJECT)"  location: "JS frame :: resource://gre/components/ConsoleAPI.js :: CA_notifyObservers :: line 330"  data: no]
Source File: resource://gre/components/ConsoleAPI.js
Line: 330

Error: too much recursion
Source File: resource://gre/modules/devtools/WebConsoleUtils.jsm
Line: 1455

Comment 11

[Loic]

Is it a regression? Or an old bug in Firefox?

Comment 12

[Richard Maher]

Hi Loic,

Please accept my apology for the rudeness in my earlier reply and thank you for your support.

As a further update, that seems to confirm a Windows only issue, is that FireFox 19.02 works just fine on OS/X. No errors and as many tabs as you like.

Thanks all for the updates and interest.

Cheers Richard Maher

Comment 13

[Richard Maher]

Hi Boris, What info dou yo need? Cheers Richard

Comment 14

[Boris Zbarsky [:bzbarsky]]

Richard, I'm asking Benjamin for any info he may have about why this is going on; presumably something to do with the way we do the calls into and out of the Java Plug-in on Windows...

Comment 15

[Richard Maher]

No worries Boris. I'm not used to the interface and thought you needed something from me.

Speaking of bugzilla, should this bug now be set to CONFIRMED? It's reproduceable at will and seems to only affect FireFox on Windows.

Also here's a couple of liks that might help those who are curious: -
http://docs.oracle.com/javase/6/docs/technotes/guides/jweb/applet/applet_execution.html
http://jdk6.java.net/plugin2/liveconnect/

Please let me know if there is anything else I can do.

Cheers Richard Maher

Comment 16

[Benjamin Smedberg]

I don't have much insight here without a stack. It is certainly possible to recursively call through plugins to content and back in a way that would eventually eat the stack, but quick inspection of the testcase doesn't make it clear whether that's what's happening here.

If somebody wants to debug, you should be able to set a breakpoint at js_ReportOverRecursed to record the stack at the time of the issue.

Comment 17

[Richard Maher]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #16)
> I don't have much insight here without a stack. It is certainly possible to
> recursively call through plugins to content and back in a way that would
> eventually eat the stack, but quick inspection of the testcase doesn't make
> it clear whether that's what's happening here.

Just to reiterate :-) My code works on Chrome, Safari, Opera and most tellingly the same version of FireFox on OS/X. It might just need different stimulus on those platforms but I respectfully submit that with all available symptoms taken into account we're talking about a FF bug on Windows here.
> 
> If somebody wants to debug, you should be able to set a breakpoint at
> js_ReportOverRecursed to record the stack at the time of the issue.

I know you people get literally thousands of bugs a day and many more glamorous or higher yeilding than this but I beg someone out there to pick this up and run with it because the functionality I wish to base on this "simple" platform is truly interesting and useful indeed.

Cheers Richard Maher

Comment 18

[Richard Maher]

FYI here is the related IE bug report: -
http://answers.microsoft.com/en-us/ie/forum/ie10-windows_7/out-of-stack-space-error-with-java-liveconnect/5b152c7f-4f47-4161-aa66-2921f2f9ba5d

It look like a few people are also interested over there.

Look, I don't if the two bugs on separate platforms are related but prima facie you could be forgiven for thinking that some low-level WinWidgetConsole.DLL is responsible for the faulty reporting of recursion/stack-issues.

I don't know if any FF contributors are in Western Australia or if I can somehow buy credits with random acts of triage here but I am happy to buy all beers reqd or lunch/dinner or anything else that involves leaving our clothes on to get this bug seen to!

Imagine (depending on your personal philosophy/point-of-view): -
1) Helping Microsoft discover a latent bug in Windows/IE and showing Mozilla once again to be kicking **** on the technology front.
2) Highlighting another failure with Windows that results in both IE and FF flondering on Windows 7.

Cheers Richard Maher

Comment 19

[John Schoenick [:johns]]

We run java in-process in windows, letting it use its own process proxying/isolation there. I wouldn't be surprised if java was spinning nested event loops while waiting for the call into the applet to finish, which calls back into JS, which calls back into java, which gets trapped in another nested event loop...

Does setting |dom.ipc.plugins.java.enabled| fix this problem? (Note that it will likely cause other problems)

Comment 20

[John Schoenick [:johns]]

Yeah:

http://icedtea.classpath.org/hg/release/icedtea-web-1.2/file/tip/plugin/icedteanp/IcedTeaJavaRequestProcessor.cc#l268

Java is basically capturing the event loop while it waits for a response from the applet. When the applet then calls into script, that calls into an applet, we move up to two levels of captured event loops and so on until the stack is exhausted. Based on this I would say that sleeping on the main thread of an applet is not advisable.

I don't see what we could do about this on our end. We don't really know ahead of time what events might try to call further into plugins :-/

Comment 21

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #19)

Thanks very much for the responses! You onbviously have far greater knowledge of the direct issues here but please bear with me as I attempt a cross-examination (Because I'm depsperate to get this code up and running!)

> We run java in-process in windows,

Do you run Java in separate process(es) on Mac OS? I'm just wondering why the same issue is not encountered there especially as Oracle is now writing the Java plugin across all platforms.

Also, individual browser ideosynchrassies and NPAPI notwithstanding, could this be why Chrome and Opera are immune on windows?

Didn't the New Java Plugin sort a lot of this out? For example, Round Trip scenarios: -
http://docs.oracle.com/javase/6/docs/technotes/guides/jweb/applet/applet_execution.html

> I wouldn't be surprised if java was spinning
> nested event loops while waiting for the call into the applet to finish,
> which calls back into JS, which calls back into java, 

But this does not happen in my example! The call "back into Java" is done from an event "setTimeout".

Also, would the stack issues then not occur if I was running a single Tester object? You'll recall from my bug description that the error does not occur if I comment out the "var tester2 = new tester(2,20)".

Furthermore why would multiple tabs in FireFox with a single Tester result in only 2 tabs being able to survive concurrently?

Comment 22

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #20)
> Yeah:
> 
> http://icedtea.classpath.org/hg/release/icedtea-web-1.2/file/tip/plugin/
> icedteanp/IcedTeaJavaRequestProcessor.cc#l268
> 
> Java is basically capturing the event loop while it waits for a response
> from the applet. 

Java is capturing the JS event loop while it's waiting for the Applet to return to JS?

> When the applet then calls into script, that calls into an
> applet, 

No it doesn't it sets a timeout that will call in from the EDT (or whatever it's called in JS) n'est-ce pas?

> we move up to two levels of captured event loops and so on until the
> stack is exhausted. Based on this I would say that sleeping on the main
> thread of an applet is not advisable.

Not sleeping, just waiting for one level of call to return from JS.
> 
> I don't see what we could do about this on our end. We don't really know
> ahead of time what events might try to call further into plugins :-/

Could this call.apply bug be related 
https://bugzilla.mozilla.org/show_bug.cgi?id=794427

I know is says Mac OS but who knows? Although my bug has been around long before 18 :-(

Cheers Richard Maher

Comment 23

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #19)
> 
> Does setting |dom.ipc.plugins.java.enabled| fix this problem? 

No.

Also FYI, the much more sophisticated functionality I am basing on this "simple" example, requires the historical and supported behaviour of the <PARAM name="classloader_cache" value="true"> parameter. 

BTW, if your OOPP capability (a la mode de LCIE) is meant to be used by all but the sick-of-life let me insist that you continue to enable this **** by choice! The having to dig around for TabProcGrowth < 2 in IE is a severe pain in the ****!

Cheers Richard Maher

Comment 24

[Benjamin Smedberg]

At this point, this issue is not a priority for the project. The java plugin is a closed-source blob that has a very poor track record of spinning nested event loops when it is not correct or safe to do so, and so we have been more and more aggressively preventing it from doing so. The correct solution to this bug is to stop using Java.

If somebody wants to provide a backtrace which identifies exactly how the recursion happens, I can provide details about how you might be able to fix the problem. Until then, this issue is not going to be tracked.

Comment 25

[Richard Maher]

Hi Ben,

A bit suprised at the level of hostility and prejudice against Java (unless you're channelling Steve Jobs :-) but I do appreciate your candour.
 
Ok, can someone please (pretty please!) supply Ben with the information he requires? 

I'm willing to pay dollars, if need be, to a charity or to send AUD$ to whereever. With the current exchange rate AUD$100 would still be US$100 after fees/charges or you could take the physical currency to dinner parties and regale your friends with anecdotes about the silly plastic notes.

Ok it's not a huge amount of money but I've got 3 kids in high school and I don't think I'm asking for the moon on a stick here. For those who contribute and have a dev/build environment set up could you not please take the reproduceable-at-will example, stick a break-point in there and take a snapshot of the stack and send it to Ben? (or attach it here)

What I believe, and infer from his response that Ben does too, is that an over zealous and unsophisticated recursion sheriff lurking in the code that is detecting false positives. Please help Bob Marley shoot this bloke!

Cheers Richard Maher

PS. Please indulge my re-opening of this ticket for a month or so. Priority 5 sure isn't gonna trigger any SLA alerts :-(

Comment 26

[Benjamin Smedberg]

The Mozilla project definitely has hostility toward Java, but it's not prejudicial. It is clear that the design of the Java plugin and code is fundamentally insecure, and the repeated exploits against it and slow response times make it clear that the Java plugin is not something that should be part of the web. We have to continue supporting it for legacy sites, but Java will be blocked by default and users will have to agree to override the security warning in order to enable Java in the near future.

Comment 27

[Adam Roach [:abr]]

(In reply to Richard Maher from comment #25)

> Ok, can someone please (pretty please!) supply Ben with the information he
> requires? 

This issue isn't really in my area of expertise, but I hate to see you dead in the water. Here's your path forward:

Follow the instructions at https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions to get a working debug build under Windows. I won't lie; this will take you a whole day if you're starting from scratch and don't have any of the tools installed. 

Start the built debug Firefox and attach to it using the debugger from the IDE you installed when you followed those directions. Place a breakpoint on entry to the js_ReportOverRecursed function. Run your test case until the debugger fires. Save the stack trace in a file and attach it to this bug. 

I would do this myself, but my Windows build environment is hosed at the moment. Good luck.

Comment 28

[Benjamin Smedberg]

Note that it is not necessary to build Firefox yourself. It is possible to debug a release build of Firefox using the Mozilla symbol server: https://developer.mozilla.org/en-US/docs/Using_the_Mozilla_symbol_server

Comment 29

[John Schoenick [:johns]]

Note also that you're probably going to get an incomplete backtrace nested inside npjp2.dll, which means you'll need to find an older JVM build that still had fastdebug symbols available, or an OpenJDK build which might behave differently.

Either way, java appears to be capturing event loops inside calls to the applet, which is something that is going to end up being very hard to fix from the browser's end. It would likely be easier in the long run to avoid doing things like sleeping inside main thread applet calls.

Comment 30

[Richard Maher]

Attachment: Stack Track from break at js_ReportOverRecursed

I think this is what was requested. Please let me know if something more is needed

Comment 31

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #29)
> Note also that you're probably going to get an incomplete backtrace nested
> inside npjp2.dll, which means you'll need to find an older JVM build that
> still had fastdebug symbols available, or an OpenJDK build which might
> behave differently.
> 
> Either way, java appears to be capturing event loops inside calls to the
> applet, which is something that is going to end up being very hard to fix
> from the browser's end. 

John, once again based on all available evidence, I put it to you that you are completely mistaken.

> It would likely be easier in the long run to avoid
> doing things like sleeping inside main thread applet calls.

I agree. Look this is not the "normal" use case. I am stess testing the synchronous version of a SEND function that will normally be asynchronous (as in XMLHTTPRequest) BUT, if user requirements dictate, it will be capable of having the Applet Worker thread WAIT for a NotifyAll(). And in the meantime any number of Callbacks may occur until the Boolean EOF is returned as true from the JS Callback.

Pretty sexy isn't it? 

ONCE AGAIN, it works with FF and Safari without batting an eyelid for any number of Tester objects and TABS on Mac OS/X and it works with Chrome and Opera on Windows.

I believe that XHR in the multiple setTimeout() scenario also experiences issues with FF on Windows but I'm still fleshing that out.

Anyway I'm still not sure why Sheriff Ben Brown hate JavIy but in case someone has a need for music to debug by (or has never heard the Marley version): -
http://www.youtube.com/watch?v=fVcFg-aOAdU

Cheers Richard Maher

Comment 32

[Richard Maher]

https://bugzilla.mozilla.org/show_bug.cgi?id=839945

Related?

Comment 33

[Richard Maher]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #26)
> . . .but Java will be blocked by default and users will have to agree to
> override the security warning in order to enable Java in the near future.

Churlish, unworthy, and quite frankly proprietary :-(

Comment 34

[John Schoenick [:johns]]

(In reply to Richard Maher from comment #31)
> John, once again based on all available evidence, I put it to you that you
> are completely mistaken.

I've yet to see any other theory. A full stacktrace would be helpful either way.

> > It would likely be easier in the long run to avoid
> > doing things like sleeping inside main thread applet calls.
> 
> I agree. Look this is not the "normal" use case. I am stess testing the
> synchronous version of a SEND function that will normally be asynchronous
> (as in XMLHTTPRequest) BUT, if user requirements dictate, it will be capable
> of having the Applet Worker thread WAIT for a NotifyAll(). And in the
> meantime any number of Callbacks may occur until the Boolean EOF is returned
> as true from the JS Callback.
> 
> Pretty sexy isn't it? 

The idea is that if java is handling the worker thread blocking by spinning a nested event loop, which icedtea appears to be based on c20, this behavior, on a loop, could exhaust the stack.

> ONCE AGAIN, it works with FF and Safari without batting an eyelid for any
> number of Tester objects and TABS on Mac OS/X and it works with Chrome and
> Opera on Windows.

Event ordering could help us escape the nested event loop in this specific test case, or OS X's java applet code could handle this differently - it'd be interesting to test with Oracle's JRE as well as OpenJDK in that case.

> I believe that XHR in the multiple setTimeout() scenario also experiences
> issues with FF on Windows but I'm still fleshing that out.
> 
> Anyway I'm still not sure why Sheriff Ben Brown hate JavIy but in case
> someone has a need for music to debug by (or has never heard the Marley
> version): -
> http://www.youtube.com/watch?v=fVcFg-aOAdU

The recursion error is fired, IIRC, when the stack is near the limit, not based on any sort of recursion heuristic, so it's unlikely that this is a false positive.

(In reply to Richard Maher from comment #30)
> Created attachment 737832 [details]
> Stack Track from break at js_ReportOverRecursed

Unfortunately this stack is incomplete, it ends at |user32.dll!_InternalCallWinProc@20() Unknown|, likely because we're nested inside npjp2.dll which lacks symbols. See c29

Comment 35

[Richard Maher]

Ok, I had a look at the source code and here are some relevant snippets: -

Stack> 	mozjs.dll!js::InvokeKernel(JSContext * cx, JS::CallArgs args, js::MaybeConstruct construct) Line 404	C++

JSINTER.CPP

      /* Run function until JSOP_STOP, JSOP_RETURN or error. */

(404) JSBool ok = RunScript(cx, script, ifg.fp());

      /* Propagate the return value out. */

      args.rval().set(ifg.fp()->returnValue());

      JS_ASSERT_IF(ok && construct, !args.rval().isPrimitive());

      return ok;

Stack> 	mozjs.dll!js::RunScript(JSContext * cx, JS::Handle<JSScript *> script, js::StackFrame * fp) Line 300	C++

bool
(289) js::RunScript(JSContext *cx, HandleScript script, StackFrame *fp)
{
    JS_ASSERT(script);
    JS_ASSERT(fp == cx->fp());
    JS_ASSERT(fp->script() == script);
    JS_ASSERT_IF(!fp->isGeneratorFrame(), cx->regs().pc == script->code);
    JS_ASSERT_IF(fp->isEvalFrame(), script->isActiveEval);
#ifdef JS_METHODJIT_SPEW
    JMCheckLogging();
#endif

(300) JS_CHECK_RECURSION(cx, return false);

#ifdef DEBUG
    struct CheckStackBalance {
        JSContext *cx;
        StackFrame *fp;
        CheckStackBalance(JSContext *cx)
          : cx(cx), fp(cx->fp())
        {}
        ~CheckStackBalance() {
            JS_ASSERT(fp == cx->fp());
        }
    } check(cx);
#endif


JSFRIENDAPI.H



#define JS_CHECK_RECURSION(cx, onerror)                              \
    JS_BEGIN_MACRO                                                              \
        int stackDummy_;                                                        \
        if (!JS_CHECK_STACK_SIZE(js::GetNativeStackLimit(js::GetRuntime(cx)), &stackDummy_)) { \
            js_ReportOverRecursed(cx);                                          \
            onerror;                                                            \
        }                                                                       \
    JS_END_MACRO

/*
 * This macro checks if the stack pointer has exceeded a given limit. If
 * |tolerance| is non-zero, it returns true only if the stack pointer has
 * exceeded the limit by more than |tolerance| bytes.
 */
#if JS_STACK_GROWTH_DIRECTION > 0
# define JS_CHECK_STACK_SIZE_WITH_TOLERANCE(limit, sp, tolerance)  \
    ((uintptr_t)(sp) < (limit)+(tolerance))
#else
# define JS_CHECK_STACK_SIZE_WITH_TOLERANCE(limit, sp, tolerance)  \
    ((uintptr_t)(sp) > (limit)-(tolerance))
#endif

#define JS_CHECK_STACK_SIZE(limit, lval) JS_CHECK_STACK_SIZE_WITH_TOLERANCE(limit, lval, 0)

Ok, so this needs someone to run it through with debug and put a break at the JS_RECURSION_CHECK macro call when the count is ~87 and examine the context (cx).

With all these rool your own Stacks and Frame Pointers Stack Pointers and Program Counters their is more than enough rope to hang one's self. (And judging by the number of Asserts this is not the first cx corruption that's been run down)

Or may be the cognoscente can just look at the order of the calls on the stack and say "Ahah! the second setTimeout() has been delivered before the first has left the Applet and the "CX" is unptotected"?

Anything else I can do please let me know.

Cheers Richard

Comment 36

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #34)
> (In reply to Richard Maher from comment #31)
> > John, once again based on all available evidence, I put it to you that you
> > are completely mistaken.
> 
> I've yet to see any other theory. 

John you don't seem to be listening too well either :-) Theories: -

1) The second setTimeout/Tester loop, depending on timing window, is able to stomp on memory, or be stomped on, as FireFox is not catering for the "Asynchronous System Trap (AST) Call without stack unwinding" of the event. Too much or not enough is being unstacked.
2) Heap(ish) memory is being used to maintain the cx and pointer arithmatic is leaving it uninitialized or polygomized.
3) FireFox not honouring/preserving the inherently single-threaded nature of Javascript. The pause for the Applet and the freeing up of the JS EDT for more work was beyond the designers imaginations.

> A full stacktrace would be helpful either
> way.

There is more than enough available in my last attachment. Just start examining from the other end Matron.
> 
> > > It would likely be easier in the long run to avoid
> > > doing things like sleeping inside main thread applet calls.
> > 
> > I agree. Look this is not the "normal" use case. I am stess testing the
> > synchronous version of a SEND function that will normally be asynchronous
> > (as in XMLHTTPRequest) BUT, if user requirements dictate, it will be capable
> > of having the Applet Worker thread WAIT for a NotifyAll(). And in the
> > meantime any number of Callbacks may occur until the Boolean EOF is returned
> > as true from the JS Callback.
> > 
> > Pretty sexy isn't it? 
> 
> The idea is that if java is handling the worker thread blocking by spinning
> a nested event loop, which icedtea appears to be based on c20, this
> behavior, on a loop, could exhaust the stack.

John you have to learn to let go. This is a fantasy that only you are indulging in.

> 
> > ONCE AGAIN, it works with FF and Safari without batting an eyelid for any
> > number of Tester objects and TABS on Mac OS/X and it works with Chrome and
> > Opera on Windows.
> 
> Event ordering could help us escape the nested event loop in this specific
> test case, or OS X's java applet code could handle this differently - it'd
> be interesting to test with Oracle's JRE as well as OpenJDK in that case.

Yes, simply change the millisecons for each Tester object and the number of iterations changes or have only one setTimeout loop and everything is peachy. Strange with the same amount of Stack memory being used don't you think?

Different configuration or Base OS DLLs being used on OS/X perhaps.
> 
> > I believe that XHR in the multiple setTimeout() scenario also experiences
> > issues with FF on Windows but I'm still fleshing that out.
> > 
> > Anyway I'm still not sure why Sheriff Ben Brown hate JavIy but in case
> > someone has a need for music to debug by (or has never heard the Marley
> > version): -
> > http://www.youtube.com/watch?v=fVcFg-aOAdU
> 
> The recursion error is fired, IIRC, when the stack is near the limit, not
> based on any sort of recursion heuristic, 

"heuristic"? Have you seen the code? Nn occasion theses boys wave a dead chicken over their heads and seek guidance in the colour of the entrails! Assert(SP->regyThingy(SP) || first number I thought of.

> so it's unlikely that this is a
> false positive.

Good enough for me :-)
> 
> (In reply to Richard Maher from comment #30)
> > Created attachment 737832 [details]
> > Stack Track from break at js_ReportOverRecursed
> 
> Unfortunately this stack is incomplete, it ends at
> |user32.dll!_InternalCallWinProc@20() Unknown|, likely because we're nested
> inside npjp2.dll which lacks symbols. See c29

Reposition your endoscope :-)

Cheers Richard Maher

Comment 37

[John Schoenick [:johns]]

Attachment: Linux + IcedTea stack w/nested event loop capture

(In reply to Richard Maher from comment #36)
> (In reply to John Schoenick [:johns] from comment #34)
> > (In reply to Richard Maher from comment #31)
> > > John, once again based on all available evidence, I put it to you that you
> > > are completely mistaken.
> > 
> > I've yet to see any other theory. 
> 
> John you don't seem to be listening too well either :-) Theories: -
> 
> 1) The second setTimeout/Tester loop, depending on timing window, is able to
> stomp on memory, or be stomped on, as FireFox is not catering for the
> "Asynchronous System Trap (AST) Call without stack unwinding" of the event.
> Too much or not enough is being unstacked.

This is roughly gibberish. Windows doesn't use ASTs, and stack corruption would rapidly crash, not exhaust the stack like this.

> 2) Heap(ish) memory is being used to maintain the cx and pointer arithmatic
> is leaving it uninitialized or polygomized.

So if I follow you... we polygomized our heapish memory? That could decohere all the megahertz to be sure.

> 3) FireFox not honouring/preserving the inherently single-threaded nature of
> Javascript. The pause for the Applet and the freeing up of the JS EDT for
> more work was beyond the designers imaginations.

So your code is so advanced it's blowing our mind. Got it.

> > A full stacktrace would be helpful either
> > way.
> 
> There is more than enough available in my last attachment. Just start
> examining from the other end Matron.

Your stack ends at InternalCallWinProc. This is not the process entry point. If we're in a re-entrant state we need to know if/what is causing the event loop to spin. Your stack shows approximately that *something* spun the event loop, invoking js near the stack limit.

Using OpenJDK / IcedTea on linux I was able to get the following stack which shows java... capturing the event loop. There are older java7 "fastdebug" builds available for windows that may be useful in getting a full stacktrace of the recursion error there, which would let us know if the same thing is happening there, or something else. As it stands, though, there is unfortunately nothing actionable in this bug report.

Comment 38

[Richard Maher]

Attachment: New simple.html that reveals the bug to the FF console

Comment 39

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #37)
> 
> This is roughly gibberish. Windows doesn't use ASTs, 

I know; shame. Out of curiosity, do you use an inner-mode stack/code to protect against user-mode corruption?

> and stack corruption
> would rapidly crash, not exhaust the stack like this.

I was clutching at straws and opting for a shotgun fix while you wanted key-hole surgery.

> 
> > 2) Heap(ish) memory is being used to maintain the cx and pointer arithmatic
> > is leaving it uninitialized or polygomized.
> 
> So if I follow you... we polygomized our heapish memory? That could decohere
> all the megahertz to be sure.

Ah the old megahertz decoherence trick, that's the second time I've fallen for it this week. But it's good to have you finally thinking outside the square and on the same hymne sheet with this John! Science is not all about facts and figures you know :-)
> 
> > 3) FireFox not honouring/preserving the inherently single-threaded nature of
> > Javascript. The pause for the Applet and the freeing up of the JS EDT for
> > more work was beyond the designers imaginations.
> 
> So your code is so advanced it's blowing our mind. Got it.

I'm pretty sure I deserved that, but a tad harsh nonetheless.

Look, you might have been less wrong than me but neither of our theories were correct. Also it was not my code that I was refereing too (although that is magnificent and am happy to talk about later) but the Java plugin itself.

I know your distain and blood pressure are palpable every time I mention it but please look at the diagram at this link (colours and everything!):-
http://docs.oracle.com/javase/6/docs/technotes/guides/jweb/applet/applet_execution.html

If you look at the "JavaScript Interpretter Busy" scenario that is where I was accusing FF designers of lacking imagination. The JS->Java->JS(setTimeout()  RETURN to the Java Applet is OK but the final leg RETURN to JS is just not happening leading to a pretty stock standard stack exhaustion. IOW FireFox sees timer events waiting and continues to service them exclusively to the detrement/starvation of all the Applet calls that want to return to JS. Are they on a different queue/priority?

To illustrate I have changed the JS in the attached simple.html: -

      var sendIt = function(){
                var mySeq = seqNum++;
		ctrl.send("[TO"+myId+"]", fred, false);
        	if (window.console) console.log("SENDIT "+mySeq);
		};

With Opera and Chrome the SENDIT messages appear roughly where one would expect. With FireFox you don't see a single one until there is a "too Much recursion" error then the all appear during the wind back. IE does the same but for one Tester() IE is ok while FF still fails. 

I didn't get a change to try this on FF or OS/X on Mac so if someone can do that I'd really appreciate it!

> 
> As it stands, though, there is
> unfortunately nothing actionable in this bug report.

I think you'll agree that we are no longer where we were.

Please let me know if there is anything else I can do.

Cheers Richard

Comment 40

[Richard Maher]

Here's some example output for those who don't want to run it themselves: -

[09:42:47.726] GET http://127.0.0.1/simple.html [HTTP/1.1 200 OK 215ms]
[09:42:47.898] The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must to be declared in the document or in the transfer protocol. @ http://127.0.0.1/simple.html
--
[09:43:29.805] returning from callback
[09:43:29.828] Before SEND: [TO2]
[09:43:30.105] In displatcher 137
[09:43:30.105] In callback 138
[09:43:30.106] returning from callback
[09:43:30.132] Before SEND: [TO2]
[09:43:30.406] In displatcher 138
[09:43:30.406] In callback 139
[09:43:30.406] returning from callback
[09:43:30.430] Before SEND: [TO2]
[09:43:30.706] In displatcher 139
[09:43:30.706] In callback 140
[09:43:30.707] returning from callback
[09:43:30.733] Before SEND: [TO2]
[09:43:31.007] In displatcher 140
[09:43:31.008] In callback 141
[09:43:31.008] returning from callback
[09:43:31.035] Before SEND: [TO2]
[09:43:31.308] In displatcher 141
[09:43:31.308] In callback 142
[09:43:31.309] returning from callback
[09:43:31.343] Before SEND: [TO2]
[09:43:31.609] In displatcher 142
[09:43:31.609] In callback 143
[09:43:31.609] returning from callback
[09:43:31.634] Before SEND: [TO2]
[09:43:31.910] In displatcher 143
[09:43:31.911] In callback 144
[09:43:31.911] returning from callback
[09:43:31.945] Before SEND: [TO2]
[09:43:32.211] In displatcher 144
[09:43:32.211] In callback 145
[09:43:32.211] returning from callback
[09:43:32.237] Before SEND: [TO2]
[09:43:32.511] In displatcher 145
[09:43:32.511] In callback 146
[09:43:32.512] returning from callback
[09:43:32.536] Before SEND: [TO2]
[09:43:32.812] In displatcher 146
[09:43:32.812] In callback 147
[09:43:32.813] returning from callback
[09:43:32.837] Before SEND: [TO2]
[09:43:33.113] In displatcher 147
[09:43:33.113] In callback 148
[09:43:33.113] returning from callback
[09:43:33.137] Before SEND: [TO2]
[09:43:33.414] In displatcher 148
[09:43:33.414] In callback 149
[09:43:33.415] returning from callback
[09:43:33.440] Before SEND: [TO2]
[09:43:33.715] In displatcher 149
[09:43:33.716] In callback 150
[09:43:33.716] returning from callback
[09:43:33.743] Before SEND: [TO2]
[09:43:34.016] In displatcher 150
[09:43:34.017] In callback 151
[09:43:34.017] returning from callback
[09:43:34.043] Before SEND: [TO2]
[09:43:34.318] In displatcher 151
[09:43:34.318] In callback 152
[09:43:34.318] returning from callback
[09:43:34.345] Before SEND: [TO2]
[09:43:34.619] In displatcher 152
[09:43:34.619] In callback 153
[09:43:34.620] returning from callback
[09:43:34.648] Before SEND: [TO2]
[09:43:34.920] In displatcher 153
[09:43:34.921] In callback 154
[09:43:34.921] returning from callback
[09:43:34.947] Before SEND: [TO2]
[09:43:35.221] In displatcher 154
[09:43:35.221] In callback 155
[09:43:35.222] returning from callback
[09:43:35.248] Before SEND: [TO2]
[09:43:35.522] In displatcher 155
[09:43:35.522] In callback 156
[09:43:35.523] returning from callback
[09:43:35.552] Before SEND: [TO2]
[09:43:35.823] In displatcher 156
[09:43:35.824] In callback 157
[09:43:35.824] returning from callback
[09:43:35.848] Before SEND: [TO2]
[09:43:36.124] In displatcher 157
[09:43:36.124] In callback 158
[09:43:36.125] returning from callback
[09:43:36.151] Before SEND: [TO2]
[09:43:36.425] In displatcher 158
[09:43:36.426] In callback 159
[09:43:36.426] returning from callback
[09:43:36.452] Before SEND: [TO2]
[09:43:36.726] In displatcher 159
[09:43:36.726] In callback 160
[09:43:36.727] returning from callback
[09:43:36.754] Before SEND: [TO2]
[09:43:37.027] In displatcher 160
[09:43:37.027] In callback 161
[09:43:37.028] returning from callback
[09:43:37.056] Before SEND: [TO2]
[09:43:37.328] In displatcher 161
[09:43:37.328] In callback 162
[09:43:37.329] returning from callback
[09:43:37.356] Before SEND: [TO2]
[09:43:37.629] In displatcher 162
[09:43:37.629] In callback 163
[09:43:37.630] returning from callback
[09:43:37.658] Before SEND: [TO2]
[09:43:37.930] In displatcher 163
[09:43:37.930] In callback 164
[09:43:37.931] returning from callback
[09:43:37.958] Before SEND: [TO2]
[09:43:38.231] In displatcher 164
[09:43:38.231] In callback 165
[09:43:38.232] returning from callback
[09:43:38.259] Before SEND: [TO2]
[09:43:38.532] In displatcher 165
[09:43:38.532] In callback 166
[09:43:38.533] returning from callback
[09:43:39.137] SENDIT 167
[09:43:39.137] SENDIT 166
[09:43:39.138] SENDIT 165
[09:43:39.138] SENDIT 164
[09:43:39.138] SENDIT 163
[09:43:39.138] SENDIT 162
[09:43:39.139] SENDIT 161
[09:43:39.139] SENDIT 160
[09:43:39.139] SENDIT 159
[09:43:39.139] SENDIT 158
[09:43:39.140] SENDIT 157
[09:43:39.140] SENDIT 156
[09:43:39.140] SENDIT 155
[09:43:39.140] SENDIT 154
[09:43:39.140] SENDIT 153
[09:43:39.140] SENDIT 152
[09:43:39.140] SENDIT 151
[09:43:39.140] SENDIT 150
[09:43:39.140] SENDIT 149
[09:43:39.141] SENDIT 148
[09:43:39.142] SENDIT 147
[09:43:39.142] SENDIT 146
[09:43:39.142] SENDIT 145
[09:43:39.142] SENDIT 144
[09:43:39.142] SENDIT 143
[09:43:39.142] SENDIT 142
[09:43:39.142] SENDIT 141
[09:43:39.142] SENDIT 140
[09:43:39.142] SENDIT 139
[09:43:39.142] SENDIT 138
[09:43:39.142] SENDIT 137
[09:43:39.142] SENDIT 136
[09:43:39.144] SENDIT 135
[09:43:39.144] SENDIT 134
[09:43:39.144] SENDIT 133
[09:43:39.144] SENDIT 132
[09:43:39.144] SENDIT 131
[09:43:39.144] SENDIT 130
[09:43:39.144] SENDIT 129
[09:43:39.144] SENDIT 128
[09:43:39.144] SENDIT 127
[09:43:39.144] SENDIT 126
[09:43:39.144] SENDIT 125
[09:43:39.144] SENDIT 124
[09:43:39.144] SENDIT 123
[09:43:39.145] SENDIT 122
[09:43:39.145] SENDIT 121
[09:43:39.145] SENDIT 120
[09:43:39.145] SENDIT 119
[09:43:39.145] SENDIT 118
[09:43:39.145] SENDIT 117
[09:43:39.145] SENDIT 116
[09:43:39.145] SENDIT 115
[09:43:39.145] SENDIT 114
[09:43:39.145] SENDIT 113
[09:43:39.145] SENDIT 112
[09:43:39.145] SENDIT 111
[09:43:39.152] SENDIT 110
[09:43:39.152] SENDIT 109
[09:43:39.152] SENDIT 108
[09:43:39.152] SENDIT 107
[09:43:39.152] SENDIT 106
[09:43:39.152] SENDIT 105
[09:43:39.152] SENDIT 104
[09:43:39.152] SENDIT 103
[09:43:39.152] SENDIT 102
[09:43:39.152] SENDIT 101
[09:43:39.153] SENDIT 100
[09:43:39.153] SENDIT 99
[09:43:39.153] SENDIT 98
[09:43:39.153] SENDIT 97
[09:43:39.153] SENDIT 96
[09:43:39.153] SENDIT 95
[09:43:39.153] SENDIT 94
[09:43:39.153] SENDIT 93
[09:43:39.153] SENDIT 92
[09:43:39.153] SENDIT 91
[09:43:39.153] SENDIT 90
[09:43:39.153] SENDIT 89
[09:43:39.153] SENDIT 88
[09:43:39.153] SENDIT 87
[09:43:39.154] SENDIT 86
[09:43:39.154] SENDIT 85

Comment 41

[Richard Maher]

Attachment: New simple.html that reveals the bug to the FF console

Comment 42

[Richard Maher]

> 
> I didn't get a chan[c]e to try this on FF or [Safari] on Mac so if someone can do
> that I'd really appreciate it!
> 

FireFox 20 works like a dream on Mac OS 10.8 Java 1.7-21

Safari 6.0.4 also ticks away without missing a beat.

Comment 43

[Richard Maher]

It looks like The Project has a lot more in common with Internet Explorer than one might think: -
https://connect.microsoft.com/IE/feedback/details/784918/out-of-stack-space-error-with-java-liveconnect-applet

:-(

But before you get your jollies by closing this call and kicking Java in the guts again can someone *please* take the time to proffer an explanation for why the following occurs? If it is deterministic and/or idempotent then at least I can attempt to code around it: -

    function tester(myId,timer){
     this.myId = myId;
     this.timer = timer;

     var sendIt = function(){
                var mySeq = seqNum++;
        ctrl.send("[TO"+myId+"]", fred, false);
            if (window.console) console.log("SENDIT "+mySeq);
        setTimeout(sendIt,timer);
        };

     var fred = function() {
        callNum++;
        if (window.console) console.log("In callback "+callNum);
        document.mfForm.username.value=myId+"CB<: "+callNum;
        document.mfForm.state.value=myId+"CB>: "+callNum;
//        setTimeout(sendIt,timer); MOVED UP AFTER SYNC SEND
        if (window.console) console.log("returning from callback");
        return true;
     }

IOW, if I relocate the setTimeout() call the bug is dodged. NB: I can't always perform these dodge-ball acrobatics but for those instances where I can it would be good to have some sort of recipe for success!

Cheers Richard Maher

PS. Does anyone else not find it a tad ironic that all these Java bashers and biggots are happy to promote a FF/Windows bug in their own code while throwing stones at Java for not fixing theirs???

Comment 44

[John Schoenick [:johns]]

(In reply to Richard Maher from comment #43)
> It looks like The Project has a lot more in common with Internet Explorer
> than one might think: -
> https://connect.microsoft.com/IE/feedback/details/784918/out-of-stack-space-
> error-with-java-liveconnect-applet
> 
> :-(

Spinning nested event loops makes it inherently possible to deadlock or exhaust the stack - if java can both schedule events (setTimeout(someFunction, 0) creates a pending event to call someFunction) and spin the event loop, telling the browser to process events, the only way we can prevent recursion is to refuse to run the setTimeout until java returns. That, however, would break java code that does something like

> CallJS(setTimeout(myfunc, 0))
> Sleep(5000)
> return

In that case, |myfunc| would not run for 5+ seconds, while waiting for the java call that scheduled it to exit. Either way, we're breaking something!

> But before you get your jollies by closing this call and kicking Java in the
> guts again can someone *please* take the time to proffer an explanation for
> why the following occurs? If it is deterministic and/or idempotent then at
> least I can attempt to code around it: -

How else am I expected to get my jollies?

You're wrong though, that we simply enjoy breaking java. Back when the java applet code was designed, browsers were much simpler, and spinning the event loop was likely no big deal. Now we have to allow it or we would break java entirely, but the result is its easy to construct really, really, confusing series of events that exhaust the stack.

>     function tester(myId,timer){
>      this.myId = myId;
>      this.timer = timer;
> 
>      var sendIt = function(){
>                 var mySeq = seqNum++;
>         ctrl.send("[TO"+myId+"]", fred, false);
>             if (window.console) console.log("SENDIT "+mySeq);
>         setTimeout(sendIt,timer);
>         };
> 
>      var fred = function() {
>         callNum++;
>         if (window.console) console.log("In callback "+callNum);
>         document.mfForm.username.value=myId+"CB<: "+callNum;
>         document.mfForm.state.value=myId+"CB>: "+callNum;
> //        setTimeout(sendIt,timer); MOVED UP AFTER SYNC SEND
>         if (window.console) console.log("returning from callback");
>         return true;
>      }

So, we still don't have a full stack trace showing exactly what order the Oracle JRE applet code is calling things to cause this. But, based on this code, and comment 40, it looks like it's handling things the same way that the OpenJDK applet code (icedtea) is doing it. That code is open source, see comment 20.

So if we assume both the OpenJDK and oracle applet plugins are doing the same thing, here's what would be happening in your testcase:
+-------+
| BEGIN +---------+
+-------+         |
                +-v-------------------------+
          +----->[A] Some Javascript Runs   |
          |     |+-------------------------+|
          |     ||DoSomething();           ||
          |     ||Applet->SomeFunction();+-------+
          |     ||return; // never reached!||    |
          |     |+-------------------------+|    |
  +-------+     +---------------------------+    |
  |                                              |
  |                   +--------------------------+
  |                   |
  |                   |
  |    +--------------v-------------------+
  |    |[B]Applet call code, approximately|           +----------------------+
  |    | +------------------------------+ |           |Java thread  [E]      |
  |    | |DispatchRequestToJavaThread()+--------------> +-------------------+|
  |    | |while (!IsJavaThreadDone()) { | |           | |DoSomeJavaStuff(); ||
  |    | |  ProcessJavaThreadRequests();<---------------+CallbackToJs([C]); ||
  |    | |  SpinEventLoop();    ^   v   | |           | +-------------------+|
  |    | |}               ^     |   |   | |           +----------------------+
  |    | |return;         |     |   |   | |
  |    | +----------------|-----|---|---+ |       +-----------------------------------+
  |    +------------------|-----|---|-----+       |           [C]                     |
  |                       |     |   |             |  JS which calls setTimeout(D),    |
  |                       |     |   +------------->  adding event D to the event queue|
  |                       |     |                 +---------------------v-------------+
  |                       |     +--------------------------<-Returns<---+
  |                       |
  |                       |              +-------------------------------------------+
  |                       +------------->|The event loop spins. If [D] is in the     |
  |                                      |event queue, or has already run and        |
  +-------------<- if [A] is pending --<-|scheduled [A], then [D] or [A] will now    |
                                         |run, despite [B] still not having returned!|
                                         +-----------^-----+-------------------------+
                                                     |     | if [D] is pending...
                                                     |     |
                                               +-----+-----v------------+
                                               |       [D]              |
                                               |Schedules event [A] with|
                                               |setTimeout.             |
                                               +------------------------+

Or in other words:
1) [B] Calls ProcessJavaThreadRequests(), which includes the request to run |setTimeout(fred, 0)|
2) [B] then calls SpinEventLoop, unaware that it just scheduled an event.
3) The browser runs fred, which calls setTimeout(sendit, 0)
4) The browser sees there's still more events to run (now that sendit is queued) and runs sendit
5) sendIt goes back to (1) and we recurse until the stack is empty.

At this point the stack looks like
sendIt() -> ctrl.send() -> SpinEventLoop() -> sendIt() -> ctrl.send() -> SpinEventLoop() -> ...

The way to fix this would be to refuse to run other JS events against that document until the initial sendIt() event finished. But this would mean that calls to java that take a long time (several seconds) would make the tab appear frozen until they finish. This is what any long-running (non-java) javascript call does, but it breaks java's "dont block other events!" logic entirely, and makes applet calls fully blocking.

As for why this works in some cases/browsers, I suspect either
 - Timing - if the java call schedules [D], [D] wont run if [E] exits 'fast' enough -- [B] will see IsJavaThreadDone return true, and return before having a chance to fire the event loop, which would run D. This is essentially a race condition.
 - It's possible that webkit browsers refuse to run [D] inside the event that scheduled it (which would be seen as [A] by the browser). This would mean that while they work with your code, they break java code that expects its timeouts to fire before its applet code finishes. It's also not possible to 100% detect this, as there may be multiple ways to call into java.

> IOW, if I relocate the setTimeout() call the bug is dodged. NB: I can't
> always perform these dodge-ball acrobatics but for those instances where I
> can it would be good to have some sort of recipe for success!

This is because your setTimeout() call was moved until after the call from the applet [B] returns. Prior, [D] (fred in this case) was being run by the applet code, which was waiting for [E] to finish, before E's thread fully exited. By waiting explicitly for the applet call to finish before scheduling events, you guarantee that java wont trigger those events.

Or, more generically:
Don't schedule events in javascript before, or during, a call to java, if you don't want them to run while that call to java is happening.

Comment 45

[Richard Maher]

(In reply to John Schoenick [:johns] from comment #44)

Hi John,

Thanks as always for your replies especially this detailed excellent response! With the help of your pictures for dummies (which survived wrapping) I now know what you've been trying to tell me. You're far too reasonable and you make far too bloody plausible an argument, compounded by your source-code references for iced-tea and your ballpark mud-map of the process.

(Let me also point out that my experience has been that this forum is an extremely useful resource with very engaging participants and I am not ungrateful for it)

Having said all that, I'd be absoluteley flabergasted and totally incredulous if the "new" Java plugin was found to be running similar algorithms! And I'm gonna tell you for why: -

You simply cannae change the laws of JavaScript Jim! JavaScript, and its Event dispatching model is fundamentally SINGLE-THREADED! (Forget that bollocks feature devoid mutant that some call "web-workers") A setTimeout() event simply *cannot* interrupt a currently executting setTimeout() event!!!

Mess with that and you're dealing with forces you don't understand John.

Here are some references/straws that I found to cling to to help me believe that the world has not gone completely mad and the sun will come up tommorow: -

http://stackoverflow.com/questions/7575589/how-does-javascript-handle-ajax-responses-in-the-background/7575649#7575649

http://www.w3.org/TR/html5/webappapis.html#event-loops

I simply dismiss any number of Event-Loops, Task-Queues, and Browser-Contexts that you may have and defer to what Moses brought down from the mountain and number 11 says "A Timer Event shall not interrupt another Timer Event".

I know this has nothing to do with ASTs John but because they seem to be a rosetta stone for what you and me know what were trying to do let me give you my little sojourn into nostalgia: -

    function dclAST(astAddr) 
       {
       setTimeout(astAddr,0);
       return;
       }
I'm in an AST and I know that no other AST (of same or lesser mode) can interrupt me so it goes on the "do later" queue". It does not simply become some dodgy "Synchronous Event" nested or otherwise because someone decides to spin the chocolate wheel.

Don't you have a mutex or some such that tells the chocolate wheel that it's already spinning???

if (g_main_somethingthere && !lib$ast_in_prog)
   spinTheLoop();

How bout that?

Sorry, it's just that I now understand what you say is happening (backed up also by FF behaviour if the setTimeout(func,secs) has secs set to 5) and I'm a little bit beside myself. Sorry if this comes across as more gibberish but I had to put pen to paper. For the record, does Iced-Tea exhibit the stack overflow?

Anyway I'll look more tomorrow. Thanks again.
 
> Spinning nested event loops makes it inherently possible to deadlock or
> exhaust the stack - if java can both schedule events
> (setTimeout(someFunction, 0) creates a pending event to call someFunction)
> and spin the event loop, telling the browser to process events, the only way
> we can prevent recursion is to refuse to run the setTimeout until java
> returns. That, however, would break java code that does something like
> 
> > CallJS(setTimeout(myfunc, 0))
> > Sleep(5000)
> > return

Yes but I insist that any code that relies on a timeout interrupting an active timeout IS BROKEN.

> 
> 
> How else am I expected to get my jollies?

Tough call. Java bashing is way up there :-)
> 
> You're wrong though, that we simply enjoy breaking java. Back when the java
> applet code was designed, browsers were much simpler, and spinning the event
> loop was likely no big deal. Now we have to allow it or we would break java
> entirely, but the result is its easy to construct really, really, confusing
> series of events that exhaust the stack.

Fair enough.

> 
> >     function tester(myId,timer){
> >      this.myId = myId;
> >      this.timer = timer;
> > 
> >      var sendIt = function(){
> >                 var mySeq = seqNum++;
> >         ctrl.send("[TO"+myId+"]", fred, false);
> >             if (window.console) console.log("SENDIT "+mySeq);
> >         setTimeout(sendIt,timer);
> >         };
> > 
> >      var fred = function() {
> >         callNum++;
> >         if (window.console) console.log("In callback "+callNum);
> >         document.mfForm.username.value=myId+"CB<: "+callNum;
> >         document.mfForm.state.value=myId+"CB>: "+callNum;
> > //        setTimeout(sendIt,timer); MOVED UP AFTER SYNC SEND
> >         if (window.console) console.log("returning from callback");
> >         return true;
> >      }
> 
> So, we still don't have a full stack trace showing exactly what order the
> Oracle JRE applet code is calling things to cause this. 

I'll ask around tomorrow.

> But, based on this
> code, and comment 40, it looks like it's handling things the same way that
> the OpenJDK applet code (icedtea) is doing it. That code is open source, see
> comment 20.

Looks that way.
> 
> So if we assume both the OpenJDK and oracle applet plugins are doing the
> same thing, 

I say/prey no.

> here's what would be happening in your testcase:
. . . great diagram + explanation

> As for why this works in some cases/browsers, I suspect either
I'll load up FF OS/X tomorrow

>  - It's possible that webkit browsers refuse to run 
Chrome definitely looks to block other events (As I'd expect) Opera not so much.

> 
Thanks again. Cheers Richard

Comment 46

[Richard Maher]

if (!g_main_context_acquire())
 stopBeingSilly() Perhaps?

http://www.gtk.org/api/2.6/glib/glib-The-Main-Event-Loop.html#g-main-context-acquire

Does that not implicitly happen with g_main_context_iteration() anyway?

Look, I maintain Iced Tea is a red herring anyway. Chrome, Opera, on Windows and Safari and *FireFox* on OS/X are unbreakable! 

"Race Condition" my left buttock! I have loaded each tab up with 4 Tester objects banging away at 10s of ms and up to 7 tabs at a time and with varying degrees of performance (Opera flies) it all works.

If you say that you can display a stack overflow with my example on FireFox OS/X with any combination of Tester()s, setTimeouts and Thread.sleeps() then I've got 3 sheep stations and an oil rig that says you're talking **** :-)

Cheers Richard

Comment 47

[Richard Maher]

Hi (again) John,

I've calmed down and re-evaluated what I thought I understood about JavaScript and Events. The penny is beginning to drop, and my foolish notion of separate execution contexts for an "event" -vs- "mainline" in JavaScript sees me looking like a numpty harkening to days gone by :-(

I was/am using synchronized applet methods and "dclAST" in an attempt to single-thread the JS-Applet context but you've quite rightly pointed out the twilight zone of the plugin where no Java or JavaScript is executing yet the Java can't RETURN as a scheduled event is there on the queue. Until there is a quiet-point (sea of tranquility with no active events) then stack exhaustion could happen. In my "bang-away" example there is no quiet-point so a combo of shifting the setTimeout() call plus making the Java i/o asynchronous will save the day.

Look, I've really learned a lot from your replies and will re-close this ticket shortly. But if I can ask just two more questions that'd be great: -

1) WRT Iced-Tea "254 if (pthread_self() == itnp_plugin_thread_id)". Can the JVM thread, that I'm running in, in anyway influence the outcome of that condition? IOW does the pthread check have anything to do with the currently executing JVM Thread?
2) If you, someone, anyone could show my example exhausting the stack with FireFox OS/X then I can walk away a happy man!

Thanks again for bearing with me and to those who replied/listened!

Cheers Richard Maher

Comment 48

[Richard Maher]

One more thing to point out in my epiphany was: if you were to enforce what I was asking you to do then the amazing LiveConnect functionality that I take full advantage of would not be available to me :-(

Sorry and thanks again for persisting. Cheers Richard

Comment 49

[jonesr]

Richard, if you are still reading this, I wondered if you were still able to get this to work correctly under firefox 21 on mac os x? I'm getting LiveConnect failures under FF 21 on Mac OS X (only platform tested so far), that I haven't seen before, and was trying to look at existing bug reports to see if they are realted before creating one of my own.

Regards,
Richard.

Comment 50

[John Schoenick [:johns]]

(In reply to jonesr from comment #49)
> Richard, if you are still reading this, I wondered if you were still able to
> get this to work correctly under firefox 21 on mac os x? I'm getting
> LiveConnect failures under FF 21 on Mac OS X (only platform tested so far),
> that I haven't seen before, and was trying to look at existing bug reports
> to see if they are realted before creating one of my own.

Bug 872969, maybe

Comment 51

[jonesr]

Thanks John,

It does indeed sound exactly like that. I missed it in my search, as I was looking for liveconnect, and that bug hadn't used that phrase yet. I've included some details, and link to a relatively simple test case in Bug 872969

Regards,
Richard.

Comment 52

[John Schoenick [:johns]]

(In reply to Richard Maher from comment #47)
> Look, I've really learned a lot from your replies and will re-close this
> ticket shortly. But if I can ask just two more questions that'd be great: -
> 
> 1) WRT Iced-Tea "254 if (pthread_self() == itnp_plugin_thread_id)". Can the
> JVM thread, that I'm running in, in anyway influence the outcome of that
> condition? IOW does the pthread check have anything to do with the currently
> executing JVM Thread?

I'm not very familiar with IcedTea code, but it looks like the JavaRequestProcessor can be used as the main dispatcher for an Applet, or in other contexts. In the latter it skips the event loop spinning behavior, but also processing callbacks from the java thread for the browser, so even if it could be manipulated, it would break other things.

> 2) If you, someone, anyone could show my example exhausting the stack with
> FireFox OS/X then I can walk away a happy man!

Testing a bunch of event loop re-entry scenarios doesn't sound very fun, but I would guess something along the lines of:

> javaFunction() {
>   evalJS("setTimeout(callAppletJavaFunction(), 0)");
>   Sleep(900000);
> }

Would likely wreck havoc, as the call would never return but events would be running. I'm not sure how java attempts to force the event loop on OS X or how that works with our OOP implementation, so it might instead crash, freeze, or uncover some kind of wonderful new failure state.