Last Comment Bug 609663 - JSD doesn't support jsdIDebuggerService.interruptHook
: JSD doesn't support jsdIDebuggerService.interruptHook
Status: RESOLVED WONTFIX
[firebug-p1]
:
Product: Core
Classification: Components
Component: JavaScript Engine (show other bugs)
: Trunk
: x86 Windows Vista
: -- normal with 4 votes (vote)
: ---
Assigned To: general
:
Mentors:
: 586104 (view as bug list)
Depends on:
Blocks: JaegerDebug
  Show dependency treegraph
 
Reported: 2010-11-04 11:51 PDT by Jan Honza Odvarko [:Honza] PTO 07/23 - 08/08
Modified: 2011-01-07 16:18 PST (History)
16 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
betaN+


Attachments
Test extension (xpi) (5.15 KB, application/x-xpinstall)
2010-11-10 10:08 PST, Jan Honza Odvarko [:Honza] PTO 07/23 - 08/08
no flags Details

Description Jan Honza Odvarko [:Honza] PTO 07/23 - 08/08 2010-11-04 11:51:11 PDT
The current implementation of JSD doesn't currently support single stepping (in the debugger).

Here is how to reproduce that:

1) Install Firebug from: http://www.softwareishard.com/temp/firebug-1.7X.0a4.608763.xpi
2) Load page: http://getfirebug.com/tests/content/net/601/issue601.html
3) Enable the Script panel, reload the page
4) Create breakpoint on line 37
5) Press the "Post" button on the page
6) The debugger should halt on the breakpoint
7) Press F10 (step over) to single-step to move to the next executable line -> BUG 

Note that single-stepping is Firebug's key feature and the debugger is not much usable without it.

Honza
Comment 1 Andreas Gal :gal 2010-11-04 11:54:47 PDT
As long breakpoints work you can always emulate single stepping by setting a breakpoint at the next instructions and all exception targets, no?
Comment 2 Jan Honza Odvarko [:Honza] PTO 07/23 - 08/08 2010-11-04 12:04:34 PDT
(In reply to comment #1)
> As long breakpoints work you can always emulate single stepping by setting a
> breakpoint at the next instructions and all exception targets, no?
Yes, this should be feasible. Should we see it as a workaround or the plan is to leave the current JSD APIs this way?

John, do you see any problems with such workaround?

Honza
Comment 3 Andreas Gal :gal 2010-11-04 12:09:43 PDT
I am guessing here, but I think Jaegermonkey would implement single-stepping like this since they don't have an interpreter dispatch loop. So we could do this in JSD, or in firebug.
Comment 4 David Mandelin [:dmandelin] 2010-11-04 12:12:27 PDT
(In reply to comment #3)
> I am guessing here, but I think Jaegermonkey would implement single-stepping
> like this since they don't have an interpreter dispatch loop. So we could do
> this in JSD, or in firebug.

That's exactly right.
Comment 5 John J. Barton 2010-11-04 12:19:09 PDT
Sorry folks, we already do that. There is no jsd API for single stepping.  Let me see if I can figure out what operation is failing.
Comment 6 John J. Barton 2010-11-04 12:33:31 PDT
You can start with Bug 609687 I can't even trace.
Comment 7 John J. Barton 2010-11-04 12:41:50 PDT
I take it back I can trace, just not run chromebug.  I'll have to look into the details. Looks like the interrupt hook is calling us.
Comment 8 John J. Barton 2010-11-04 14:41:44 PDT
Actually what I saw in a quick peek was function call tracing in FF4, because the interrupt hook is not working so we don't trap on the next line, we exit the function and the function call tracing begins. 

To be concrete, the call we expect is from:
  /**
107      * Called before the next PC is executed.
108      */
109     attribute jsdIExecutionHook interruptHook;

After each call back we compute the line number based on the PC and decide if we have moved away from the previous line (ie we have stepped). If so, we break. We also set function call hooks in case the function returns or calls before the step and to implement step in and step out. These seem to work, or at least we see tracing.
Comment 9 John J. Barton 2010-11-04 14:51:39 PDT
Steps to reproduce what I did:
1) Install Firebug from:
http://www.softwareishard.com/temp/firebug-1.7X.0a4.608763.xpi
(I didn't do this, I'm using SVN directly)
2) Load page: https://getfirebug.com/tests/content/script/singleStepping/index.html
3) Enable the Script panel, reload the page
4) Create breakpoint on line 14
5) Press the button on the page
6) The debugger will halt on the breakpoint
7.) Step over or F10

If you use Firebug Icon Menu > Tracing > Options > FBS_STEP, then when you step over you should see the step tracing.
Comment 10 David Mandelin [:dmandelin] 2010-11-04 14:57:28 PDT
(In reply to comment #8)
> Actually what I saw in a quick peek was function call tracing in FF4, because
> the interrupt hook is not working so we don't trap on the next line, we exit
> the function and the function call tracing begins. 
> 
> To be concrete, the call we expect is from:
>   /**
> 107      * Called before the next PC is executed.
> 108      */
> 109     attribute jsdIExecutionHook interruptHook;

AFAIK, this hook will not ever be implemented for JaegerMonkey. I would expect that making a call to an interrupt handler before every bytecode would destroy any performance advantage from using the compiler. So, for each feature that now uses the interrupt hook, we must choose:

1. If the feature can be implemented without the interrupt hook (i.e., it's possible to do by Fx4 release), do that.

2. Otherwise, we would have to exit JM mode and go to the interpreter. We'd have to do new code in the engine to add this feature, but one time would take care of it all, and I think it would not be too hard. Tracer integration already uses related features.
Comment 11 John J. Barton 2010-11-04 15:06:15 PDT
If the user is single stepping they get no value from the performance of the compiler anyway. Or are you saying that its just not practical, eg the non-debugging performance would suffer just for the potential to interrupt?

The only other implementation I can imagine off hand is to set a breakpoint on every line of the function. We don't in general know what the next step will do. (The line table code in jsd was (at least the last time I check) so slow I could not analyze more than about 100 lines with out visible slowdowns so that could be an issue with this approach).
Comment 12 David Mandelin [:dmandelin] 2010-11-04 15:14:56 PDT
(In reply to comment #11)
> If the user is single stepping they get no value from the performance of the
> compiler anyway. 

Agreed. Using option 2, switching to the interpreter to single-step, and switching back after the user continues [1], is a reasonable implementation of single-stepping.

> The only other implementation I can imagine off hand is to set a breakpoint on
> every line of the function. We don't in general know what the next step will
> do. 

Yeah, that doesn't sound good. But I think we could provide a new API that would tell you the possible successor bytecodes of the current bytecode. I guess exceptions are harder--I think we'd just have to make sure to call you back any time we throw/catch/whatever an exception while single-stepping.

[1] We can't switch back to JM code at an arbitrary point. We would run in the interpreter until we got to the next "safe point", then jump in.
Comment 13 David Anderson [:dvander] 2010-11-04 15:39:28 PDT
It seems reasonable for JM to do a "single-step" recompilation that places a trap on every source line. If js_PCToLineNumber is slow we can fix that.
Comment 14 Brendan Eich [:brendan] 2010-11-04 16:15:34 PDT
(In reply to comment #13)
> It seems reasonable for JM to do a "single-step" recompilation that places a
> trap on every source line. If js_PCToLineNumber is slow we can fix that.

There's already a thread-local get-source-note cache, so js_PCToLineNumber should not blow up quadratically. Question is what are you gonna do for Firefox 4, which needs to be "reliable and soon" more than "fast" at debugged code perf.

/be
Comment 15 David Mandelin [:dmandelin] 2010-11-04 16:17:00 PDT
If David thinks using recompilation for single-stepping is good then it probably is. And as long as the basic recompilation mechanism is solid, it may actually be technically easier than achieving arbitrary JM/interpreter switching.
Comment 16 John J. Barton 2010-11-04 16:52:49 PDT
I just want to be completely sure that we are not missing something here: is it the case that the interruptHook is not (and won't be) implemented?  And the related question is: is any other feature of jsd not implemented that we should investigate? Given that you want 'soon' I just want to check that we are not in for another surprise.
Comment 17 John J. Barton 2010-11-04 16:59:39 PDT
There are other Firebug features that rely on interruptHook: several of the break-on-next features. Unlike single step we can't use breakpoint on all PCs to implement these. 

There is
 attribute jsdICallHook topLevelHook;
which Firebug does not use because it crashed Firefox back in FF2 plus we don't really know what it does. Perhaps it could work for  our break-on-next.
Comment 18 John J. Barton 2010-11-05 10:34:25 PDT
A few more things that came to me:

The set-breakpoints-on-every line scheme would require bug 449473, Extend jsdIScript to include the maximum value of the program counter.

Looking back at my notes, the performance issue is with isLineExecutable(). 

Note that single stepping in outer or top-level functions (the ones that run directly after compile), would require setting breakpoints on every line of the file since we only know the line range. Some files run to 75kloc now.
Comment 19 David Mandelin [:dmandelin] 2010-11-09 14:25:39 PST
*** Bug 586104 has been marked as a duplicate of this bug. ***
Comment 20 John J. Barton 2010-11-09 20:59:35 PST
I don't understand why the blocking flag was removed here.
Comment 21 Michael Lefevre 2010-11-10 01:42:37 PST
(In reply to comment #20)
> I don't understand why the blocking flag was removed here.

This bug has never had a blocking flag - maybe you were thinking of one of the other single-stepping bugs (2 of which are blockers)?
Comment 22 John J. Barton 2010-11-10 07:22:48 PST
(In reply to comment #21)
> This bug has never had a blocking flag - maybe you were thinking of one of the
> other single-stepping bugs (2 of which are blockers)?

No, since I don't know about those bugs. I guess I was just assuming, since without a fix here JS debugging does not work so it should obviously block.
Comment 23 Steve Roussey (:sroussey) 2010-11-10 07:42:18 PST
What are the other bugs?

I thought interruptHook was used for break-on-property change/add/remove. Is it used for break-on-dom-mutation?
Comment 24 John J. Barton 2010-11-10 07:47:30 PST
(In reply to comment #23)
> I thought interruptHook was used for break-on-property change/add/remove. Is it
> used for break-on-dom-mutation?

break-on-property uses Object.watch
break-on-dom-mutation uses...DOM mutation
break-on-script uses... interruptHook

But the key point: interruptHook is used for JS single steps.
Comment 25 Michael Lefevre 2010-11-10 08:51:23 PST
(In reply to comment #22)
> No, since I don't know about those bugs.

Sorry - I'm not sure I'm adding anything useful here. Possibly they are not as related as I think - you know more about this than I do. But I was thinking of bug 610793 and bug 594054
Comment 26 Brendan Eich [:brendan] 2010-11-10 09:48:59 PST
Michael is helping, because we need specific user-level task bugs like "Support single-stepping with Firebug in [JaegerMonkey-]compiled code."

This bug wants the interruptHook, an old interpreter-only hook that calls before each bytecode dispatch and lets the debugger steer by forcing throw or return of a value. Nice old API but it's not coming back in JM'ed code. If you really need it for a user-facing task, please identify the task. Perhaps there is another way to implement that.

If not, then Firebug will have to force JS to stay in the interpreter. Is that on the table? It would seem to be the "fix" for this bug, if this bug really needs to be fixed.

You can't get the old API in all its detailed semantics, without actually using it. Which means no JM, slower debugging when using this hook -- could be ok for the user but we have to engineer things so JM runs in other debugging scenarios, and turns off and back on appropriately.

/be
Comment 27 David Mandelin [:dmandelin] 2010-11-10 09:59:50 PST
Steve Fink is working on the JM side of single-stepping now. I don't know exactly what he's planning, but in comments 13-14 we discussed getting this to work with JM. I guess you're concerned about timing, which is valid. We should get Steve's opinion on that once he's analyzed the situation, but I think it's doable for 4.
Comment 28 Jan Honza Odvarko [:Honza] PTO 07/23 - 08/08 2010-11-10 10:08:04 PST
Created attachment 489533 [details]
Test extension (xpi)

I have created a simple test extension that is using the JSD interruptHook and demonstrates the problem.

STR:
1) Install the extension
2) Load: http://getfirebug.com/tests/issues/bugzilla/b609663/test.html and follow instructions on the page.

Perhaps, we could start with the extension as with the "user level task" and evolve it into working version. The source code is here:
http://code.google.com/p/fbug/source/browse/#svn/tests/mozilla/bug609663

John, would it be hard to improve the example to demonstrate what we need?

Anyway, this is an important problem for Firebug and should marked as beta8 blocker.

Honza
Comment 29 John J. Barton 2010-11-10 10:18:18 PST
The user level tasks that are broken now are step over, step into, step out,
break-on-next-script.

The single steps get no benefit from optimizations in the developers code: interpreter stepping works great.

break-on-next is essential a cross-stack version of single step, the user or debug engine wants to stop on the next thing that runs. Having the next call stack run de-optimized would not be ideal. A different API for this case would be a blessing anyway.

Nevertheless being able to execute single PC steps is an important tool since we don't have JS access to byte codes.
Comment 30 Steve Fink [:sfink] [:s:] 2010-11-10 23:04:05 PST
Since right now it appears that we won't be supporting the interruptHook in JM, I'd like to move discussion of how to implement single-stepping to bug 610793.

(If we decide to implement interruptHook after all, I'll come back here.)
Comment 31 David Mandelin [:dmandelin] 2010-11-23 16:31:20 PST
Steve, what's your current plan? This bug just got marked blocking, but I think single-step is really the key requirement, and the significance of this bug depends on your strategy for single-step. Or is there another reason we need this?
Comment 32 Steve Fink [:sfink] [:s:] 2010-11-23 18:16:45 PST
We do not need the full interruptHook semantics for single-stepping, and there hasn't been any other usage identified that would require them, so I'll resolve this bug as WONTFIX.

Is it a bad sign when a blocking bug gets marked WONTFIX? :-)

It is possible to implement interruptHook semantics in JM, easier in fact than one-interrupt-per-line. If someone needs it for something, they can open a new bug. For now, it would be a pointless extra maintenance cost.

David, can you clear the blocking flag? Or is that someone else? Or does it not matter, since the bug is marked RESOLVED?
Comment 33 John J. Barton 2010-11-23 22:11:47 PST
(In reply to comment #32)
> We do not need the full interruptHook semantics for single-stepping, and there
> hasn't been any other usage identified that would require them, 

Please see comment 17 above.
https://bugzilla.mozilla.org/show_bug.cgi?id=609663#c17
Comment 34 Steve Fink [:sfink] [:s:] 2010-11-29 11:20:03 PST
Right, though interruptHook is still not the right approach for that, since it requires recompiling the world (where "world"=="compartment", maybe?).

I filed bug 615277 for the break-on-next functionality.
Comment 35 John J. Barton 2011-01-07 16:18:40 PST
For my future reference, the resolution here is on bug 610793 which adds
 JSD script property enableSingleStepInterrupts

Note You need to log in before you can comment on or make changes to this bug.