338978 - SoC: Streamlined JS debugging for Web developers

Status: RESOLVED FIXED

(Other Applications Graveyard :: Venkman JS Debugger, defect)

Description

[chris hofmann]

Giving webdevelopers a better JavaScript debugging experience
	Gijs Kruitbosch 	Mike Shaver

Comment 1

[Benjamin Smedberg]

Adam/Javier: this is not particularly related to the Eclipse projects, but if you have suggestions/requests for the core JSD APIs, you should get in touch with Hannibal.

Comment 2

[:Gijs (he/him)]

I'm mildly surprised finding email about this in my inbox, and not being CC-ed, but then again, I watch rginda. ;-)


So, I talked to a bunch of people when at XTech 2006, and I tried to remember what they told me, and put all that up at http://wiki.mozilla.org/User:GijsKruitbosch/JS_Debugging . Feel free to look over it and comment on the bug, email me, whichever. As the wiki says, suggestions, itches, problems, any feedback is welcome.


Mike, have you mailed Hewitt about this? (and more importantly, did he mail back yet?)

Adam/Javier/Benjamin: does Eclipse have a (SoC/internship/other) project of its own for this stuff?

I'm correcting the spelling in the summary for a bit.

Comment 3

[Mike Shaver (:shaver emeritus)]

You shame me so effectively!

I just mailed Joe, and pointed him at this bug.  If someone could put the canonical URL to the SoC project description in the URL field above, I would be grateful.  (My internet here is a little tempermental, so crawling the web for it is a daunting prospect.)

I'm on vacation for the rest of the week, but I'll check up on this by the middle of next week at the latest.

Not sure this is in the right component, but I guess it'll do for now.

Comment 4

[:Gijs (he/him)]

(In reply to comment #3)
> If someone could put the
> canonical URL to the SoC project description in the URL field above, I would be
> grateful.

My pleasure.

Obligatory blog entry:
http://gijs.codingo.org/blog/2006/05/barcamp-end-soc-start/

Comment 5

[jhp (no longer active)]

This may be outside the scope of this project, but here are some of the issues that Adam and I have with using JSD within the Java-based AJAX Toolkit Framework.

First, the JSD APIs need to be cleaned up to not be so dependent on Javascript.  There are several methods which _must_ be called from Javascript, so we cannot make use of them.  For example, just starting the debugger service only works from JS (we hacked around it by directly calling jsdASObserver).  Just fixing this would make the debugger more usable in other languages, such as Java, Python, etc.

Adam also had an issue on how stepping is handled.  If I recall correctly, JSD doesn't provide correct stepping support, instead relying on stopping after each instruction and comparing line numbers.

Adam, you had some other issues.  Could you post a comment here?

Comment 6

[James Ross]

(In reply to comment #5)
> Adam also had an issue on how stepping is handled.  If I recall correctly, JSD
> doesn't provide correct stepping support, instead relying on stopping after
> each instruction and comparing line numbers.

JSD steps by the same units as SpiderMonkey - instructions. Venkman wants to step by line, so it does some comparisons in its interrupt handler. I consider JSD to be correct here, especially as Venkman's definition of a line varies depending on its own settings.

Comment 7

[Adam L. Peller]

I understand that JSD is a very low-level API designed to expose SpiderMonkey.  However, features like line stepping are going to be fairly standard where debuggers are concerned (and typically covered in a debug API)  Having to build this logic myself was a bit of a surprise, and looking at Venkman, it seems like there is occasionally some guesswork involved -- I'm told that stepping in Venkman isn't 100% reliable (and probably isn't in my code either!) but I apologize if I'm spreading false rumors :-)

I guess I'm just trying to make a case for raising the bar for JSD and putting some abstractions in it, for reasons of quality and also performance -- crossing in and out of the JSD on a per-instruction basis can be expensive, especially when you're crossing an XPCOM boundary.  The same might be said for implementing conditional breakpoints.

Comment 8

[Adam L. Peller]

This may be way outside the scope of this project, but it would also be nice to have access to object allocation / memory usage / reference counting information, or whatever might be required to track down memory leaks.  At a minimum, something to gauge JS memory usage might be nice.  I imagine showing the actual memory usage including native objects would be impossible, but even just the JS data would be nice.

Similarly, getting access to what's scheduled for window timeouts (and when) might be nice.

Comment 9

[Adam L. Peller]

Also, a plug for another bug I filed: bug 331518.  That covers technical API details.  The feature I want to implement in ATF, which I think would benefit Venkman users as well, is to be able to multitask when you debug.  Today, Venkman uses the Window object (the XPCOM accessor I want in 331518) to set the modal bit and block other windows.  It might be nice to be able to actually debug different windows simultaneously.

Comment 10

[James Ross]

(In reply to comment #7)
> I'm told that stepping in
> Venkman isn't 100% reliable (and probably isn't in my code either!) but I
> apologize if I'm spreading false rumors :-)

The only bugs on record that I know of are when SpiderMonkey's instructions and PC to line map data does not correspond with user expectations; that's got nothing to do with what Venkman or JSD does.

FWIW, I don't object to *adding* a stepToNextLine() or whatever you seem to want, but I dispute that the current behavior is incorrect.

Comment 11

[James Ross]

Also, Bugzilla is the absolute WORST place for any of this. You're just rambling about things, which I don't think helps anyone. Someone do something with the wiki before this gets annoying.

Comment 12

[Joe Hewitt]

I would argue that the step functions should be implemented inside JSD for performance reasons.  Stepping over a large function can result in a gazillion calls to the javascript interrupt, which has to compare the file name and line number each time.  This makes stepping very slow at times.

Comment 13

[:Gijs (he/him)]

Just a note that there is now a separate topic about the JSD API doing by-line hooks instead of by-instruction hooks in the news://news.mozilla.org:119/mozilla.dev.apps.js-debugger newsgroup (alternatively, http://groups.google.com/group/mozilla.dev.apps.js-debugger).

Comment 14

[:Gijs (he/him)]

A bit of an update to keep people posted (and/or spam their inbox):

This week was an exam week for me, so I was very busy with lots of other stuff. What I did do is below.

So, I tried doing some profiling on jsd wednesday afternoon, on my laptop (which runs Ubuntu Linux), and failed because my build failed trying to build jprof. I need to file a bug for that (and probably fix it myself, unless someone knows things that are not in http://lxr.mozilla.org/mozilla/source/tools/jprof/README.html), but unfortunately I was stupid enough to start my build without logging it so I need more info before I do file a bug. I'd CC dbaron, but without more info that's probably not much use, and I'm not sure he's interested in the general topic here. I'll make sure to CC him on bugs I file about jprof specifically, though.

I'll try to give it another shot tomorrow, after upgrading to Ubuntu Dapper tonight. Actually, now that I think about it, I'll probably just make this pc dual boot into that (I have a good 120GB of unused unpartitioned space on this machine, and it's a whole lot faster anyway).

In the meantime, some additional points have been made in the js debugger newsgroup (see my previous comment). Anyone is (very) welcome to contribute to the discussion there. :-)

Lastly, I've had some discussion with Mike about if/how/what should be done about stepping in JSD (as outlined by Silver on the newsgroups), given that we can't change interfaces on branch, and the extension is supposed to work on branch. Interestingly, the policy is once again being discussed on m.d.planning.   I'll try to do some more talking/planning/asking-around tonight or tomorrow.

Comment 15

[:Gijs (he/him)]

OK, profiling stuff is here:
http://wiki.mozilla.org/User:GijsKruitbosch/JS_Debugging/Profiling

Blog entry about what it took:
gijs.codingo.org/blog/2006/06/profiling-venkman-jsd-startup/

Comment 16

[:Gijs (he/him)]

Halfway mark coming up, summary for people I'm not mailing (as opposed to my mentors):

I've been doing all the preparational work up to Thursday the 16th, which is now 10 days ago for me (by a few minutes). At that point I started hacking. I've since, looking at the dependencies, fixed 4 bugs (and duped and wontfixed one), and I've mailed Joe about a problem in Firebug (actually, it's a GC gone amok, but only after *loads* of js errors in Firebug, so I suppose we'd best fix those errors and see if behaviour improves - I can try to figure out how to debug GC crashes when I have time left (I have no idea how to do that, to be honest)).

This means I've crossed off about 3.5 tasks off my list (the half because I'm not sure if the gmail crash is now fixed, I need to check back on that one). Given that I have until August 21, am going to be out of the running for one week (July 9th until and including July 16th) that leaves me with 4 + 31 + 21 - 8 = 48 days. Also taking into account that I'm currently still having classes (I'll be done with classes and one exam resit by July 4), I feel I'm making useful progress. I'll start hacking on Firebug in earnest once a source code repository for it is up, which is something Joe is working hard on. :-)

I think this should do the trick of keeping you informed, if you have questions please mail me directly (but CC Mike and Rob).

Comment 17

[:Gijs (he/him)]

Thought I'd best give a slight update here again... There's about 2 weeks left.

My task list looks like this:
http://wiki.mozilla.org/User:GijsKruitbosch/JS_Debugging/Tasks#Importance-ordered_Task_List

Things I still want to finish before this is over are issues 8, 10 and 12. I have way too many crazy UI ideas for 10 (involving <canvas/> drawn samples of the windows you'd eval to and all that fancy stuff), so I'll need to sort that out, and I'll need to get some people to talk about some theoretics involved in 8. Then I've asked some friends to poke at Ian, and I'll do some of that myself next week, and hopefully things will Be Alright.

Additionally, the top few items all have patches - but they need to be reviewed and I may well have to change (more, we've been busy on them for quite some time now!) stuff. But all in all, we're getting somewhere so I'm pretty happy.

If someone feels ignored or has ideas about stuff that needs to / they want to happen then they're most likely too late by now. If you still feel you can make a reasonable case, send me mail.

I apologize for any errors in the English used above, it's 3am (just finished another patch) and I'm tired. Tomorrow will be a day of rest, uh, I mean, cleaning the house. Then we'll see what happens next week.

Comment 18

[:Gijs (he/him)]

T - 7 days. I've finished what I thought would be my last patch for FireBug during the SoC, but the last Venkman thing I wanted to do (bug 341758) might be a WFM. I need to look into this better, hopefully with some help from Rob and/or James and/or Mike, and decide if/what we can improve in Venkman's startup routine still. If there's nothing we feel we can do, I might consider doing another item off the task list and/or take a different item to fix for this last week, if possible at all.

Comment 19

[:Gijs (he/him)]

Exactly one day left, well, 30 minutes ago anyway - hard data:

Fixed: 11
Patched (FireBug things that are not checked into its svn repository yet (but have so far not had (unfixed) negative comments)): 4
Wontfix: 1
Worksforme: 1

'Resolved': 11 + 4 + 1 + 1 = 17.
Open: 5
Security issues which were over my head: 1 (if we're looking at things really generally - there were multiple issues caused by one problem, AIUI).
Total: 23

So that's half, or 2/3 or 3/4 fixed, depending how you look at it.
See also: http://wiki.mozilla.org/User:GijsKruitbosch/JS_Debugging/Tasks


I've put up some tables with patches, bug numbers where applicable, and some xpi files on http://gijs.codingo.org/mozilla/soc/results.php .
Feel free to play around with them. I'm leaving for a holiday monday evening (CEST), before that I can theoretically see what I can do about late issues you find, but obviously with all the bagpacking and such I can't promise much.

I apologize for the lack of a last-minute rush, as is custom with normal students. In general, I'd blame the fact that I'm taking a holiday, as well as, er, some personal things (http://gijs.codingo.org/blog/2006/08/robbed-hiatus/). I do plan on continuing work as much as possible after returning from holidays, fix any regressions, etc.


If I were to defend what has changed over this summer in terms of having done my job, I suppose I would point at the reduced venkman startup time (on average, you should see a cut of about 1/3 to 1/2 on Venkman startup time). That's bug 341758, for which credit must be given to James Ross and Mike Shaver for helping me figure things out and nudging me to not give up. Then there's the debug button in the slow script dialog (bug 341764), several enhancements to profiling in Venkman as well as quite a few new features for FireBug (watch support, breakpoint listing, evaluation of js in different windows/tabs, inline editing of local variables, etc.) I personally think these should make life easier when debugging javascript in Firefox (and to a lesser extent, SeaMonkey and embedded Gecko runtimes in Eclipse, etc.).
I hope I haven't disappointed too many people (I'll just assume at least a few will be), if I have I'll try to make up for it after the programme ends. Feel free to mail me any questions I haven't answered (sufficiently).

Comment 20

[Mike Shaver (:shaver emeritus)]

Well, I'm certainly not disappointed.  I think the state of JS debugging got a lot better this summer, and even more so once Hewitt has time to review and incorporate your excellent Firebug thanks.

Congratulations on your accomplishments, and thanks, sincerely, for your work!