We'd like to be able to determine what is causing the slowdown when the event loop is non-responsive. bug 606574 added some instrumentation that gives us this data, but we should be able to tie into a profiler to see what C++ is executing at those times. I have two possibilities here:
1) Post-processing, run a profiler over the entire session + the responsiveness logging, and correlate the results. I'm not sure if this is possible with the profilers we have to deal with.
2) Modify the instrumentation code to only wait 50ms for a response, and trigger the profiler start/stop internally, so we only sample the non-responsive portions of the app.
I tried working with my existing code + the JS_StopProfiling API here to get Shark samples, but I didn't have any luck. I wanted to use Shark in "WTF" mode, where it constantly records and the "stop profiling" command signals it to save the last N samples, but I couldn't seem to get that to work. I'm guessing I'll have to take the #2 approach I described above and actually modify the instrumentation to wait no more than 50ms and then start profiling.
DTPerformanceSession on OS X should allow us to programmatic start/stop a profiling session and dump it to disk for it to be read in by instrument. Also see Bug 667036. I would like for this to be integrated to dump profiling sessions for each tp5 page load.
Related: see bug 677985. It adds an API for tracking all native functions called from JS. It's a companion to bug 507012, which tracks all calls to JS (and at times pretended to give some info on native calls, but it was never correct nor complete.) Ted, bug 677985 should implement what you wanted when you sent me mail several months ago about missing many native calls.
I've had some success integrating the work from bug 606574 with the profiler from bug 683229. I'm going to take this bug in that direction. If you can think of an alternative let us know.
See my blog post for more details:
If this helps guide this process: I'd like to take this data and display it in an about:responsiveness page, similar to about:memory.
When logging is enabled, you could say things like:
"Show me all event loop hangs from the past hour, limiting stacks to depth 4 and sorting by [sum over all delays [length of this delay] ^ 2]."
That sounds great. We're waiting on the built in profiler to be ported to windows/mac before continuing on this work. We plan on having an extension that would reports hangs and show you profile information.
What work is left to do?
Even if this data were Linux-only, it would be *huge*.
Once we finish the ports well be working on the html/js front end. Ideally we would have stack walking were possible but I think we will post pone that for now to get something usable.
The new profiler (bug 713227 and friends) will fix this.
The profiler landed, and we've been sucessfully using this on all platforms.