Open Bug 914429 Opened 8 years ago Updated 9 months ago

[jsdbg2] We need measurements of Debugger's performance impact

Categories

(Core :: JavaScript Engine, defect)

defect
Not set
normal

Tracking

()

People

(Reporter: jimb, Unassigned)

References

(Blocks 2 open bugs)

Details

Attachments

(2 files)

At the moment, we make no regular measurements of the impact Debugger has on the performance of debuggee JavaScript. At the very least, we should know:

1) What is the performance impact of making a given benchmark a debuggee, but setting no hooks, versus letting it run at SpiderMonkey's fastest speed? Versus the Baseline compiler?

2) How expensive is the act of adding or removing a global as a debuggee, itself?

3) What is the further impact of setting an onEnterFrame hook? An onNewScript hook? etc.
I've put up some very rough code for measuring this here:
https://github.com/jimblandy/benchmark-debugger

The variance here is way too high; I can get the stddev down to around 3%, but here's what I have:

--ion,    debugger, onEnterFrame
warmup
measure
[Stats total: 206.26856884765598s, mean: 0.412537137695312s, stddev: 11%]

--no-ion, debugger, onEnterFrame
warmup
measure
[Stats total: 209.06723315429664s, mean: 0.4181344663085933s, stddev: 12%]

--ion,    debugger, no hooks
warmup
measure
[Stats total: 76.72733569335945s, mean: 0.1534546713867189s, stddev: 10%]

--no-ion, debugger, no hooks
warmup
measure
[Stats total: 76.6684204101562s, mean: 0.1533368408203124s, stddev: 17%]

--ion
warmup
measure
[Stats total: 14.706278320312501s, mean: 0.029412556640625004s, stddev: 11%]

--no-ion
warmup
measure
[Stats total: 41.867698242187494s, mean: 0.08373539648437499s, stddev: 17%]
Here's the full assortment of compilers vs. Debugger arrangements:

--ion,    debugger, onEnterFrame
warmup
measure
[Stats total: 238.5119206542969s, mean: 0.47702384130859377s, stddev: 4%]

--ion,    debugger, no hooks
warmup
measure
[Stats total: 77.43573144531248s, mean: 0.15487146289062498s, stddev: 9%]

--ion
warmup
measure
[Stats total: 20.861109374999998s, mean: 0.04172221875s, stddev: 95%]

--no-ion, debugger, onEnterFrame
warmup
measure
[Stats total: 246.48064135742163s, mean: 0.49296128271484324s, stddev: 1%]

--no-ion, debugger, no hooks
warmup
measure
[Stats total: 78.03183471679682s, mean: 0.15606366943359365s, stddev: 3%]

--no-ion
warmup
measure
[Stats total: 43.8149606933594s, mean: 0.08762992138671881s, stddev: 7%]

--no-ion --no-baseline, debugger, onEnterFrame
warmup
measure
[Stats total: 279.48205371093775s, mean: 0.5589641074218755s, stddev: 11%]

--no-ion --no-baseline, debugger, no hooks
warmup
measure
[Stats total: 138.29736279296876s, mean: 0.2765947255859375s, stddev: 1%]

--no-ion --no-baseline
warmup
measure
[Stats total: 118.26902050781254s, mean: 0.2365380410156251s, stddev: 1%]
That "--ion, no debugger" measurement has a startlingly high standard deviation. I'd like to see the distribution for those times...
Assignee: general → nobody
Here's a new plot, showing the impact after Shu's work on disabling JITs on a per-frame and per-script basis.

Just making something a debuggee imposes no measurable overhead.

The onEnterFrame hook is still expensive, because it affects the behavior of every debuggee script.

Now we're in a position to ask more interesting questions:

- What is the overhead of using Debugger.Frame.prototype.eval and Debugger.Object.prototype.evalInGlobal? (This is the web console's usage pattern. I would expect no lasting impact, once the stack frame in which we performed the evaluation has returned.)

- What is the overhead of making a compartment a debuggee and calling Debugger.Memory.prototype.takeCensus? (I would expect none.)

- What is the overhead of setting Debugger.Memory.prototype.trackingAllocationSites? (I would expect this to still be significant, but perhaps less than before.)
Summary: [jsdbg2] We have no measurements of Debugger's performance impact → [jsdbg2] We need measurements of Debugger's performance impact
You need to log in before you can comment on or make changes to this bug.