We need a tool to analyze add-on performance impact during add-on review and testing. We need objective data on memory and performance impact in various areas, including the following: • Startup time • New window time • Page load time • GC/CC time • GC/CC frequency • Memory consumption • Memory fragmentation • Compartment counts/zombie compartments Some of these things will need better support from Firefox (e.g., I don't know a way to get a useful startup timestamp at millisecond precision), others we already have the interfaces we need. The add-on will have to be accompanied by testing procedures, and will need to store baseline values for a profile before any extensions have been installed. I'm looking for input on what other areas are most crucial for testing.
Although I think getting this data during local testing is great, I should point out that if you really want to catch problems, you should be looking at telemetry data, too. It takes a strong statistical analysis to tease out whether slow startup time is due to an add-on, and which add-on it might be due to, but that's the best source of red flags indicating "this add-on is problematic in the real world".
I agree that telemetry data is important. It's also a bit harder to work with and only lets us catch problems once they emerge in the wild. I'd like use to be able to catch the most egregious issues during review. I'll file bugs for obtaining relevant telemetry data once this is complete.
Yeah, we've seen over and over that the most egregious cases of poor performance can be detected with even a little bit of local testing, so it would be great if we could have a little more coverage there without causing the AMO reviewers too much extra work.
You can use about:startup extension for startup timings.
(In reply to Girish Sharma [:Optimizer] from comment #4) > You can use about:startup extension for startup timings. I'm interested in having a single extension that collects all of the relevant data in one place, though about:startup's method of collecting startup time must be useful.
Yes, I know, but like other things, CC/GC timings, frequency etc, you [will need]/[can take] help from addons like about:ccdump, or memchaser, so the same goes for startup timings, you can take code help from about:startup and easily develop that part of your add-on.
Ah, isn't amalgamating code from random add-ons that sort of do what you want how we wind up with things like Tab Mix Plus? Let's try to avoid that. Again, for the moment I'm interested in what things we want to track, not how to track them. Implementation is not going to be a problem.
a year ago