to ship higher quality betas and effiently create stable branch points we need some better tools that tell us when mozilla-central builds are in good shape, and when they are not. We can't use the standard crash per 100 user metric, but we can rely on crash counts and relatively stable updates rates for development build users. I've put together some charts that help tell us about crash volume distribution of builds spanning over the last 10 days for any given day with a couple of different ways to express relative crashiness of recent builds and help us to see if we are trending up or down. This first one shows crash counts for a given day, and which collection of bulds are producing the most crashes http://people.mozilla.com/~chofmann/crash-stats/20101129/floating-build-crash-rate.png and the second one shows the same numbers expressed as a pct. of the total crashes. http://people.mozilla.com/~chofmann/crash-stats/20101129/floating-build-crash-rate-pct.png we need to start producing charts like this and integrate these into places like http://crash-stats.mozilla.com/products/Firefox/versions/4.0b8pre/builds data retrieval is pretty simple. foreach day (list_of_several_days_since_4.0b8pre_started_being_built) print total_crash_count print crash_count_from_blds_from_last_day last2days last3days... print pct_of_crashes_in_builds_from_last_day last2days last3days end then plot this table of numbers in two graphs. I'll attach a .cvs of the table I used to generate the charts above as an example.
for example the graph above shows that oct 13-21 would have been a bad point to create a branch or ship a beta. by splitting the various recent builds and getting good visibiity of what's happening with latest builds we can pick up the actually state of the latest builds two or three days to a week earlier than just looking at the overall crash count for something like the total collection of 4.0b8pre builds
Component: Socorro → General
Product: Webtools → Socorro
we're moving rates and similar overviews to mission control instead
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.