Closed Bug 916099 Opened 8 years ago Closed 7 years ago

Autophone - s1s2 - Regression in "time to throbber stop" on Aug 28 2013

Categories

(Firefox for Android Graveyard :: General, defect, P5)

All
Android
defect

Tracking

(fennec+)

RESOLVED INCOMPLETE
Tracking Status
fennec + ---

People

(Reporter: lucasr, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: regression)

Attachments

(5 files)

http://phonedash.mozilla.org/#/org.mozilla.fennec/throbberstop/local-blank/2013-08-22/2013-08-29/notcached/noerrorbars/standarderror shows a regression in "Time to throbber stop" for "Local blank page" on Aug 28, for most devices.
tracking-fennec: --- → ?
Need to narrow down which commit caused the regression. Leaving it nom'd for now.
Flags: needinfo?(wlachance)
Autophone is more :bc's department.
Flags: needinfo?(wlachance) → needinfo?(bclary)
Thanks Lucas and thanks wlach!

I'm in the process of obtaining more phones in order to perform bisections on the results. It will probably be after the summit before the phones arrive however. I'll leave the needinfo for tracking this.
Kevin, I don't think we need an autophone set up to get this regression range. Is this something you can pick up?
tracking-fennec: ? → 26+
Flags: needinfo?(kbrosnan)
We thought that maybe bug 888482 was to blame, it's in the range, but it was backed out too.
The screen shot shows local testing using an instance of autophone running on linux x86_64 without dummynet with nightly builds for 20130827, 20130828 for everything but the lg revolution which was tested using tinderbox builds from the inbound archive. The lack of dummynet is not significant for this test which only uses the local storage on the device.

The nightly builds on the other devices show a slight increase but the lg shows the regression more dramatically and the use of inbound archive helps narrow this down quite a bit.

It appears the regression began with #2 and continued to increase with #3 after which it was pretty stable.

1 20130827181620 mozilla-inbound 4fa161a24995 Tue Aug 27 2013 (4fa161a24995): 6547±41

2 20130827194118 mozilla-inbound aa33608f75c1 Tue Aug 27 2013 (aa33608f75c1): 6820±26

3 20130827210518 mozilla-inbound 2a3ed74a0c95 Tue Aug 27 2013 (2a3ed74a0c95): 7032±47

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4fa161a24995&tochange=2a3ed74a0c95
Flags: needinfo?(bclary)
A number of mobile checkins over that period. I looked at most of them, maybe bug 909400?
Summary: Regression in "time to throbber stop" on Aug 28 2013 → Autophone - s1s2 - Regression in "time to throbber stop" on Aug 28 2013
Bob - How do you want to move ahead with the bisection here? I asked in a different bug, but I don't want to push all of the work to you.
In that other bug, I've had problems getting the bisection to give a good result. I did a bisection on custom builds on inbound but that only fingered the merge changeset. Trying mozilla-central was indeterminate. I think the date ranges are offset more than I thought on mozilla-central from mozilla-inbound. I re-confirmed the regression on inbound but doing nightly builds on mozilla-central appears to show the regression happened earlier than on inbound. I'm retesting a larger range in mozilla-central using nightly builds at the moment to see if I can narrow down the revisions that bound the regression so I can reliably bisect it with custom builds. Let me get a good result on that bug and take a look at this one. I haven't come up with a reliable set of instructions for bisecting yet. fyi, I'm still officially on pto through friday but will be plinking away at this. I'll keep you informed.
The two phonedash-2013-08* plots show the actual results from tests conducted in August 2013. The two sophie-2013-08* plots show the results reproduced locally attempting to investigate the regression from August 2013.

It is very difficult to distinguish the changesets on or after Aug 28. The regression is readily apparent when looking at the longer term graphs, but when attempting to use inbound for narrowing the range it appears there are several small regressions that add up.
tracking-fennec: 26+ → +
filter on [mass-p5]
Priority: -- → P5
9 months since last activity.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Flags: needinfo?(kbrosnan)
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.