Autophone - s1s2 - Regression in "time to throbber stop" on Aug 28 2013

RESOLVED INCOMPLETE

Status

()

Firefox for Android
General
P5
normal
RESOLVED INCOMPLETE
5 years ago
4 years ago

People

(Reporter: lucasr, Unassigned)

Tracking

(Blocks: 1 bug, {regression})

unspecified
All
Android
regression
Points:
---

Firefox Tracking Flags

(fennec+)

Details

Attachments

(5 attachments)

(Reporter)

Description

5 years ago
http://phonedash.mozilla.org/#/org.mozilla.fennec/throbberstop/local-blank/2013-08-22/2013-08-29/notcached/noerrorbars/standarderror shows a regression in "Time to throbber stop" for "Local blank page" on Aug 28, for most devices.
(Reporter)

Updated

5 years ago
tracking-fennec: --- → ?
(Reporter)

Comment 1

5 years ago
Need to narrow down which commit caused the regression. Leaving it nom'd for now.
Flags: needinfo?(wlachance)
Keywords: regressionwindow-wanted
Autophone is more :bc's department.
Flags: needinfo?(wlachance) → needinfo?(bclary)

Comment 3

5 years ago
Thanks Lucas and thanks wlach!

I'm in the process of obtaining more phones in order to perform bisections on the results. It will probably be after the summit before the phones arrive however. I'll leave the needinfo for tracking this.
Kevin, I don't think we need an autophone set up to get this regression range. Is this something you can pick up?
tracking-fennec: ? → 26+
Flags: needinfo?(kbrosnan)
We thought that maybe bug 888482 was to blame, it's in the range, but it was backed out too.

Comment 6

5 years ago
Created attachment 8341954 [details]
screen shot of local tests

The screen shot shows local testing using an instance of autophone running on linux x86_64 without dummynet with nightly builds for 20130827, 20130828 for everything but the lg revolution which was tested using tinderbox builds from the inbound archive. The lack of dummynet is not significant for this test which only uses the local storage on the device.

The nightly builds on the other devices show a slight increase but the lg shows the regression more dramatically and the use of inbound archive helps narrow this down quite a bit.

It appears the regression began with #2 and continued to increase with #3 after which it was pretty stable.

1 20130827181620 mozilla-inbound 4fa161a24995 Tue Aug 27 2013 (4fa161a24995): 6547±41

2 20130827194118 mozilla-inbound aa33608f75c1 Tue Aug 27 2013 (aa33608f75c1): 6820±26

3 20130827210518 mozilla-inbound 2a3ed74a0c95 Tue Aug 27 2013 (2a3ed74a0c95): 7032±47

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4fa161a24995&tochange=2a3ed74a0c95
Flags: needinfo?(bclary)

Updated

5 years ago
Keywords: regressionwindow-wanted → regression
A number of mobile checkins over that period. I looked at most of them, maybe bug 909400?

Updated

5 years ago
Blocks: 953342

Updated

5 years ago
Summary: Regression in "time to throbber stop" on Aug 28 2013 → Autophone - s1s2 - Regression in "time to throbber stop" on Aug 28 2013
Bob - How do you want to move ahead with the bisection here? I asked in a different bug, but I don't want to push all of the work to you.

Comment 9

5 years ago
In that other bug, I've had problems getting the bisection to give a good result. I did a bisection on custom builds on inbound but that only fingered the merge changeset. Trying mozilla-central was indeterminate. I think the date ranges are offset more than I thought on mozilla-central from mozilla-inbound. I re-confirmed the regression on inbound but doing nightly builds on mozilla-central appears to show the regression happened earlier than on inbound. I'm retesting a larger range in mozilla-central using nightly builds at the moment to see if I can narrow down the revisions that bound the regression so I can reliably bisect it with custom builds. Let me get a good result on that bug and take a look at this one. I haven't come up with a reliable set of instructions for bisecting yet. fyi, I'm still officially on pto through friday but will be plinking away at this. I'll keep you informed.

Comment 10

5 years ago
Created attachment 8357248 [details]
phonedash-2013-08.png

Comment 11

5 years ago
Created attachment 8357250 [details]
phonedash-2013-08-23.png

Comment 12

5 years ago
Created attachment 8357252 [details]
sophie-2013-08.png

Comment 13

5 years ago
Created attachment 8357253 [details]
sophie-2013-08-23.png

Comment 14

5 years ago
The two phonedash-2013-08* plots show the actual results from tests conducted in August 2013. The two sophie-2013-08* plots show the results reproduced locally attempting to investigate the regression from August 2013.

It is very difficult to distinguish the changesets on or after Aug 28. The regression is readily apparent when looking at the longer term graphs, but when attempting to use inbound for narrowing the range it appears there are several small regressions that add up.
tracking-fennec: 26+ → +
filter on [mass-p5]
Priority: -- → P5
9 months since last activity.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → INCOMPLETE
Flags: needinfo?(kbrosnan)
You need to log in before you can comment on or make changes to this bug.