916099 - Autophone - s1s2 - Regression in "time to throbber stop" on Aug 28 2013

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: General, defect, P5)

Description

[Lucas Rocha (:lucasr)]

http://phonedash.mozilla.org/#/org.mozilla.fennec/throbberstop/local-blank/2013-08-22/2013-08-29/notcached/noerrorbars/standarderror shows a regression in "Time to throbber stop" for "Local blank page" on Aug 28, for most devices.

Comment 1

[Lucas Rocha (:lucasr)]

Need to narrow down which commit caused the regression. Leaving it nom'd for now.

Comment 2

[William Lachance (:wlach)]

Autophone is more :bc's department.

Comment 3

[Bob Clary [:bc] (inactive)]

Thanks Lucas and thanks wlach!

I'm in the process of obtaining more phones in order to perform bisections on the results. It will probably be after the summit before the phones arrive however. I'll leave the needinfo for tracking this.

Comment 4

[Brad Lassey [:blassey] (use needinfo?)]

Kevin, I don't think we need an autophone set up to get this regression range. Is this something you can pick up?

Comment 5

[Mark Finkle (:mfinkle) (use needinfo?)]

We thought that maybe bug 888482 was to blame, it's in the range, but it was backed out too.

Comment 6

[Bob Clary [:bc] (inactive)]

Attachment: screen shot of local tests

The screen shot shows local testing using an instance of autophone running on linux x86_64 without dummynet with nightly builds for 20130827, 20130828 for everything but the lg revolution which was tested using tinderbox builds from the inbound archive. The lack of dummynet is not significant for this test which only uses the local storage on the device.

The nightly builds on the other devices show a slight increase but the lg shows the regression more dramatically and the use of inbound archive helps narrow this down quite a bit.

It appears the regression began with #2 and continued to increase with #3 after which it was pretty stable.

1 20130827181620 mozilla-inbound 4fa161a24995 Tue Aug 27 2013 (4fa161a24995): 6547±41

2 20130827194118 mozilla-inbound aa33608f75c1 Tue Aug 27 2013 (aa33608f75c1): 6820±26

3 20130827210518 mozilla-inbound 2a3ed74a0c95 Tue Aug 27 2013 (2a3ed74a0c95): 7032±47

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4fa161a24995&tochange=2a3ed74a0c95

Comment 7

[Kevin Brosnan [Ex-Mozilla]]

A number of mobile checkins over that period. I looked at most of them, maybe bug 909400?

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

Bob - How do you want to move ahead with the bisection here? I asked in a different bug, but I don't want to push all of the work to you.

Comment 9

[Bob Clary [:bc] (inactive)]

In that other bug, I've had problems getting the bisection to give a good result. I did a bisection on custom builds on inbound but that only fingered the merge changeset. Trying mozilla-central was indeterminate. I think the date ranges are offset more than I thought on mozilla-central from mozilla-inbound. I re-confirmed the regression on inbound but doing nightly builds on mozilla-central appears to show the regression happened earlier than on inbound. I'm retesting a larger range in mozilla-central using nightly builds at the moment to see if I can narrow down the revisions that bound the regression so I can reliably bisect it with custom builds. Let me get a good result on that bug and take a look at this one. I haven't come up with a reliable set of instructions for bisecting yet. fyi, I'm still officially on pto through friday but will be plinking away at this. I'll keep you informed.

Comment 10

[Bob Clary [:bc] (inactive)]

Attachment: phonedash-2013-08.png

Comment 11

[Bob Clary [:bc] (inactive)]

Attachment: phonedash-2013-08-23.png

Comment 12

[Bob Clary [:bc] (inactive)]

Attachment: sophie-2013-08.png

Comment 13

[Bob Clary [:bc] (inactive)]

Attachment: sophie-2013-08-23.png

Comment 14

[Bob Clary [:bc] (inactive)]

The two phonedash-2013-08* plots show the actual results from tests conducted in August 2013. The two sophie-2013-08* plots show the results reproduced locally attempting to investigate the regression from August 2013.

It is very difficult to distinguish the changesets on or after Aug 28. The regression is readily apparent when looking at the longer term graphs, but when attempting to use inbound for narrowing the range it appears there are several small regressions that add up.

Comment 15

[Brad Lassey [:blassey] (use needinfo?)]

filter on [mass-p5]

Comment 16

[Richard Newman [:rnewman]]

9 months since last activity.