904088 - Autophone - s1s2 - Regression in "time to throbber stop" on Aug 9, 2013

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: General, defect)

Description

[Geoff Brown [:gbrown]]

http://phonedash.mozilla.org/#/org.mozilla.fennec/throbberstop/local-blank/2013-08-05/2013-08-12/notcached/noerrorbars/standarderror shows a regression in "Time to throbber stop" for "Local blank page" on Aug 9, for several devices:

00_23_76_96_cc_6f_nexus-one
00_23_76_b1_92_39_nexus-one
44_a7_cf_ea_f9_73_lg-revolution
78_d6_f0_cf_8d_11_nexus-s
78_d6_f0_cf_d2_17_nexus-s
c8_aa_21_ac_0c_b5_droid-pro

For example, for 00_23_76_b1_92_39_nexus-one compare:

Thu Aug 08 2013 (55b6b3b78d37): 5944±26
Fri Aug 09 2013 (e33c2011643e): 8417±92

Comment 1

[Brad Lassey [:blassey] (use needinfo?)]

fixed by back out (and the relanding didn't seem to get the same regression)

Comment 2

[Bob Clary [:bc] (inactive)]

There is a marked regression in throbber stop (and therefore total throbber) time on 2013-08-09.

http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/local-blank/2013-08-08/2013-08-11/notcached/noerrorbars/standarderror

http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/local-twitter/2013-08-08/2013-08-11/notcached/noerrorbars/standarderror

http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/remote-blank/2013-08-08/2013-08-11/notcached/noerrorbars/standarderror

http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/remote-twitter/2013-08-08/2013-08-11/notcached/noerrorbars/standarderror

The regression range is
2013-08-09 16:25:40 (2904a317c74b) to 2013-08-09 16:30:41 (0c2ce8aaa005)

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=2904a317c74b&tochange=0c2ce8aaa005

This occurs on all tests local and remote though local is more pronounced.

Significant regression:

00_23_76_96_cc_6f_nexus-one Android 2.3.6
00_23_76_b1_92_39_nexus-one Android 2.3.6
78_d6_f0_cf_8d_11_nexus-s   Android 2.3.6
78_d6_f0_cf_d2_17_nexus-s   Android 2.3.6
c8_aa_21_ac_0c_b5_droid-pro Android 2.3.3

Insignificant or undetectable regression:

98_0c_82_33_ec_8d_samsung-gs2 Android 2.3.6
20_64_32_4f_e9_4f_samsung-gs2 Android 4.0.3
20_64_32_21_35_70_samsung-gs2 Android 4.0.4
5c_0a_5b_b4_31_32_samsung-gs3 Android 4.0.4
38_aa_3c_1e_a8_ee_samsung-gs3 Android 4.0.4
80_96_b1_ee_55_e0_atrix       Android 4.0.4

I retested using 90_21_55_09_87_95_nexus-one Android 2.3.7 using inbound archive and the regression is clear. See attached screen shot.

Looking at the graph for the remainder of August, http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/local-twitter/2013-08-08/2013-08-31/notcached/noerrorbars/standarderror this does not appear to be fixed. 

I re-testd the nightly runs for August with 90_21_55_09_87_95_nexus-one and did not see this specific regression was fixed.

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: 90_21_55_09_87_95_nexus-one Android 2.3.7 screenshot phonedash-2013-08-09.png

Comment 4

[Mark Finkle (:mfinkle) (use needinfo?)]

What can we do to help narrow down the regression range? I can try to make some builds. How do we run the test itself? I can post some builds too, if that makes it easier to run the tests.

Comment 5

[Bob Clary [:bc] (inactive)]

Yeah, inbound archive did a great job of narrowing it down but only to that merge with 160 changesets. Unfortunately the phonedash graphing is based on buildid/builddate so naively using the system to bisect and graph the results from custom builds won't work.

The test files aren't currently available anywhere but I will post them to phonedash later today. I can do builds pretty fast here so in the mean time I will attempt to to bisect the range and see what the procedure looks like.

Comment 6

[Bob Clary [:bc] (inactive)]

I tried a bisect on inbound but it only fingered the 142104:0c2ce8aaa005 merge change set again. Running the test locally and reporting to a local instance of phonedash was helpful since I could in this case easily distinguish the 'good' result. I'll try to bisect mozilla-central this time.

Comment 7

[Bob Clary [:bc] (inactive)]

Attachment: phonedash-inbound-vs-mozilla-central.png

Bisecting m-c finally gives:

The first bad revision is:
changeset:   141852:e33c2011643e
parent:      141809:55b6b3b78d37
parent:      141851:e881258c3ac5
user:        Matt Brubeck <[e-mail]>
date:        Thu Aug 08 21:54:44 2013 -0700
summary:     Merge m-c to fx-team

The screenshot shows two runs of mozilla-inbound which agree with each other and one run of mozilla central nightlies which shows the regression occurred on mozilla-central before it occurred on inbound. I lost some time trying to bisect with the wrong date range on mozilla-central.

If the regressor isn't obvious, then to go further we will need to bisect fx-team. Just looking at the bug summaries, I wonder if bug 853844 isn't relevant.

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Bob Clary [:bc:] from comment #7)

> If the regressor isn't obvious, then to go further we will need to bisect
> fx-team. Just looking at the bug summaries, I wonder if bug 853844 isn't
> relevant.

Great work! Bug 853844 is certainly a good candidate if for no other reason than we are changing code in the very methods that control "throbber start/stop".

Looks like we are firmly committed to removing the spinner (See Bug 917896), so that might be the best approach for "fixing" this regression.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

almost 2 years with no activity, can we close this as wontfix?

Comment 10

[:Margaret Leibovic]

Based on comment 8, it looks like this should have been addressed by bug 917896.

But at this point, there's not much actionable here. Let's just close this.