Talos Regression Ts increase 54% on Android, Nov 30

RESOLVED WONTFIX

Status

()

Firefox for Android
General
RESOLVED WONTFIX
5 years ago
5 years ago

People

(Reporter: gbrown, Assigned: bnicholson)

Tracking

unspecified
x86
Android
Points:
---

Firefox Tracking Flags

(firefox19 unaffected, firefox20- wontfix, firefox21- wontfix, fennec-)

Details

(Reporter)

Description

5 years ago
I cannot find dev-tree-management alerts for this regression on m-i or m-c (missing alerts reported in bug 815816), but the regression can be seen on Ts graphs: http://graphs.mozilla.org/graph.html#tests=[[16,63,20],[16,11,20],[16,52,20]]&sel=none&displayrange=90&datatype=running. 

On m-i, the regression is on Nov 30; I think the first regression is http://hg.mozilla.org/integration/mozilla-inbound/rev/70e354775e1b (5948) vs the previous changeset http://hg.mozilla.org/integration/mozilla-inbound/rev/743f9704f233 (3150).

On m-c, the regression is on Dec 1; http://hg.mozilla.org/mozilla-central/rev/ecdf0e332f17 (5838) vs previous changeset http://hg.mozilla.org/mozilla-central/rev/0e6c4d6047db (3279).

Note that there is a lot of noise in this regression: results both increased and became more erratic; some test runs produce results near the pre-regression range.

An alert for mozilla-aurora was reported in dev-tree-management Digest, Vol 49, Issue 49:

Message: 4
Date: Wed, 09 Jan 2013 13:41:20 -0000
From: nobody@cruncher.build.mozilla.org
To: dev-tree-management@lists.mozilla.org
Subject: <Regression> Mozilla-Aurora - Ts - Android 2.2 (Native) -
        54.3%
Message-ID:
        <20130109134120.532631042ED@cruncher.srv.releng.scl3.mozilla.com>
Content-Type: text/plain; charset="us-ascii"

Regression: Mozilla-Aurora - Ts - Android 2.2 (Native) - 54.3% increase
-----------------------------------------------------------------------
    Previous: avg 3145.322 stddev 356.677 of 30 runs up to revision 1903a40f342d
    New     : avg 4852.908 stddev 298.683 of 5 runs since revision a459eaaea5ae
    Change  : +1707.586 (54.3% / z=4.787)
    Graph   : http://mzl.la/13gF1KU

Changeset range: http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=1903a40f342d&tochange=a459eaaea5ae


S1/S2 testing and eideticker have also shown recent startup time regressions: bug 825612 and bug 827361.
(Reporter)

Updated

5 years ago
tracking-fennec: --- → ?
requesting tracking for a start up perf regression in 20
tracking-fennec: ? → 20+
tracking-firefox20: --- → ?

Comment 2

5 years ago
Brad - do you know who can take a look at this on the mobile side? Not sure if any QA work is necessary (or makes sense) before the egr investigation.
Assignee: nobody → blassey.bugs
status-firefox19: --- → unaffected
status-firefox20: --- → affected
status-firefox21: --- → affected
tracking-firefox20: ? → +
tracking-firefox21: --- → +
Range from m-i based on Geoff's comment 0:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=743f9704f233&tochange=70e354775e1b

I concur, this looks like the range. Ts was really smooth before the regression, now it's noisy (bumpy). There is a lot of OMX lib work in the range for Honeycomb and Gingerbread. The talos runs are for Froyo (2.2) so I don't think the code is firing, but maybe the *.so libs are affecting startup somehow?

Changing the profile folder on 2.2 is also in the range and could be doing something weird.
It looks like bug 768532 might be doing file io in the startup path. Assigning to Brian to investigate
Assignee: blassey.bugs → bnicholson
(Reporter)

Comment 5

5 years ago
Another consideration with bug 768532 is that it moved the database location for Tegra tests from /data to /mnt/sdcard...I/O performance may be different.
(In reply to Brad Lassey [:blassey] from comment #4)
> It looks like bug 768532 might be doing file io in the startup path.
> Assigning to Brian to investigate

(In reply to Geoff Brown [:gbrown] from comment #5)
> Another consideration with bug 768532 is that it moved the database location
> for Tegra tests from /data to /mnt/sdcard...I/O performance may be different.

You're both right, but I think Geoff has more likely found the real cause. The extra file I/O Brad mentions is limited to Android 2.2 and we _could_ remove the check altogether in Firefox 22, assuming all the existing Froyo users have updated using Fx20 and Fx21.
(In reply to Mark Finkle (:mfinkle) from comment #6)

> You're both right, but I think Geoff has more likely found the real cause.
> The extra file I/O Brad mentions is limited to Android 2.2 and we _could_
> remove the check altogether in Firefox 22, assuming all the existing Froyo
> users have updated using Fx20 and Fx21.

Is there any way to reason about how many users might not have updated using 20/21?
Can someone prepare a patch removing the check so we can test out that it actually does help with this regression?
In both Release and Beta, the Google Dev Console shows most releases getting to ~50% in a week or so after release. Uptake starts to slow down after we hit that level.

I will point out that we have ~3.3% of Froyo (Android 2.2) users on Release. Not enough to cause us to treat this issue too critically.

We could make a patch to remove the Froyo "move the database" check. At 3.3% we are close to "remove the Froyo support" IMO.
To sum up, let's not track this bug.
Not tracking then - thanks for the update.
tracking-firefox20: + → -
tracking-firefox21: + → -
Status: NEW → RESOLVED
tracking-fennec: 20+ → -
Last Resolved: 5 years ago
Resolution: --- → WONTFIX
(Assignee)

Updated

5 years ago
status-firefox20: affected → wontfix
status-firefox21: affected → wontfix
You need to log in before you can comment on or make changes to this bug.