Talos regression SVG-ASAP 4% on Android 4.0, 4% on Android 2.2 on Jan 7 2014

NEW
Assigned to

Status

()

Firefox for Android
Testing
P3
normal
4 years ago
4 months ago

People

(Reporter: gbrown, Assigned: snorp)

Tracking

({perf, regression})

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
Message: 1
Date: Tue, 07 Jan 2014 04:22:55 -0000
From: nobody@cruncher.build.mozilla.org
To: dev-tree-management@lists.mozilla.org
Subject: <Regression> Mozilla-Inbound - SVG-ASAP - Android 4.0.4 -
        4.07%
Message-ID:
        <20140107042255.EF754104657@cruncher.srv.releng.scl3.mozilla.com>
Content-Type: text/plain; charset="us-ascii"

Regression: Mozilla-Inbound - SVG-ASAP - Android 4.0.4 - 4.07% increase
-----------------------------------------------------------------------
    Previous: avg 5885.826 stddev 46.161 of 12 runs up to revision 4a613dc5e5c7
    New     : avg 6125.643 stddev 37.883 of 12 runs since revision acce7a37a66c
    Change  : +239.818 (4.07% / z=5.195)
    Graph   : http://mzl.la/1gcKV4T

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4a613dc5e5c7&tochange=acce7a37a66c

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/acce7a37a66c
    : Brian Smith <brian@briansmith.org> - Bug 898431: Update NSS to NSS 3.15.4 RTM (NSS_3_15_4_RTM), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=898431

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=898431 - Update Mozilla to use NSS that support false start and OCSP GET (likely 3.15.4) (once it's ready)


------------------------------

Message: 2
Date: Tue, 07 Jan 2014 05:04:33 -0000
From: nobody@cruncher.build.mozilla.org
To: dev-tree-management@lists.mozilla.org
Subject: <Regression> Mozilla-Inbound - SVG-ASAP - Android 2.2
        (Native) - 3.96%
Message-ID:
        <20140107050433.2354810452D@cruncher.srv.releng.scl3.mozilla.com>
Content-Type: text/plain; charset="us-ascii"

Regression: Mozilla-Inbound - SVG-ASAP - Android 2.2 (Native) - 3.96% increase
------------------------------------------------------------------------------
    Previous: avg 7130.144 stddev 33.447 of 12 runs up to revision 4a613dc5e5c7
    New     : avg 7412.781 stddev 26.439 of 12 runs since revision acce7a37a66c
    Change  : +282.637 (3.96% / z=8.450)
    Graph   : http://mzl.la/1cKIq5J

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4a613dc5e5c7&tochange=acce7a37a66c

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/acce7a37a66c
    : Brian Smith <brian@briansmith.org> - Bug 898431: Update NSS to NSS 3.15.4 RTM (NSS_3_15_4_RTM), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=898431

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=898431 - Update Mozilla to use NSS that support false start and OCSP GET (likely 3.15.4) (once it's ready)



I checked the logs manually and verified that bug 898431 caused the regression.
Assignee: nobody → brian
tracking-fennec: ? → 29+
Note: Between NSS 3.15.4 RTM and NSS 3.15.5 beta 1, there was nothing (AFAICT) relevant to this test that changed, but somehow I "fixed" the regression and then some. AFAICT there is nothing relevant to performance *at all* that changed between those two versions except code size decreased. Therefore, my initial conclusion is that this test is (overly-)sensitive to code size changes.

Improvement: Mozilla-Inbound - SVG-ASAP - Android 2.2 (Native) - 5.93% decrease
-------------------------------------------------------------------------------
    Previous: avg 7587.084 stddev 124.364 of 12 runs up to revision c305df97cbc6
    New     : avg 7136.903 stddev 39.413 of 12 runs since revision c4bb238b15bb
    Change  : -450.182 (5.93% / z=3.620)
    Graph   : http://mzl.la/19pJu4e

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c305df97cbc6&tochange=c4bb238b15bb

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/c4bb238b15bb
    : Brian Smith <brian@briansmith.org> - Bug 958916: Update to NSS 3.15.5 beta 1 (NSS_3_15_5_BETA1), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=958916

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=958916 - Update Mozilla to use NSS 3.15.5 beta to support VS 2013
Assignee: brian → nobody
Assignee: nobody → snorp
tracking-fennec: 29+ → ---
(Reporter)

Comment 2

4 months ago
This can't be relevant now, can it? Time to wontfix?
Component: General → Testing
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.