958129 - Talos regression SVG-ASAP 4% on Android 4.0, 4% on Android 2.2 on Jan 7 2014

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: Testing, defect, P3)

Description

[Geoff Brown [:gbrown]]

Message: 1
Date: Tue, 07 Jan 2014 04:22:55 -0000
From: [e-mail]
To: [e-mail]
Subject: <Regression> Mozilla-Inbound - SVG-ASAP - Android 4.0.4 -
        4.07%
Message-ID:
        <[e-mail]>
Content-Type: text/plain; charset="us-ascii"

Regression: Mozilla-Inbound - SVG-ASAP - Android 4.0.4 - 4.07% increase
-----------------------------------------------------------------------
    Previous: avg 5885.826 stddev 46.161 of 12 runs up to revision 4a613dc5e5c7
    New     : avg 6125.643 stddev 37.883 of 12 runs since revision acce7a37a66c
    Change  : +239.818 (4.07% / z=5.195)
    Graph   : http://mzl.la/1gcKV4T

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4a613dc5e5c7&tochange=acce7a37a66c

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/acce7a37a66c
    : Brian Smith <[e-mail]> - Bug 898431: Update NSS to NSS 3.15.4 RTM (NSS_3_15_4_RTM), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=898431

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=898431 - Update Mozilla to use NSS that support false start and OCSP GET (likely 3.15.4) (once it's ready)


------------------------------

Message: 2
Date: Tue, 07 Jan 2014 05:04:33 -0000
From: [e-mail]
To: [e-mail]
Subject: <Regression> Mozilla-Inbound - SVG-ASAP - Android 2.2
        (Native) - 3.96%
Message-ID:
        <[e-mail]>
Content-Type: text/plain; charset="us-ascii"

Regression: Mozilla-Inbound - SVG-ASAP - Android 2.2 (Native) - 3.96% increase
------------------------------------------------------------------------------
    Previous: avg 7130.144 stddev 33.447 of 12 runs up to revision 4a613dc5e5c7
    New     : avg 7412.781 stddev 26.439 of 12 runs since revision acce7a37a66c
    Change  : +282.637 (3.96% / z=8.450)
    Graph   : http://mzl.la/1cKIq5J

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4a613dc5e5c7&tochange=acce7a37a66c

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/acce7a37a66c
    : Brian Smith <[e-mail]> - Bug 898431: Update NSS to NSS 3.15.4 RTM (NSS_3_15_4_RTM), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=898431

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=898431 - Update Mozilla to use NSS that support false start and OCSP GET (likely 3.15.4) (once it's ready)



I checked the logs manually and verified that bug 898431 caused the regression.

Comment 1

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Note: Between NSS 3.15.4 RTM and NSS 3.15.5 beta 1, there was nothing (AFAICT) relevant to this test that changed, but somehow I "fixed" the regression and then some. AFAICT there is nothing relevant to performance *at all* that changed between those two versions except code size decreased. Therefore, my initial conclusion is that this test is (overly-)sensitive to code size changes.

Improvement: Mozilla-Inbound - SVG-ASAP - Android 2.2 (Native) - 5.93% decrease
-------------------------------------------------------------------------------
    Previous: avg 7587.084 stddev 124.364 of 12 runs up to revision c305df97cbc6
    New     : avg 7136.903 stddev 39.413 of 12 runs since revision c4bb238b15bb
    Change  : -450.182 (5.93% / z=3.620)
    Graph   : http://mzl.la/19pJu4e

Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c305df97cbc6&tochange=c4bb238b15bb

Changesets:
  * http://hg.mozilla.org/integration/mozilla-inbound/rev/c4bb238b15bb
    : Brian Smith <[e-mail]> - Bug 958916: Update to NSS 3.15.5 beta 1 (NSS_3_15_5_BETA1), r=me
    : http://bugzilla.mozilla.org/show_bug.cgi?id=958916

Bugs:
  * http://bugzilla.mozilla.org/show_bug.cgi?id=958916 - Update Mozilla to use NSS 3.15.5 beta to support VS 2013

Comment 2

[Geoff Brown [:gbrown]]

This can't be relevant now, can it? Time to wontfix?

Comment 3

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.