Closed Bug 785069 Opened 7 years ago Closed 4 years ago

tcheckerboard2 has regressed substantially since Jul 22nd

Categories

(Core :: Graphics, defect, P5)

x86
macOS
defect

Tracking

()

RESOLVED INCOMPLETE
Tracking Status
firefox17 - wontfix
firefox18 - ---
fennec + ---

People

(Reporter: jrmuizel, Unassigned)

References

Details

(Keywords: regression)

We've gone from around 700 to 2800. Some of this was recovered by bug 771154 but we're still not where we were.

http://graphs.mozilla.org/graph.html#tests=[[204,63,20]]&sel=1342858138614.059,1345724660781.5469&displayrange=90&datatype=running

This was also seen in bug 777357 but nothing actionable came from it.
It would be nice to find the actual changeset that caused the regression. This should be possible using tryserver to bisect the range.
tracking-fennec: --- → ?
tracking-fennec: ? → 17+
Assignee: nobody → chrislord.net
Was this meant to be a 17 tracking nomination? Also, isn't the larger regression from 7/25 or am I reading the chart incorrectly?
Yes this was supposed to be 17 tracking nomination. Both regressions from the 22nd and the 25th should be figured out.
Does this affect Firefox Desktop at all? The graph seems to only point to Android results.
(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #4)
> Does this affect Firefox Desktop at all? The graph seems to only point to
> Android results.

Mobile only, tcheckerboard2 is a mobile test.
A regression window for this bug would still be very useful, has there been a problem with obtaining one?
This is a test that runs on the infrastructure. Need to consult with the A-team about running the test locally.
(In reply to Kevin Brosnan [:kbrosnan] from comment #7)
> This is a test that runs on the infrastructure. Need to consult with the
> A-team about running the test locally.

You should just be able to use try-server
Blocks: 777357
I picked up try access and am in the process of bisecting this.
QA Contact: kbrosnan
The large 38000 pt regression in tchkb2 regression range is.

Due to skipped revisions, the first bad revision could be any of:
changeset:   100765:169ff207ed19
user:        David Zbarsky <dzbarsky@gmail.com>
date:        Fri Jul 27 13:23:44 2012 -0700
summary:     Bug 706179: Add support for animations to the Layers API r=roc, dbaron, cjones

changeset:   100766:a34baed70c1b
user:        David Zbarsky <dzbarsky@gmail.com>
date:        Fri Jul 27 13:23:44 2012 -0700
summary:     Bug 755084 Part 1: Move ElementAnimations, ElementAnimation, AnimationProperty, and AnimationPropertySegment classes to the header file. r=dbaron

changeset:   100767:f9ccdd490bd7
user:        David Zbarsky <dzbarsky@gmail.com>
date:        Fri Jul 27 13:23:44 2012 -0700
summary:     Bug 755084 Part 2: Perform CSS animations of transform and opacity on the compositor r=roc, dbaron

Working on the smaller regression rage from comment 0.
Depends on: 788522
David, can you investigate further here?  We're nearing merge day and 17 will be on Beta so we'll need to know whether there will be a backout or a low-risk forward fix needed for this perf regression.
Assignee: chrislord.net → dzbarsky
Lukas there is no need to do anything for Firefox 17 as this is not enabled on Firefox for Android see bug 788522.
(In reply to Kevin Brosnan [:kbrosnan] from comment #12)
> Lukas there is no need to do anything for Firefox 17 as this is not enabled
> on Firefox for Android see bug 788522.

The animations aren't enabled, but it looks like the regression is related to the change, not the animations. IMO it is still worth investigating this further as the performance reduction (at the time) was pretty substantial as can be seen in the graph on top of this bug:

http://graphs.mozilla.org/graph.html#tests=[[204,63,20]]&sel=1342858138614.059,1345724660781.5469&displayrange=90&datatype=running
(In reply to Kevin Brosnan [:kbrosnan] from comment #10)
> The large 38000 pt regression in tchkb2 regression range is.
> 
> Due to skipped revisions, the first bad revision could be any of:
> changeset:   100765:169ff207ed19
> user:        David Zbarsky <dzbarsky@gmail.com>
> date:        Fri Jul 27 13:23:44 2012 -0700
> summary:     Bug 706179: Add support for animations to the Layers API r=roc,
> dbaron, cjones
> 
> changeset:   100766:a34baed70c1b
> user:        David Zbarsky <dzbarsky@gmail.com>
> date:        Fri Jul 27 13:23:44 2012 -0700
> summary:     Bug 755084 Part 1: Move ElementAnimations, ElementAnimation,
> AnimationProperty, and AnimationPropertySegment classes to the header file.
> r=dbaron
> 
> changeset:   100767:f9ccdd490bd7
> user:        David Zbarsky <dzbarsky@gmail.com>
> date:        Fri Jul 27 13:23:44 2012 -0700
> summary:     Bug 755084 Part 2: Perform CSS animations of transform and
> opacity on the compositor r=roc, dbaron
> 
> Working on the smaller regression rage from comment 0.

I'd like to have a look at this, does the regression correspond to any single one of these patches? There could be a big easy win here. Or not.
As I recall it resulted in failures when pushed to try. Trivial to retest, bad part is that try is so slow.


Pushed a34baed70c1b to try as https://tbpl.mozilla.org/?tree=Try&rev=3d867d108457
Build failed

nsStackWalk.cpp
/usr/bin/ccache /tools/android-ndk-r5c/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-g++ -o nsStackWalk.o -c  -fvisibility=hidden -DXP_LINUX -DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -DIMPL_THEBES  -DSTATIC_EXPORTABLE_JS_API -DEXCLUDE_SKIA_DEPENDENCIES  -DOS_LINUX=1 -DOS_POSIX=1  -D_IMPL_NS_COM -I/builds/slave/try-andrd/build/ipc/chromium/src -I/builds/slave/try-andrd/build/ipc/glue -I../../ipc/ipdl/_ipdlheaders  -I/builds/slave/try-andrd/build/xpcom/base/../build -I/builds/slave/try-andrd/build/xpcom/ds  -I/builds/slave/try-andrd/build/xpcom/base -I. -I../../dist/include  -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nspr -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nss      -fPIC -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wtype-limits -Wempty-body -Wno-ctor-dtor-privacy -Wno-overlength-strings -Wno-invalid-offsetof -Wno-variadic-macros -Wno-long-long -mandroid -fno-short-enums -fno-exceptions -Wno-psabi -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=softfp -I/builds/slave/try-andrd/build/obj-firefox/build/stlport -I/tools/android-ndk-r5c/sources/cxx-stl/stlport/stlport -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -std=gnu++0x -pipe -DNDEBUG -DTRIMMED -g -Os -freorder-blocks -fno-reorder-functions -fomit-frame-pointer  -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -DMOZILLA_CLIENT -include ../../mozilla-config.h -MD -MF .deps/nsStackWalk.o.pp /builds/slave/try-andrd/build/xpcom/base/nsStackWalk.cpp
../../../xpcom/base/nsStackWalk.cpp:156: warning: 'bool IsCriticalAddress(void*)' defined but not used
../../../xpcom/base/nsTraceRefcntImpl.cpp:591: warning: 'PRInt32* GetCOMPtrCount(void*)' defined but not used
In file included from ../../dist/include/BasicLayers.h:9,
                 from ../../dist/include/AndroidBridge.h:26,
                 from /builds/slave/try-andrd/build/xpcom/base/nsSystemInfo.cpp:20:
../../dist/include/Layers.h:543: error: 'nsStyleAnimation' was not declared in this scope
../../dist/include/Layers.h:543: error: template argument 1 is invalid
../../dist/include/Layers.h:544: error: 'nsStyleAnimation' was not declared in this scope
../../dist/include/Layers.h:544: error: template argument 1 is invalid
nsMemoryReporterManager.cpp
/usr/bin/ccache /tools/android-ndk-r5c/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-g++ -o nsMemoryReporterManager.o -c  -fvisibility=hidden -DXP_LINUX -DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -DIMPL_THEBES  -DSTATIC_EXPORTABLE_JS_API -DEXCLUDE_SKIA_DEPENDENCIES  -DOS_LINUX=1 -DOS_POSIX=1  -D_IMPL_NS_COM -I/builds/slave/try-andrd/build/ipc/chromium/src -I/builds/slave/try-andrd/build/ipc/glue -I../../ipc/ipdl/_ipdlheaders  -I/builds/slave/try-andrd/build/xpcom/base/../build -I/builds/slave/try-andrd/build/xpcom/ds  -I/builds/slave/try-andrd/build/xpcom/base -I. -I../../dist/include  -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nspr -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nss      -fPIC -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wtype-limits -Wempty-body -Wno-ctor-dtor-privacy -Wno-overlength-strings -Wno-invalid-offsetof -Wno-variadic-macros -Wno-long-long -mandroid -fno-short-enums -fno-exceptions -Wno-psabi -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=softfp -I/builds/slave/try-andrd/build/obj-firefox/build/stlport -I/tools/android-ndk-r5c/sources/cxx-stl/stlport/stlport -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -std=gnu++0x -pipe -DNDEBUG -DTRIMMED -g -Os -freorder-blocks -fno-reorder-functions -fomit-frame-pointer  -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -DMOZILLA_CLIENT -include ../../mozilla-config.h -MD -MF .deps/nsMemoryReporterManager.o.pp /builds/slave/try-andrd/build/xpcom/base/nsMemoryReporterManager.cpp
FunctionTimer.cpp
/usr/bin/ccache /tools/android-ndk-r5c/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-g++ -o FunctionTimer.o -c  -fvisibility=hidden -DXP_LINUX -DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -DIMPL_THEBES  -DSTATIC_EXPORTABLE_JS_API -DEXCLUDE_SKIA_DEPENDENCIES  -DOS_LINUX=1 -DOS_POSIX=1  -D_IMPL_NS_COM -I/builds/slave/try-andrd/build/ipc/chromium/src -I/builds/slave/try-andrd/build/ipc/glue -I../../ipc/ipdl/_ipdlheaders  -I/builds/slave/try-andrd/build/xpcom/base/../build -I/builds/slave/try-andrd/build/xpcom/ds  -I/builds/slave/try-andrd/build/xpcom/base -I. -I../../dist/include  -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nspr -I/builds/slave/try-andrd/build/obj-firefox/dist/include/nss      -fPIC -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wtype-limits -Wempty-body -Wno-ctor-dtor-privacy -Wno-overlength-strings -Wno-invalid-offsetof -Wno-variadic-macros -Wno-long-long -mandroid -fno-short-enums -fno-exceptions -Wno-psabi -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=softfp -I/builds/slave/try-andrd/build/obj-firefox/build/stlport -I/tools/android-ndk-r5c/sources/cxx-stl/stlport/stlport -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -std=gnu++0x -pipe -DNDEBUG -DTRIMMED -g -Os -freorder-blocks -fno-reorder-functions -fomit-frame-pointer  -isystem /tools/android-ndk-r5c/platforms/android-5/arch-arm/usr/include  -DMOZILLA_CLIENT -include ../../mozilla-config.h -MD -MF .deps/FunctionTimer.o.pp /builds/slave/try-andrd/build/xpcom/base/FunctionTimer.cpp
make[6]: *** [nsSystemInfo.o] Error 1
make[6]: *** Waiting for unfinished jobs....
make[6]: Leaving directory `/builds/slave/try-andrd/build/obj-firefox/xpcom/base'
make[5]: Leaving directory `/builds/slave/try-andrd/build/obj-firefox/xpcom'
make[5]: *** [libs] Error 2
make[4]: *** [libs_tier_platform] Error 2
make[4]: Leaving directory `/builds/slave/try-andrd/build/obj-firefox'
make[3]: *** [tier_platform] Error 2
make[3]: Leaving directory `/builds/slave/try-andrd/build/obj-firefox'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/builds/slave/try-andrd/build/obj-firefox'
make[1]: *** [realbuild] Error 2
make[1]: Leaving directory `/builds/slave/try-andrd/build'
make: *** [build] Error 2
OK - so the regression from 7/25 is no longer happening, and can be tabled. The regression from 7/22 (700 to 2800) from comment 0 still needs to be nailed down. This needs gfx team attention, since I think you all will be able to find the regressing bug most easily.

JP - can you assign?
Assignee: dzbarsky → jpr
Keywords: qawanted
Assignee: jpr → bgirard
(In reply to Kevin Brosnan [:kbrosnan] from comment #17)
> Pushed f9ccdd490bd7 to try as
> https://tbpl.mozilla.org/?tree=Try&rev=e0247be88f2f

tcheck2 failed, re-trigerring.
(In reply to Benoit Girard (:BenWa) from comment #19)
> (In reply to Kevin Brosnan [:kbrosnan] from comment #17)
> > Pushed f9ccdd490bd7 to try as
> > https://tbpl.mozilla.org/?tree=Try&rev=e0247be88f2f
> 
> tcheck2 failed, re-trigerring.

I think Kevin was still trying to find the 7/25 regression (which no longer exists), not the 7/22 one. Not sure this try build helps us.
At this point it's very unlikely we'll find a fix for FF17. Untracking.
(In reply to Alex Keybl [:akeybl] from comment #21)
> At this point it's very unlikely we'll find a fix for FF17. Untracking.

Shouldn't we still evaluate whether this is substantial enough to block the release? Where's the bar for how much worse a new release can be before we won't ship it? It feels like a bit of a failure to say that we can't solve this for FF17 so we'll ignore it till FF18.
(In reply to Jeff Muizelaar [:jrmuizel] from comment #23)
> (In reply to Alex Keybl [:akeybl] from comment #21)
> > At this point it's very unlikely we'll find a fix for FF17. Untracking.
> 
> Shouldn't we still evaluate whether this is substantial enough to block the
> release? Where's the bar for how much worse a new release can be before we
> won't ship it? It feels like a bit of a failure to say that we can't solve
> this for FF17 so we'll ignore it till FF18.

Tracking bugs are not blockers, and QA has not called out this regression as having major user impact in their daily testing. We have to be realistic. Just because a bug isn't tracking doesn't mean it can't be fixed, nominated, and uplifted.
tracking-fennec: 17+ → +
Assignee: bgirard → nobody
filter on [mass-p5]
Priority: -- → P5
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.