Last Comment Bug 676831 - [OGL] GL Reftest on Android
: [OGL] GL Reftest on Android
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Graphics (show other bugs)
: unspecified
: ARM Android
: -- normal (vote)
: mozilla9
Assigned To: Benoit Girard (:BenWa)
:
: Milan Sreckovic [:milan]
Mentors:
Depends on: 683195 683198 672704 683199 685694 691349
Blocks: opengl-mobile
  Show dependency treegraph
 
Reported: 2011-08-05 07:52 PDT by Benoit Girard (:BenWa)
Modified: 2016-02-03 10:06 PST (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Flag Failures (41.32 KB, patch)
2011-09-19 13:14 PDT, Benoit Girard (:BenWa)
joe: review+
Details | Diff | Splinter Review

Description Benoit Girard (:BenWa) 2011-08-05 07:52:46 PDT
We need to get test coverage for opengl reftest on android.
Comment 1 Benoit Girard (:BenWa) 2011-08-22 11:17:42 PDT
Trying a test on android to get an idea for what state we are at for reftest + talos:
http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=23fa4d835b88
Comment 2 Benoit Girard (:BenWa) 2011-08-30 12:15:02 PDT
We also made some test to see where we stand for talos. We didn't notice any significant regression. Some results were +2% which are on the high end of the regression.

The results are suspicious so I ran a patch to force a regression to make sure the test were being executed properly. This appears to be the case as test show a ~270% regression on tp4mobile no chrome and fennecbench zoom. I'm not sure why we didn't see a regression with tp4 mobile.
http://hg.mozilla.org/try/rev/4cca6e24dd2d

Everything looks normal with enable OGL and no regression being forced:
http://hg.mozilla.org/try/rev/1d364e6248ac

http://graphs-new.mozilla.org/graph.html#tests=[[84,23,20],[85,23,20],[70,23,20],[69,23,20]]&sel=none&displayrange=30&datatype=running
(If viewing >30 days from now change the displayrange)
Comment 3 Benoit Girard (:BenWa) 2011-09-07 16:44:04 PDT
Ran into remote-reftest misreporting success (doesn't block this):
https://bugzilla.mozilla.org/show_bug.cgi?id=685360
Comment 4 Benoit Girard (:BenWa) 2011-09-19 13:14:02 PDT
Created attachment 560996 [details] [diff] [review]
Flag Failures

For now I flagged all the failing test so that we can enable test coverage for OGL. Of importance is that all canvas/css-gradients/image-element/svg are disable for now.

We will work towards supporting all android reftest but for the time being I suggest that we ignore these test for this configuration and start automated testing to prevent further regressions.

Follow up task will be to remove all occurrences of |Android&&layersOpenGL| in **/reftest.list files.
Comment 5 Joe Drew (not getting mail) 2011-09-19 13:29:00 PDT
Comment on attachment 560996 [details] [diff] [review]
Flag Failures

Review of attachment 560996 [details] [diff] [review]:
-----------------------------------------------------------------

We should definitely have at least two followup bugs; I don't think we can release with any skip-if(Android&&layersOpenGL), especially not if they are skipped because they crash, but we should also try to fix all the failures before releasing.
Comment 6 Ed Morley [:emorley] 2011-09-27 03:28:14 PDT
https://hg.mozilla.org/mozilla-central/rev/5edba813e1e6
Comment 7 Joe Drew (not getting mail) 2012-01-20 15:16:34 PST
I presume those follow-up bugs were filed?

Note You need to log in before you can comment on or make changes to this bug.