Closed
Bug 1031097
Opened 10 years ago
Closed 7 years ago
Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | (various crashes and failures)
Categories
(Core :: Graphics: CanvasWebGL, defect)
Tracking
()
RESOLVED
INCOMPLETE
People
(Reporter: gbrown, Unassigned)
References
Details
(Whiteboard: [some tests disabled on Android 2.3])
Attachments
(2 files)
2.01 KB,
patch
|
jgilbert
:
review+
|
Details | Diff | Splinter Review |
16.99 KB,
patch
|
Details | Diff | Splinter Review |
With the switch to aws, mochitest-gl has several failures and then reboots Android, and then retries...
2338 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-vertex-array-object.html] Test page expected to fail, but all 24 tests passed
2453 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-abs.html] Test page expected to fail, but all 24 tests passed
2747 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-faceforward.html] Test page expected to fail, but all 24 tests passed
3111 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sign.html] Test page expected to fail, but all 24 tests passed
3121 INFO TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sin.html] Test failed - images are different
3137 INFO TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sin.html] (WebGL test error) 1 failure(s) and 0 timeout(s)
3163 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-step-float.html] Test page expected to fail, but all 24 tests passed
3189 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-step-gentype.html] Test page expected to fail, but all 24 tests passed
3215 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test page expected to fail, but all 24 tests passed
3864 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/limits/gl-max-texture-dimensions.html] Test page expected to fail, but all 3 tests passed
3868 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/limits/gl-min-textures.html] Test page expected to fail, but all 2 tests passed
DMError: Remote Device Error: unable to connect to 127.0.0.1 after 5 attempts
Automation Error: Received unexpected exception while running application
Mochi-Remote ERROR | Automation Error: Exception caught while running tests
DMError: Remote Device Error: unable to connect to 127.0.0.1 after 5 attempts
Reporter | ||
Updated•10 years ago
|
Keywords: intermittent-failure
Geoff pushed https://hg.mozilla.org/mozilla-central/rev/6c1da87675d9 to m-c to try to fix this, and I pushed it to Aurora as https://hg.mozilla.org/releases/mozilla-aurora/rev/a101306a4aa6
Reporter | ||
Comment 2•10 years ago
|
||
...but that did not help at all.
Now I notice:
19:06:42 INFO - 1 INFO TEST-START | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html
19:06:42 INFO - 2 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL vendor: Google (VMware, Inc.)
19:06:42 INFO - 3 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL renderer: Android Emulator OpenGL ES Translator (Gallium 0.4 on llvmpipe (LLVM 0x300))
19:06:42 INFO - 4 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | OS detected as: android
19:06:42 INFO - 5 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Version: null
19:06:42 INFO - 6 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL driver detected as: mesa
19:06:42 INFO - 7 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Failing tests file: failing_tests_android.txt
19:06:42 INFO - 8 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Skipped tests file: skipped_tests_android.txt
On aws, llvmpipe is in the GL renderer string, and that over-rides the emulator logic, selecting the failing/skipped_tests_android.txt instead of the emulator versions.
Reporter | ||
Comment 3•10 years ago
|
||
Second attempt -- force use of the _x86 (emulator) files that I was expecting: https://hg.mozilla.org/mozilla-central/rev/49c42fe2b315
Reporter | ||
Comment 4•10 years ago
|
||
That failed again, but not as bad. I'll try skipping 2 more tests:
https://hg.mozilla.org/mozilla-central/rev/be076357691c
These need merging around to the integration branches and uplifted to Aurora.
Flags: needinfo?(cbook)
Comment 6•10 years ago
|
||
(In reply to Wes Kocher (:KWierso) from comment #5)
> These need merging around to the integration branches and uplifted to Aurora.
yeah but we wait for comment #4 also right ?
Flags: needinfo?(cbook)
Reporter | ||
Comment 7•10 years ago
|
||
We are making progress, but the job is still perma-failing: https://tbpl.mozilla.org/php/getParsedLog.php?id=42621848&tree=Mozilla-Central&full=1.
I'll verify on try before my next attempt.
Comment 8•10 years ago
|
||
(In reply to Geoff Brown [:gbrown] from comment #7)
> We are making progress, but the job is still perma-failing:
> https://tbpl.mozilla.org/php/getParsedLog.php?id=42621848&tree=Mozilla-
> Central&full=1.
>
> I'll verify on try before my next attempt.
btw merged now whats was on m-c :) there are some gl crashes/issues on m-c i guess you are working on
Reporter | ||
Updated•10 years ago
|
Summary: Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-vertex-array-object.html] Test page expected to fail, but all 24 tests passed → Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | (various crashes and failures)
Comment 9•10 years ago
|
||
Reporter | ||
Comment 10•10 years ago
|
||
Skipping a few more tests finally gives a consistently green run:
https://tbpl.mozilla.org/?tree=Try&rev=cf9830a55cc4&showall=1
https://hg.mozilla.org/mozilla-central/rev/c90b38c47a1d
(I also removed entries from the failed list that are now skipped.)
Comment 11•10 years ago
|
||
Reporter | ||
Comment 12•10 years ago
|
||
:jgilbert -- This is a minor follow-up patch, but mostly I want to check in with you to explain what happened here and see if you want any other changes.
We introduced test infrastructure changes yesterday that moved Android 2.3 mochitests from ix slaves to aws. I accidentally failed to effectively test Android 2.3 mochitest-gl on aws before the production change. In production, crashes in mochitest-gl caused retries (blues on tbpl) which were not noticed until late in the day.
My initial attempt to skip the crashing tests failed because on aws, "llvmpipe" is in the GL renderer string, and that over-rides the emulator logic, selecting the failing/skipped_tests_android.txt files instead of the emulator versions. I hacked around that to select the emulator failing/skipped lists; this patch is an alternative. I don't feel strongly about either approach.
Even with the "correct" failing/skipped lists used, there were numerous failures, and unexpected passes, and crashes. Since the unexpected failures and passes do not happen on the Android x86 emulator, I felt I had to skip those. So in the end I added these to skipped_tests_android_x86.txt:
+conformance/extensions/oes-vertex-array-object.html
+conformance/glsl/functions/glsl-function-abs.html
+conformance/glsl/functions/glsl-function-faceforward.html
+conformance/glsl/functions/glsl-function-sign.html
+conformance/glsl/functions/glsl-function-sin.html
+conformance/glsl/functions/glsl-function-step-float.html
+conformance/glsl/functions/glsl-function-step-gentype.html
+conformance/glsl/functions/glsl-function-smoothstep-float.html
+conformance/limits/gl-max-texture-dimensions.html
+conformance/limits/gl-min-textures.html
+conformance/misc/error-reporting.html
+conformance/misc/object-deletion-behaviour.html
+conformance/misc/type-conversion-test.html
+conformance/reading/read-pixels-test.html
+conformance/renderbuffers/framebuffer-object-attachment.html
+conformance/textures/gl-teximage.html
+conformance/textures/origin-clean-conformance.html
+conformance/textures/tex-image-and-sub-image-2d-with-image.html
+conformance/textures/tex-image-and-sub-image-2d-with-array-buffer-view.html
+conformance/textures/texture-mips.html
+conformance/textures/texture-size.html
+conformance/textures/texture-size-cube-maps.html
This puts the list of skipped Android emulator tests back to a state similar to what it was before my recent cleanup in bug 1020642. :)
Attachment #8447287 -
Flags: review?(jgilbert)
Reporter | ||
Updated•10 years ago
|
Keywords: intermittent-failure → leave-open
Whiteboard: [some tests disabled on Android 2.3]
Reporter | ||
Updated•10 years ago
|
Component: General → Canvas: WebGL
Product: Firefox for Android → Core
Updated•10 years ago
|
Attachment #8447287 -
Flags: review?(jgilbert) → review+
Reporter | ||
Comment 13•10 years ago
|
||
Comment 14•10 years ago
|
||
Reporter | ||
Comment 15•10 years ago
|
||
Leaving open for follow-up on disabled tests as/if needed.
Assignee: gbrown → nobody
Reporter | ||
Comment 16•9 years ago
|
||
The minimum Android api level supported by Firefox 48 is 15. Android 2.3 and 4.0 tests no longer run on trunk. This provides an opportunity to clean up Android skip and fail annotations significantly. I tried removing all of the Android annotations, then added back those that caused failures on Android 4.3 until I got a clean try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=25d507df4e51
I made one hand-edit to _webgl-conformance.ini: generate-wrappers-and-manifest.py added e10s to the skip-if in the DEFAULT section; this seemed unrelated to my changes, and, unsure of the history here, I was reluctant to introduce such a change, so I undid that change to the DEFAULT skip-if.
Attachment #8734586 -
Flags: review?(jgilbert)
Comment 17•9 years ago
|
||
Comment on attachment 8734586 [details] [diff] [review]
update manifests for android mochitest-gl
Review of attachment 8734586 [details] [diff] [review]:
-----------------------------------------------------------------
We're landing 1.0.3 conformance tests soon (bug 1193526), so this should wait until after that.
Attachment #8734586 -
Flags: review?(jgilbert)
Comment 18•7 years ago
|
||
Android 2 is no longer supported.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Comment 19•7 years ago
|
||
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in
before you can comment on or make changes to this bug.
Description
•