Closed Bug 1031097 Opened 10 years ago Closed 7 years ago

Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | (various crashes and failures)

Categories

(Core :: Graphics: CanvasWebGL, defect)

x86_64
Linux
defect
Not set
normal

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: gbrown, Unassigned)

References

Details

(Whiteboard: [some tests disabled on Android 2.3])

Attachments

(2 files)

With the switch to aws, mochitest-gl has several failures and then reboots Android, and then retries... 2338 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-vertex-array-object.html] Test page expected to fail, but all 24 tests passed 2453 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-abs.html] Test page expected to fail, but all 24 tests passed 2747 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-faceforward.html] Test page expected to fail, but all 24 tests passed 3111 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sign.html] Test page expected to fail, but all 24 tests passed 3121 INFO TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sin.html] Test failed - images are different 3137 INFO TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-sin.html] (WebGL test error) 1 failure(s) and 0 timeout(s) 3163 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-step-float.html] Test page expected to fail, but all 24 tests passed 3189 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-step-gentype.html] Test page expected to fail, but all 24 tests passed 3215 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test page expected to fail, but all 24 tests passed 3864 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/limits/gl-max-texture-dimensions.html] Test page expected to fail, but all 3 tests passed 3868 INFO TEST-UNEXPECTED-PASS | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/limits/gl-min-textures.html] Test page expected to fail, but all 2 tests passed DMError: Remote Device Error: unable to connect to 127.0.0.1 after 5 attempts Automation Error: Received unexpected exception while running application Mochi-Remote ERROR | Automation Error: Exception caught while running tests DMError: Remote Device Error: unable to connect to 127.0.0.1 after 5 attempts
Blocks: 979921
...but that did not help at all. Now I notice: 19:06:42 INFO - 1 INFO TEST-START | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html 19:06:42 INFO - 2 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL vendor: Google (VMware, Inc.) 19:06:42 INFO - 3 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL renderer: Android Emulator OpenGL ES Translator (Gallium 0.4 on llvmpipe (LLVM 0x300)) 19:06:42 INFO - 4 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | OS detected as: android 19:06:42 INFO - 5 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Version: null 19:06:42 INFO - 6 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | GL driver detected as: mesa 19:06:42 INFO - 7 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Failing tests file: failing_tests_android.txt 19:06:42 INFO - 8 INFO TEST-INFO | /tests/content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | Skipped tests file: skipped_tests_android.txt On aws, llvmpipe is in the GL renderer string, and that over-rides the emulator logic, selecting the failing/skipped_tests_android.txt instead of the emulator versions.
Second attempt -- force use of the _x86 (emulator) files that I was expecting: https://hg.mozilla.org/mozilla-central/rev/49c42fe2b315
That failed again, but not as bad. I'll try skipping 2 more tests: https://hg.mozilla.org/mozilla-central/rev/be076357691c
These need merging around to the integration branches and uplifted to Aurora.
Flags: needinfo?(cbook)
(In reply to Wes Kocher (:KWierso) from comment #5) > These need merging around to the integration branches and uplifted to Aurora. yeah but we wait for comment #4 also right ?
Flags: needinfo?(cbook)
We are making progress, but the job is still perma-failing: https://tbpl.mozilla.org/php/getParsedLog.php?id=42621848&tree=Mozilla-Central&full=1. I'll verify on try before my next attempt.
(In reply to Geoff Brown [:gbrown] from comment #7) > We are making progress, but the job is still perma-failing: > https://tbpl.mozilla.org/php/getParsedLog.php?id=42621848&tree=Mozilla- > Central&full=1. > > I'll verify on try before my next attempt. btw merged now whats was on m-c :) there are some gl crashes/issues on m-c i guess you are working on
Summary: Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-vertex-array-object.html] Test page expected to fail, but all 24 tests passed → Android 2.3 (aws) content/canvas/test/webgl-conformance/test_webgl_conformance_test_suite.html | (various crashes and failures)
Blocks: 1031364
Skipping a few more tests finally gives a consistently green run: https://tbpl.mozilla.org/?tree=Try&rev=cf9830a55cc4&showall=1 https://hg.mozilla.org/mozilla-central/rev/c90b38c47a1d (I also removed entries from the failed list that are now skipped.)
:jgilbert -- This is a minor follow-up patch, but mostly I want to check in with you to explain what happened here and see if you want any other changes. We introduced test infrastructure changes yesterday that moved Android 2.3 mochitests from ix slaves to aws. I accidentally failed to effectively test Android 2.3 mochitest-gl on aws before the production change. In production, crashes in mochitest-gl caused retries (blues on tbpl) which were not noticed until late in the day. My initial attempt to skip the crashing tests failed because on aws, "llvmpipe" is in the GL renderer string, and that over-rides the emulator logic, selecting the failing/skipped_tests_android.txt files instead of the emulator versions. I hacked around that to select the emulator failing/skipped lists; this patch is an alternative. I don't feel strongly about either approach. Even with the "correct" failing/skipped lists used, there were numerous failures, and unexpected passes, and crashes. Since the unexpected failures and passes do not happen on the Android x86 emulator, I felt I had to skip those. So in the end I added these to skipped_tests_android_x86.txt: +conformance/extensions/oes-vertex-array-object.html +conformance/glsl/functions/glsl-function-abs.html +conformance/glsl/functions/glsl-function-faceforward.html +conformance/glsl/functions/glsl-function-sign.html +conformance/glsl/functions/glsl-function-sin.html +conformance/glsl/functions/glsl-function-step-float.html +conformance/glsl/functions/glsl-function-step-gentype.html +conformance/glsl/functions/glsl-function-smoothstep-float.html +conformance/limits/gl-max-texture-dimensions.html +conformance/limits/gl-min-textures.html +conformance/misc/error-reporting.html +conformance/misc/object-deletion-behaviour.html +conformance/misc/type-conversion-test.html +conformance/reading/read-pixels-test.html +conformance/renderbuffers/framebuffer-object-attachment.html +conformance/textures/gl-teximage.html +conformance/textures/origin-clean-conformance.html +conformance/textures/tex-image-and-sub-image-2d-with-image.html +conformance/textures/tex-image-and-sub-image-2d-with-array-buffer-view.html +conformance/textures/texture-mips.html +conformance/textures/texture-size.html +conformance/textures/texture-size-cube-maps.html This puts the list of skipped Android emulator tests back to a state similar to what it was before my recent cleanup in bug 1020642. :)
Attachment #8447287 - Flags: review?(jgilbert)
Whiteboard: [some tests disabled on Android 2.3]
Component: General → Canvas: WebGL
Product: Firefox for Android → Core
Attachment #8447287 - Flags: review?(jgilbert) → review+
Leaving open for follow-up on disabled tests as/if needed.
Assignee: gbrown → nobody
The minimum Android api level supported by Firefox 48 is 15. Android 2.3 and 4.0 tests no longer run on trunk. This provides an opportunity to clean up Android skip and fail annotations significantly. I tried removing all of the Android annotations, then added back those that caused failures on Android 4.3 until I got a clean try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=25d507df4e51 I made one hand-edit to _webgl-conformance.ini: generate-wrappers-and-manifest.py added e10s to the skip-if in the DEFAULT section; this seemed unrelated to my changes, and, unsure of the history here, I was reluctant to introduce such a change, so I undid that change to the DEFAULT skip-if.
Attachment #8734586 - Flags: review?(jgilbert)
See Also: → 1257948
Comment on attachment 8734586 [details] [diff] [review] update manifests for android mochitest-gl Review of attachment 8734586 [details] [diff] [review]: ----------------------------------------------------------------- We're landing 1.0.3 conformance tests soon (bug 1193526), so this should wait until after that.
Attachment #8734586 - Flags: review?(jgilbert)
Depends on: 1193526
Android 2 is no longer supported.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: