Closed Bug 1020642 Opened 7 years ago Closed 6 years ago

Review failed/skipped webgl-conformance tests for Android

Categories

(Testing :: General, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(firefox32 fixed, firefox33 fixed)

RESOLVED FIXED
mozilla33
Tracking Status
firefox32 --- fixed
firefox33 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

Details

Attachments

(1 file)

content/canvas/test/webgl-conformance contains platform-specific text files for failed and skipped webgl-conformance tests. 

There are variants for "android" and "android_x86". Android 2.3 mochitest-gl accidentally uses "android_x86"; it should probably have its own variant, or use "android". 

Some "android" tests may have been added for Android 2.2 and may be okay on Android 4.0 -- let's review which entries are really needed.

Some tests are marked as both skipped and failing for "android" -- they should probably just be skipped.
Using multiple try runs, I found that many more webgl-conformance tests can be run on Android without triggering test failures -- reflected in these updated lists.

Note that the new entries added to the failed tests lists were previously skipped (they don't crash and seem to fail reliably).

When I reported this bug, I noted that the Android 2.3 Opt and Android 4.2 x86 Opt tests both use the same failed/skipped_tests_android_x86.txt lists. That's because glVendor and glRenderer are identical for these platforms (both run in an emulator). We could differentiate based on kOSVersion, or perhaps some more appropriate property, but it looks like that would only allow a couple of additional tests to be run -- I don't think it warrants a change.
Attachment #8444002 - Flags: review?(jgilbert)
Comment on attachment 8444002 [details] [diff] [review]
remove many entries from Android webgl-conformance failing/skipped lists

Review of attachment 8444002 [details] [diff] [review]:
-----------------------------------------------------------------

Awesome, thank you!
Attachment #8444002 - Flags: review?(jgilbert) → review+
(In reply to Geoff Brown [:gbrown] from comment #1)
> When I reported this bug, I noted that the Android 2.3 Opt and Android 4.2
> x86 Opt tests both use the same failed/skipped_tests_android_x86.txt lists.
> That's because glVendor and glRenderer are identical for these platforms
> (both run in an emulator). We could differentiate based on kOSVersion, or
> perhaps some more appropriate property, but it looks like that would only
> allow a couple of additional tests to be run -- I don't think it warrants a
> change.

Yep. Ideally, we want to be able to set failed/skipped tests by slave configuration.
https://hg.mozilla.org/mozilla-central/rev/ca69915acaf3
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla33
You need to log in before you can comment on or make changes to this bug.