1268747 - Skip more android gl tests.

Status: RESOLVED INCOMPLETE

(Core :: Graphics: CanvasWebGL, defect)

Description

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

After webgl conformance test 1.0.3 landing, there are many intermittent on Android which should be driver bug. I'll find it out and skip it.

Comment 1

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

https://treeherder.mozilla.org/logviewer.html#?job_id=26842731&repo=mozilla-inbound
test_conformance__more__conformance__quickCheckAPI-B1.html

https://treeherder.mozilla.org/logviewer.html#?job_id=26841629&repo=mozilla-inbound
test_conformance__misc__delayed-drawing.html

https://treeherder.mozilla.org/logviewer.html#?job_id=26838478&repo=mozilla-inbound
test_conformance__glsl__samplers__glsl-function-texture2dprojlod.html

https://treeherder.mozilla.org/logviewer.html#?job_id=26764975&repo=mozilla-inbound
test_conformance__misc__null-object-behaviour.html

Comment 2

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Attachment: Skip some gl conformance tests on android due to driver bugs.

Comment 3

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

try https://treeherder.mozilla.org/#/jobs?repo=try&revision=36b69cd4317f

Comment 4

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8751073 [details] [diff] [review]
Skip some gl conformance tests on android due to driver bugs.

Review of attachment 8751073 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/canvas/test/webgl-conf/mochitest-errata.ini
@@ +276,5 @@
> +# Crashes
> +skip-if = (os == 'android')
> +[generated/test_conformance__misc__null-object-behaviour.html]
> +# Crashes
> +skip-if = (os == 'android')

How consistent is this one?
This is a really useful test, and we should try to keep it around.

Comment 5

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

More failures

https://treeherder.mozilla.org/logviewer.html#?job_id=27912450&repo=mozilla-inbound#L1881
test_conformance__more__conformance__constants.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27903373&repo=mozilla-inbound#L1829
test_conformance__glsl__samplers__glsl-function-texture2d-bias.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27900860&repo=mozilla-inbound#L1879
test_conformance__more__conformance__constants.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27897615&repo=mozilla-inbound#L1843
test_conformance__glsl__variables__gl-fragcoord-xy-values.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27880685&repo=mozilla-inbound#L1791
test_conformance__glsl__bugs__compare-loop-index-to-uniform.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27873892&repo=mozilla-inbound#L1874
test_conformance__more__conformance__methods.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27866444&repo=mozilla-inbound#L1800
test_conformance__glsl__constructors__glsl-construct-ivec2.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27864974&repo=mozilla-inbound#L1855
test_conformance__misc__boolean-argument-conversion.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27852974&repo=mozilla-inbound#L1855
test_conformance__glsl__functions__glsl-function-max-float.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27834882&repo=mozilla-inbound#L1026
test_conformance__glsl__variables__gl-fragdata-and-fragcolor.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27821826&repo=mozilla-inbound#L1869
test_conformance__more__conformance__getContext.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27819027&repo=mozilla-inbound#L1777
test_conformance__glsl__bugs__constant-precision-qualifier.html

https://treeherder.mozilla.org/logviewer.html#?job_id=27811621&repo=mozilla-inbound#L1770
test_conformance__glsl__bugs__compare-loop-index-to-uniform.html

Comment 6

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

(In reply to Jeff Gilbert [:jgilbert] from comment #4)
> How consistent is this one?
> This is a really useful test, and we should try to keep it around.

Not seen this in recent try. I'll un-mark this test.

Comment 7

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Attachment: Skip some gl conformance tests on android due to driver bugs.

Comment 8

[Kelsey Gilbert [:jgilbert]]

(In reply to Morris Tseng [:mtseng] from comment #6)
> (In reply to Jeff Gilbert [:jgilbert] from comment #4)
> > How consistent is this one?
> > This is a really useful test, and we should try to keep it around.
> 
> Not seen this in recent try. I'll un-mark this test.

Please post links to your try runs.

Comment 9

[Kelsey Gilbert [:jgilbert]]

The failures you're seeing seem random. Are they consistent?
What's the failure rate for these tests?
Do more tests start failing after removing these ones?

I expect that these crashes are basically random, not due to the tests themselves, but rather some previous test that actually claims to pass.

Comment 10

[Kelsey Gilbert [:jgilbert]]

'suspect', not 'expect'.

Comment 11

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

(In reply to Jeff Gilbert [:jgilbert] from comment #9)
> The failures you're seeing seem random. Are they consistent?
I got those failures from recent inbound failures. Some of them seems consistent. But I'll do more experiment.

> What's the failure rate for these tests?
I'll try to collect failure rate of those tests.

> Do more tests start failing after removing these ones?
I have no idea. I'll send some try for testing this.

> 
> I expect that these crashes are basically random, not due to the tests
> themselves, but rather some previous test that actually claims to pass.
Yap, this might possible. I'll do more testing for this. I'll update once having some findings.

Comment 12

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Comment on attachment 8752645 [details] [diff] [review]
Skip some gl conformance tests on android due to driver bugs.

Clear review since the real problem of those failures are not clear to us now.

Comment 13

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Jeff, as your suspicion, there are other failures when I disabled above tests. So this failure might appear randomly. Do you have any idea to fix this issue? Thanks for advice.

Comment 14

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Here is my try link:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1989e4c49a80&selectedJob=21392249

Comment 15

[Kelsey Gilbert [:jgilbert]]

(In reply to Morris Tseng [:mtseng] from comment #14)
> Here is my try link:
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=1989e4c49a80&selectedJob=21392249

Well, one part of the approach is catching the crash in a debugger with Mesa symbols, ideally under RR. 
We need to know where this is crashing intermittantly in order to figure out what's causing the crash.

Comment 16

[Morris Tseng [:mtseng] [:Morris] (Inactive)]

Assigned to vliu since he had some findings with this.

Comment 17

[Vincent Liu[:vliu]]

(In reply to Morris Tseng [:mtseng] from comment #16)
> Assigned to vliu since he had some findings with this.

Unfortunately the crash I met on my fennec build is different case compared with try result. So the current information can't have any help. :(

I will try to rebase my fennec build and see if there has any clue.