1323122 - Upgrade 2.0.0 Webgl conformance tests to top-of-tree

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Vincent Liu[:vliu]]

Since 2.0.0 Webgl conformance tests has some changes, we want to update to top-of-tree.

Comment 1

[Vincent Liu[:vliu]]

Attachment: Update WebGL Conformance Test with html/js.

Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with html/js files. Could you please have a review? Thanks.

Comment 2

[Vincent Liu[:vliu]]

Attachment: Update WebGL Conformance Test with misc files.

Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with misc files. Could you please have a review? Thanks.

Comment 3

[Vincent Liu[:vliu]]

Attachment: Update WebGL Conformance Test with misc files.

Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with misc files. Could you please have a review? Thanks.

Comment 4

[Vincent Liu[:vliu]]

Attachment: Disable some image_bitmap_* test because most of test are timeout on different platforms.

Hi Jeff,
Most test cases under image_bitmap_from_image_bitmap and image_bitmap_from_image_data were timeout from try result. Disable them temporarily.

Comment 5

[Vincent Liu[:vliu]]

Attachment: Disable test cases for Timeout.

Hi Jeff,
Disable attached test cases because Timeout happens on both Windows and Mac platform.

Comment 6

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8820576 [details] [diff] [review]
Disable test cases for Timeout.

Review of attachment 8820576 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/canvas/test/webgl-conf/mochitest-errata.ini
@@ +189,5 @@
>  skip-if = (os == 'linux') || (os == 'android')
>  [generated/test_conformance__glsl__bugs__pow-of-small-constant-in-user-defined-function.html]
>  skip-if = (os == 'android')
> +[generated/test_2_conformance2__textures__misc__tex-image-with-bad-args-from-dom-elements.html]
> +skip-if = (os == 'win') || (os == 'mac')

skip-if = 1

Comment 7

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

From upstream.

Comment 8

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8820567 [details] [diff] [review]
Update WebGL Conformance Test with misc files.

From upstream.

Comment 9

[Kelsey Gilbert [:jgilbert]]

Don't land this yet. We'll take it after we go into code freeze for 51.

Comment 10

[Kelsey Gilbert [:jgilbert]]

Ok, please land this.

Comment 11

[Vincent Liu[:vliu]]

Attachment: Disable some image_bitmap_* test because most of test are timeout or unexpected-error on different platforms.

(In reply to Vincent Liu[:vliu] from comment #4)
> Created attachment 8820569 [details] [diff] [review]
> Disable some image_bitmap_* test because most of test are timeout on
> different platforms.
> 
> Hi Jeff,
> Most test cases under image_bitmap_from_image_bitmap and
> image_bitmap_from_image_data were timeout from try result. Disable them
> temporarily.

Hi Jeff,
Since try result still got lots of timed out test cases under deqp, this proposed patch also disable this part based on previous one. Could you please have a review? Thanks

Comment 12

[Vincent Liu[:vliu]]

Attachment: Disable crashed and failed test case.

Hi Jeff,
This patch tried to disable failed/crashed test cases. Could you please have a review? Thanks

Comment 13

[Vincent Liu[:vliu]]

Attachment: Add generated test case

Hi Jeff,
This patch tried to add generated test cases. Could you please have a review? Thanks

Comment 14

[Vincent Liu[:vliu]]

try result: https://treeherder.mozilla.org/#/jobs?repo=try&revision=28d9ec105f1d044b0e6af4e735ed722864ed1441

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1c68d650f3b1
Update WebGL Conformance Test with html/js. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/5e64e0d0d90f
Update WebGL Conformance Test with misc files. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/817b01c4d2fa
Disable some image_bitmap_* test because most of test are timeout or unexpected-error on different platforms. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/5eb44aa0ad72
Disable crashed and failed test case. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/95c02de400f4
Add generated test case r=jgilbert

Comment 16

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/1c68d650f3b1
https://hg.mozilla.org/mozilla-central/rev/5e64e0d0d90f
https://hg.mozilla.org/mozilla-central/rev/817b01c4d2fa
https://hg.mozilla.org/mozilla-central/rev/5eb44aa0ad72
https://hg.mozilla.org/mozilla-central/rev/95c02de400f4

Comment 17

[Gerry Chang [:gchang]]

It's too late as we've built 51 RC. Mark 51 as won't fix. This maybe worth uplifting to Aurora52.
Hi :vliu,
Do you think this is worth uplifting to Aurora52?

Comment 18

[Vincent Liu[:vliu]]

(In reply to Gerry Chang [:gchang] from comment #17)
> It's too late as we've built 51 RC. Mark 51 as won't fix. This maybe worth
> uplifting to Aurora52.
> Hi :vliu,
> Do you think this is worth uplifting to Aurora52?

Yes, it is worth uplifting to Aurora52.

Comment 19

[Kelsey Gilbert [:jgilbert]]

We should take this on 51 because it's test-only.

Comment 20

[Sylvestre Ledru [:Sylvestre]]

We are releasing 51 next Tuesday, what would be the point of uplift it?

Comment 21

[Kelsey Gilbert [:jgilbert]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #20)
> We are releasing 51 next Tuesday, what would be the point of uplift it?

Improved test coverage if we had to chemspill anything. If it's free to take it, I'd rather have it. If it's a real pain to uplift and green up, maybe not, but I'd prefer to have it rather than not.

Comment 22

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

Approval Request Comment
[Feature/Bug causing the regression]: webgl tests update
[User impact if declined]: reduced coverage for webgl issues compared to nightly53
[Is this code covered by automated tests?]:
[Has the fix been verified in Nightly?]:
[Needs manual test from QE? If yes, steps to reproduce]: 
[List of other uplifts needed for the feature/fix]:
[Is the change risky?]: no
[Why is the change risky/not risky?]: test-only
[String changes made/needed]: none

Comment 23

[Julien Cristau [:jcristau]]

Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

webgl test update, beta52+

Moving approvals from aurora/beta to beta/release since we're past the merge.

Comment 24

[Gerry Chang [:gchang]]

Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

We will release 51 today. It's not critical. Release51-.

Comment 25

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/6167714a708c
https://hg.mozilla.org/releases/mozilla-beta/rev/03f0e0f2e1ef
https://hg.mozilla.org/releases/mozilla-beta/rev/ec035e4616b9
https://hg.mozilla.org/releases/mozilla-beta/rev/76c5bae7c958
https://hg.mozilla.org/releases/mozilla-beta/rev/5558672d0ed6

Comment 26

[Carsten Book [:Tomcat]]

had to back this out because this caused unexpected passes like 
https://treeherder.mozilla.org/logviewer.html#?job_id=71497329&repo=mozilla-beta
https://treeherder.mozilla.org/logviewer.html#?job_id=71490874&repo=mozilla-beta

Comment 27

[Sylvestre Ledru [:Sylvestre]]

Vincent, should we let this one ride the train? thanks

Comment 28

[Vincent Liu[:vliu]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> Vincent, should we let this one ride the train? thanks

I'd tried to send a try on beta. [1] was the test result. From this result, I suggest not uplifting to beta.
Jeff, how do you think?

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee

Comment 29

[Kelsey Gilbert [:jgilbert]]

(In reply to Vincent Liu[:vliu] from comment #28)
> (In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> > Vincent, should we let this one ride the train? thanks
> 
> I'd tried to send a try on beta. [1] was the test result. From this result,
> I suggest not uplifting to beta.
> Jeff, how do you think?
> 
> [1]:
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee

We are uplifting to beta. Let me know if you need help marking failures.

Comment 30

[Vincent Liu[:vliu]]

(In reply to Jeff Gilbert [:jgilbert] from comment #29)
> (In reply to Vincent Liu[:vliu] from comment #28)
> > (In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> > > Vincent, should we let this one ride the train? thanks
> > 
> > I'd tried to send a try on beta. [1] was the test result. From this result,
> > I suggest not uplifting to beta.
> > Jeff, how do you think?
> > 
> > [1]:
> > https://treeherder.mozilla.org/#/
> > jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee
> 
> We are uplifting to beta. Let me know if you need help marking failures.

Ok. I will arrange these patches for beta.

Comment 31

[Vincent Liu[:vliu]]

Attachment: 0004-Bug-1323122-Uplift-Disable-crashed-and-failed-test-c.patch

Hi Jeff,

The attached patch tried to add fail-if condition from original landed patch in Nightly for some fail tests on beta(52). Could you please have review? Thanks.

Comment 32

[Vincent Liu[:vliu]]

Attachment: 0005-Bug-1323122-Uplift-Add-generated-test-case-r-jgilber.patch

Hi Jeff,

The patched were generated test cases based on Comment 31. Can you also have a review? Thanks

Comment 33

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/a4760079c39b
https://hg.mozilla.org/releases/mozilla-beta/rev/b760d6ecfa57
https://hg.mozilla.org/releases/mozilla-beta/rev/0bb72b3a3aaa
https://hg.mozilla.org/releases/mozilla-beta/rev/b27536a1d853
https://hg.mozilla.org/releases/mozilla-beta/rev/09ced06e2df9