Closed Bug 1323122 Opened 3 years ago Closed 3 years ago

Upgrade 2.0.0 Webgl conformance tests to top-of-tree

Categories

(Core :: Canvas: WebGL, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla53
Tracking Status
firefox51 --- wontfix
firefox52 --- fixed
firefox53 --- fixed

People

(Reporter: vliu, Assigned: vliu)

Details

(Whiteboard: test-only)

Attachments

(8 files, 2 obsolete files)

2.71 MB, patch
jgilbert
: review+
Details | Diff | Splinter Review
7.46 MB, patch
jgilbert
: review+
Details | Diff | Splinter Review
1.34 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
3.02 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
30.50 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
820.93 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
26.59 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
823.89 KB, patch
jgilbert
: review+
Details | Diff | Splinter Review
Since 2.0.0 Webgl conformance tests has some changes, we want to update to top-of-tree.
Assignee: nobody → vliu
Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with html/js files. Could you please have a review? Thanks.
Attachment #8820563 - Flags: review?(jgilbert)
Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with misc files. Could you please have a review? Thanks.
Hi Jeff,
The attached file intends to update 2.0.0 WebGL conformance tests with misc files. Could you please have a review? Thanks.
Attachment #8820567 - Flags: review?(jgilbert)
Attachment #8820565 - Attachment is obsolete: true
Hi Jeff,
Most test cases under image_bitmap_from_image_bitmap and image_bitmap_from_image_data were timeout from try result. Disable them temporarily.
Attachment #8820569 - Flags: review?(jgilbert)
Hi Jeff,
Disable attached test cases because Timeout happens on both Windows and Mac platform.
Attachment #8820576 - Flags: review?(jgilbert)
Comment on attachment 8820576 [details] [diff] [review]
Disable test cases for Timeout.

Review of attachment 8820576 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/canvas/test/webgl-conf/mochitest-errata.ini
@@ +189,5 @@
>  skip-if = (os == 'linux') || (os == 'android')
>  [generated/test_conformance__glsl__bugs__pow-of-small-constant-in-user-defined-function.html]
>  skip-if = (os == 'android')
> +[generated/test_2_conformance2__textures__misc__tex-image-with-bad-args-from-dom-elements.html]
> +skip-if = (os == 'win') || (os == 'mac')

skip-if = 1
Attachment #8820576 - Flags: review?(jgilbert) → review+
Attachment #8820569 - Flags: review?(jgilbert) → review+
Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

From upstream.
Attachment #8820563 - Flags: review?(jgilbert) → review+
Comment on attachment 8820567 [details] [diff] [review]
Update WebGL Conformance Test with misc files.

From upstream.
Attachment #8820567 - Flags: review?(jgilbert) → review+
Don't land this yet. We'll take it after we go into code freeze for 51.
Whiteboard: dont-land
Ok, please land this.
Flags: needinfo?(vliu)
Whiteboard: dont-land
(In reply to Vincent Liu[:vliu] from comment #4)
> Created attachment 8820569 [details] [diff] [review]
> Disable some image_bitmap_* test because most of test are timeout on
> different platforms.
> 
> Hi Jeff,
> Most test cases under image_bitmap_from_image_bitmap and
> image_bitmap_from_image_data were timeout from try result. Disable them
> temporarily.

Hi Jeff,
Since try result still got lots of timed out test cases under deqp, this proposed patch also disable this part based on previous one. Could you please have a review? Thanks
Attachment #8825049 - Flags: review?(jgilbert)
Attachment #8820569 - Attachment is obsolete: true
Attachment #8825049 - Flags: review?(jgilbert) → review+
Hi Jeff,
This patch tried to disable failed/crashed test cases. Could you please have a review? Thanks
Attachment #8825363 - Flags: review?(jgilbert)
Hi Jeff,
This patch tried to add generated test cases. Could you please have a review? Thanks
Attachment #8825364 - Flags: review?(jgilbert)
Attachment #8825363 - Flags: review?(jgilbert) → review+
Attachment #8825364 - Flags: review?(jgilbert) → review+
Pushed by vliu@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1c68d650f3b1
Update WebGL Conformance Test with html/js. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/5e64e0d0d90f
Update WebGL Conformance Test with misc files. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/817b01c4d2fa
Disable some image_bitmap_* test because most of test are timeout or unexpected-error on different platforms. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/5eb44aa0ad72
Disable crashed and failed test case. r=jgilbert
https://hg.mozilla.org/integration/mozilla-inbound/rev/95c02de400f4
Add generated test case r=jgilbert
It's too late as we've built 51 RC. Mark 51 as won't fix. This maybe worth uplifting to Aurora52.
Hi :vliu,
Do you think this is worth uplifting to Aurora52?
Flags: needinfo?(vliu)
(In reply to Gerry Chang [:gchang] from comment #17)
> It's too late as we've built 51 RC. Mark 51 as won't fix. This maybe worth
> uplifting to Aurora52.
> Hi :vliu,
> Do you think this is worth uplifting to Aurora52?

Yes, it is worth uplifting to Aurora52.
Flags: needinfo?(vliu)
We should take this on 51 because it's test-only.
Whiteboard: test-only
We are releasing 51 next Tuesday, what would be the point of uplift it?
(In reply to Sylvestre Ledru [:sylvestre] from comment #20)
> We are releasing 51 next Tuesday, what would be the point of uplift it?

Improved test coverage if we had to chemspill anything. If it's free to take it, I'd rather have it. If it's a real pain to uplift and green up, maybe not, but I'd prefer to have it rather than not.
Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

Approval Request Comment
[Feature/Bug causing the regression]: webgl tests update
[User impact if declined]: reduced coverage for webgl issues compared to nightly53
[Is this code covered by automated tests?]:
[Has the fix been verified in Nightly?]:
[Needs manual test from QE? If yes, steps to reproduce]: 
[List of other uplifts needed for the feature/fix]:
[Is the change risky?]: no
[Why is the change risky/not risky?]: test-only
[String changes made/needed]: none
Attachment #8820563 - Flags: approval-mozilla-beta?
Attachment #8820563 - Flags: approval-mozilla-aurora?
Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

webgl test update, beta52+

Moving approvals from aurora/beta to beta/release since we're past the merge.
Attachment #8820563 - Flags: approval-mozilla-release?
Attachment #8820563 - Flags: approval-mozilla-beta?
Attachment #8820563 - Flags: approval-mozilla-beta+
Attachment #8820563 - Flags: approval-mozilla-aurora?
Comment on attachment 8820563 [details] [diff] [review]
Update WebGL Conformance Test with html/js.

We will release 51 today. It's not critical. Release51-.
Attachment #8820563 - Flags: approval-mozilla-release? → approval-mozilla-release-
Vincent, should we let this one ride the train? thanks
(In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> Vincent, should we let this one ride the train? thanks

I'd tried to send a try on beta. [1] was the test result. From this result, I suggest not uplifting to beta.
Jeff, how do you think?

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee
Flags: needinfo?(vliu) → needinfo?(jgilbert)
(In reply to Vincent Liu[:vliu] from comment #28)
> (In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> > Vincent, should we let this one ride the train? thanks
> 
> I'd tried to send a try on beta. [1] was the test result. From this result,
> I suggest not uplifting to beta.
> Jeff, how do you think?
> 
> [1]:
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee

We are uplifting to beta. Let me know if you need help marking failures.
Flags: needinfo?(jgilbert)
(In reply to Jeff Gilbert [:jgilbert] from comment #29)
> (In reply to Vincent Liu[:vliu] from comment #28)
> > (In reply to Sylvestre Ledru [:sylvestre] from comment #27)
> > > Vincent, should we let this one ride the train? thanks
> > 
> > I'd tried to send a try on beta. [1] was the test result. From this result,
> > I suggest not uplifting to beta.
> > Jeff, how do you think?
> > 
> > [1]:
> > https://treeherder.mozilla.org/#/
> > jobs?repo=try&revision=36fddf4e562b5ccda7e8171c64ddf253346df3ee
> 
> We are uplifting to beta. Let me know if you need help marking failures.

Ok. I will arrange these patches for beta.
Hi Jeff,

The attached patch tried to add fail-if condition from original landed patch in Nightly for some fail tests on beta(52). Could you please have review? Thanks.
Attachment #8833833 - Flags: review?(jgilbert)
Hi Jeff,

The patched were generated test cases based on Comment 31. Can you also have a review? Thanks
Attachment #8833834 - Flags: review?(jgilbert)
Attachment #8833833 - Flags: review?(jgilbert) → review+
Attachment #8833834 - Flags: review?(jgilbert) → review+
You need to log in before you can comment on or make changes to this bug.