1319801 - Move Linux64 mochitest-gl to Ubuntu 16.04

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Looks like there's some annotation updates needed for unexpected passes, but generally working OK.

https://treeherder.mozilla.org/logviewer.html#?job_id=31673269&repo=try
https://treeherder.mozilla.org/logviewer.html#?job_id=31673201&repo=try
https://treeherder.mozilla.org/logviewer.html#?job_id=31673197&repo=try

Also subject to the LSAN leaks some of the other suites have also seen. Will probably also give it the mochitest-media treatment on ASAN for now.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Should be fairly straightforward to do this, but I don't have time to work on it right now. Especially now that bug 1304156 is fixed.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

a handful of unexpected-pass, and one unexpected-fail [0]:
[task 2016-12-29T20:25:43.530374Z] 20:25:43     INFO - TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | Error: Shader program linking failed:
[task 2016-12-29T20:25:43.531158Z] 20:25:43     INFO - Shader program info log:
[task 2016-12-29T20:25:43.531223Z] 20:25:43     INFO - Must have a compiled vertex shader attached.
[task 2016-12-29T20:25:43.531265Z] 20:25:43     INFO - 
[task 2016-12-29T20:25:43.531307Z] 20:25:43     INFO - Vert shader log:
[task 2016-12-29T20:25:43.532344Z] 20:25:43     INFO - 0:2(12): error: extension `GL_ARB_gpu_shader5' unsupported in vertex shader
[task 2016-12-29T20:25:43.532394Z] 20:25:43     INFO - 
[task 2016-12-29T20:25:43.534181Z] 20:25:43     INFO - 
[task 2016-12-29T20:25:43.534999Z] 20:25:43     INFO - Frag shader log:
[task 2016-12-29T20:25:43.536092Z] 20:25:43     INFO - 0:2(12): error: extension `GL_ARB_gpu_shader5' unsupported in fragment shader
[task 2016-12-29T20:25:43.538539Z] 20:25:43     INFO -  
[task 2016-12-29T20:25:43.539348Z] 20:25:43     INFO -     errorFunc@dom/canvas/test/webgl-mochitest/webgl-util.js:77:9
[task 2016-12-29T20:25:43.540205Z] 20:25:43     INFO -     warning@dom/canvas/test/webgl-mochitest/webgl-util.js:31:5
[task 2016-12-29T20:25:43.541032Z] 20:25:43     INFO -     createProgramByIds@dom/canvas/test/webgl-mochitest/webgl-util.js:154:7
[task 2016-12-29T20:25:43.541842Z] 20:25:43     INFO -     @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:79:16
[task 2016-12-29T20:25:43.543900Z] 20:25:43     INFO -     run@dom/canvas/test/webgl-mochitest/webgl-util.js:82:7
[task 2016-12-29T20:25:43.544736Z] 20:25:43     INFO -     Async*withWebGL2@dom/canvas/test/webgl-mochitest/webgl-util.js:92:7
[task 2016-12-29T20:25:43.545553Z] 20:25:43     INFO -     @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:35:3
[task 2016-12-29T20:25:43.546640Z] 20:25:43     INFO - Not taking screenshot here: see the one that was previously logged
[task 2016-12-29T20:25:43.548654Z] 20:25:43     INFO - TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | Program linking should succeed. 
[task 2016-12-29T20:25:43.549529Z] 20:25:43     INFO -     @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:81:7
[task 2016-12-29T20:25:43.550497Z] 20:25:43     INFO -     run@dom/canvas/test/webgl-mochitest/webgl-util.js:82:7
[task 2016-12-29T20:25:43.552849Z] 20:25:43     INFO -     Async*withWebGL2@dom/canvas/test/webgl-mochitest/webgl-util.js:92:7
[task 2016-12-29T20:25:43.553850Z] 20:25:43     INFO -     @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:35:3
[task 2016-12-29T20:25:43.554910Z] 20:25:43     INFO - MEMORY STAT | vsize 1536MB | residentFast 376MB | heapAllocated 202MB
[task 2016-12-29T20:25:43.556150Z] 20:25:43     INFO - TEST-OK | dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | took 145ms


looks most likely as if we do not have GL_ARB_gpu_shader5 available- not sure if that is available in 16.04. Jeff- could you help determine what we need to do with this specific failure?

[0] https://public-artifacts.taskcluster.net/I-VbMUlSSKagd7kyVwv-eA/0/public/logs/live_backing.log

Comment 3

[Kelsey Gilbert [:jgilbert]]

(In reply to Joel Maher ( :jmaher) from comment #2)
> a handful of unexpected-pass, and one unexpected-fail [0]:
> [task 2016-12-29T20:25:43.530374Z] 20:25:43     INFO - TEST-UNEXPECTED-FAIL
> | dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | Error:
> Shader program linking failed:
> [task 2016-12-29T20:25:43.531158Z] 20:25:43     INFO - Shader program info
> log:
> [task 2016-12-29T20:25:43.531223Z] 20:25:43     INFO - Must have a compiled
> vertex shader attached.
> [task 2016-12-29T20:25:43.531265Z] 20:25:43     INFO - 
> [task 2016-12-29T20:25:43.531307Z] 20:25:43     INFO - Vert shader log:
> [task 2016-12-29T20:25:43.532344Z] 20:25:43     INFO - 0:2(12): error:
> extension `GL_ARB_gpu_shader5' unsupported in vertex shader
> [task 2016-12-29T20:25:43.532394Z] 20:25:43     INFO - 
> [task 2016-12-29T20:25:43.534181Z] 20:25:43     INFO - 
> [task 2016-12-29T20:25:43.534999Z] 20:25:43     INFO - Frag shader log:
> [task 2016-12-29T20:25:43.536092Z] 20:25:43     INFO - 0:2(12): error:
> extension `GL_ARB_gpu_shader5' unsupported in fragment shader
> [task 2016-12-29T20:25:43.538539Z] 20:25:43     INFO -  
> [task 2016-12-29T20:25:43.539348Z] 20:25:43     INFO -    
> errorFunc@dom/canvas/test/webgl-mochitest/webgl-util.js:77:9
> [task 2016-12-29T20:25:43.540205Z] 20:25:43     INFO -    
> warning@dom/canvas/test/webgl-mochitest/webgl-util.js:31:5
> [task 2016-12-29T20:25:43.541032Z] 20:25:43     INFO -    
> createProgramByIds@dom/canvas/test/webgl-mochitest/webgl-util.js:154:7
> [task 2016-12-29T20:25:43.541842Z] 20:25:43     INFO -    
> @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:79:16
> [task 2016-12-29T20:25:43.543900Z] 20:25:43     INFO -    
> run@dom/canvas/test/webgl-mochitest/webgl-util.js:82:7
> [task 2016-12-29T20:25:43.544736Z] 20:25:43     INFO -    
> Async*withWebGL2@dom/canvas/test/webgl-mochitest/webgl-util.js:92:7
> [task 2016-12-29T20:25:43.545553Z] 20:25:43     INFO -    
> @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:35:3
> [task 2016-12-29T20:25:43.546640Z] 20:25:43     INFO - Not taking screenshot
> here: see the one that was previously logged
> [task 2016-12-29T20:25:43.548654Z] 20:25:43     INFO - TEST-UNEXPECTED-FAIL
> | dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | Program
> linking should succeed. 
> [task 2016-12-29T20:25:43.549529Z] 20:25:43     INFO -    
> @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:81:7
> [task 2016-12-29T20:25:43.550497Z] 20:25:43     INFO -    
> run@dom/canvas/test/webgl-mochitest/webgl-util.js:82:7
> [task 2016-12-29T20:25:43.552849Z] 20:25:43     INFO -    
> Async*withWebGL2@dom/canvas/test/webgl-mochitest/webgl-util.js:92:7
> [task 2016-12-29T20:25:43.553850Z] 20:25:43     INFO -    
> @dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html:35:3
> [task 2016-12-29T20:25:43.554910Z] 20:25:43     INFO - MEMORY STAT | vsize
> 1536MB | residentFast 376MB | heapAllocated 202MB
> [task 2016-12-29T20:25:43.556150Z] 20:25:43     INFO - TEST-OK |
> dom/canvas/test/webgl-mochitest/test_webgl2_alpha_luminance.html | took 145ms
> 
> 
> looks most likely as if we do not have GL_ARB_gpu_shader5 available- not
> sure if that is available in 16.04. Jeff- could you help determine what we
> need to do with this specific failure?
> 
> [0]
> https://public-artifacts.taskcluster.net/I-VbMUlSSKagd7kyVwv-eA/0/public/
> logs/live_backing.log

This is probably LLVMPipe? (Mesa software) I think this driver claims support for GL_ARB_gpu_shader5, but looks like it doesn't support it in vertex shaders, only fragment shaders.

There's not much we can do here, but it sounds like webgl2 is a no-go on this hardware.
We should mark this test as failing on this test slave configuration.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

llvmpipe does not support GL_ARB_gpu_shader5 but I'm surprised that it's required for webgl2. It looks like ANGLE is injecting the requirement for GL_ARB_gpu_shader5 for GL < 4. LLVMpipe does support ES3 directly. How hard would it be for us to use an ES3 context instead?

Comment 5

[Jeff Muizelaar [:jrmuizel]]

It looks like the requirement for GL_ARB_gpu_shader5 has been removed in upstream ANGLE. We should probably just take https://chromium.googlesource.com/angle/angle/+/729a9c974499e5cd9606bbeced50f14a42534e99

Comment 6

[Kelsey Gilbert [:jgilbert]]

We should probably be trying to use ES3 if we can get it, but we also really need test coverage for WebGL2-on-GL on Linux.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

so would the act of upgrading to ubuntu 16.04 remove the coverage of webgl2-on-gl ?

Comment 8

[Jeff Muizelaar [:jrmuizel]]

No, we can just cherry-pick or update ANGLE to https://chromium.googlesource.com/angle/angle/+/729a9c974499e5cd9606bbeced50f14a42534e99. That should be sufficient for keeping this working.

Comment 9

[Kelsey Gilbert [:jgilbert]]

(In reply to Joel Maher ( :jmaher) from comment #7)
> so would the act of upgrading to ubuntu 16.04 remove the coverage of
> webgl2-on-gl ?

We also don't test WebGL2 on Linux right now anyway, unfortunately.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

the fix from comment 8 works great.  I now have one issue remaining on asan:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=024920d006f72701638700b31e3ff36824011385

SUMMARY: AddressSanitizer: 40 byte(s) leaked in 1 allocation(s).
TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at pa_xmalloc0, pulse_init, cubeb_init, mozilla::CubebUtils::GetCubebContextUnlocked

I see this leak on many other tests for ASAN- while I would like to fix this, I am not sure how to go about doing this.  It would be possible to narrow this down to a smaller set of files, or to leave ASAN on 12.04.

:jgilbert, what do you think?

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: WIP patch to run mochitest-gl on ubuntu 16.04

Comment 12

[Kelsey Gilbert [:jgilbert]]

It's a leak where pulseaudio_init is called, but not _destroy:
https://hg.mozilla.org/mozilla-central/annotate/048240a074e841c425a4da4707cf8e353074ec1d/media/libcubeb/src/cubeb_pulse.c#l532

We should probably just add this to the 'tolerable leaks' list, which I think we have.

Any ideas :kinetik? Blame blames you for at least landing the code.

Comment 13

[Matthew Gregan [:kinetik]]

(In reply to Jeff Gilbert [:jgilbert] from comment #12)
> It's a leak where pulseaudio_init is called, but not _destroy:
> https://hg.mozilla.org/mozilla-central/annotate/
> 048240a074e841c425a4da4707cf8e353074ec1d/media/libcubeb/src/cubeb_pulse.
> c#l532
> 
> We should probably just add this to the 'tolerable leaks' list, which I
> think we have.
> 
> Any ideas :kinetik? Blame blames you for at least landing the code.

The matching cubeb_destroy() (mapped to pulse_destroy() on Linux) should be called via CubebUtils::ShutdownLibrary(), which is called on process shutdown via nsLayoutStatics::Shutdown().  Is it possible that's not being called for some reason?

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

I am happy to test any changes or potential fixes or information gathering patches.

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

following up here, is there anything someone is working or something I could try out to move this forward?

Comment 16

[Kelsey Gilbert [:jgilbert]]

(In reply to Joel Maher ( :jmaher) from comment #15)
> following up here, is there anything someone is working or something I could
> try out to move this forward?

Add this leak to the 'expected leak' list and file a bug against audio to get this looked at eventually.

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: run mochitest-gl on ubuntu 16.04

Comment 18

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8829205 [details] [diff] [review]
run mochitest-gl on ubuntu 16.04

Review of attachment 8829205 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/canvas/test/webgl-conf/generated-mochitest.ini
@@ +8080,5 @@
>  [generated/test_conformance__ogles__GL__atan__atan_009_to_012.html]
>  [generated/test_conformance__ogles__GL__biConstants__biConstants_001_to_008.html]
>  [generated/test_conformance__ogles__GL__biConstants__biConstants_009_to_016.html]
>  [generated/test_conformance__ogles__GL__biuDepthRange__biuDepthRange_001_to_002.html]
> +fail-if = (os == 'android')

These changes should be made to mochitest-errata.ini, and then propagated here via dom/canvas/test/webgl-conf/generate-wrappers-and-manifest.py. (see the header comment of this generated file)

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: run mochitest-gl on ubuntu 16.04

thanks for the pointer, updated the patch using the errata and generation script.

Comment 20

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: run mochitest-gl on ubuntu 16.04

sorry for the churn here, I pushed to try one more time...

* no need to for the asan leak, so we do not need to skip a test
* an extra test was passing, so I fixed that

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c1418f0d43dd5a7f0f0aff30faaa1b5870801580&selectedJob=73682995&filter-searchStr=gl&group_state=expanded

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/465c8a6fcdcb
Move Linux64 mochitest-gl to Ubuntu 16.04. r=jgilbert

Comment 22

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/465c8a6fcdcb