1269872 - create a new 'gpu' job to replace 'gl' and add other tests which require a gpu

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

currently we run the webgl tests in a 'gl' job.  This is great as they are by themselves and we know that this job requires a gpu to run successfully.  In testing windows 7 test machines in aws, it is clear that we have many other test cases which require a GPU:
plain:
* dom/imptests/html/webgl/* (6 tests here)
* gfx/tests/mochitest/test_acceleration.html
* dom/canvas/test/ (most tests have offscreencanvas tags, possibly whole directory could be included in gpu land?):
** dom/canvas/test/test_offscreencanvas_toblob.html
** dom/canvas/test/test_offscreencanvas_toimagebitmap.html
** dom/canvas/test/test_offscreencanvas_basic_webgl.html
** dom/canvas/test/test_offscreencanvas_dynamic_fallback.html
** dom/canvas/test/test_offscreencanvas_serviceworker.html
** dom/canvas/test/test_offscreencanvas_sizechange.html
* dom/base/test/test_anonymousContent_canvas.html

mochitest-chrome:
* dom/canvas/test/chrome/test_webgl_debug_renderer_info.html

mochitest-browser-chrome (devtools):
* devtools/client/canvasdebugger/test/browser_canvas-actor-test-09.js
* devtools/client/canvasdebugger/test/browser_canvas-actor-test-10.js
* devtools/client/canvasdebugger/test/browser_canvas-actor-test-11.js
* devtools/client/canvasdebugger/test/browser_profiling-webgl.js


we would need to run mochitest in different types:
plain --subsuite=gpu
chrome --subsuite=gpu
browser-chrome --subsuite=gpu

mochitest-other job does something similar.  one thing is to see if the chrome test could be run as a plain test :)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

it seems we have crashtests that depend on webgl-experimental and throw assertions in debug mode (https://treeherder.mozilla.org/#/jobs?repo=try&revision=2d2c7681ff1d7808e404b6ea0446c08c6857add4&filter-searchStr=crashtest):
22:31:15     INFO -  JavaScript warning: file:///c:/slave/test/build/tests/reftest/tests/dom/canvas/crashtests/727547.html, line 5: Error: WebGL: Error during ANGLE OpenGL init.
22:31:15     INFO -  [6080] ###!!! ASSERTION: Fallback WARP ANGLE context failed to initialize.: 'Error', file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/gfx/gl/GLLibraryEGL.cpp, line 409

and an issue in xpcshell as well (https://treeherder.mozilla.org/#/jobs?repo=try&revision=2d2c7681ff1d7808e404b6ea0446c08c6857add4&filter-searchStr=xpcshell):
testname: test_gfxBlacklist_No_Comparison.js
error: 
23:20:31  WARNING -  TEST-UNEXPECTED-FAIL | xpcshell-unpack.ini:toolkit/mozapps/extensions/test/xpcshell/test_gfxBlacklist_No_Comparison.js | checkBlacklist - NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIGfxInfo.getFeatureStatus]
23:20:31     INFO -  checkBlacklist@c:/slave/test/build/tests/xpcshell/tests/toolkit/mozapps/extensions/test/xpcshell/test_gfxBlacklist_No_Comparison.js:74:14
23:20:31     INFO -  do_execute_soon/<.run@c:\slave\test\build\tests\xpcshell\head.js:692:9
23:20:31     INFO -  _do_main@c:\slave\test\build\tests\xpcshell\head.js:209:5
23:20:31     INFO -  _execute_test@c:\slave\test\build\tests\xpcshell\head.js:533:5
23:20:31     INFO -  @-e:1:1
23:20:31     INFO -  exiting test

in fact, my list of additional tests so far is documented here:
https://hg.mozilla.org/try/rev/c81e54baacd264dbf66d456a8a02972079241070

Once I feel we have a good list, I will figure out the ideal set of steps required to get this fixed up.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: MozReview Request: Bug 1269872 - convert webgl related crashtests to mochitest. r=jgilbert

Review commit: https://reviewboard.mozilla.org/r/50797/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/50797/

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

update:
1) convert mochitest-chrome test to mochitest (up for review), this will automatically get run in the 'gl' job
2) convert crashtests that use webgl to mochitest (testing on try prior to review), these will automatically get run in the 'gl' job
3) the one xpcshell test which is discussed in bug 1270252.
4) there are 4 tests in devtools (runs as the browser-chrome suite) which depend on webgl- TODO
  * this might require running the a second instance of the browser to execute these 4 tests.
5) rename the job to 'gpu' once 1-4 are resolved.

Comment 4

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8749180 [details]
MozReview Request: Bug 1269872 - convert webgl related crashtests to mochitest. r=jgilbert

https://reviewboard.mozilla.org/r/50797/#review47523

We need this to be a chrome test because:

* Outside of Dev and Nightly, this extension is priveleged-code-only
* This test guarantees that the code we use in about:support continues to work
* about:support is critical for diagnosing WebGL issues

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

oh great, I will figure out how to get us to run a chrome test from the new gpu job so we can keep this as chrome!

Comment 6

[Kelsey Gilbert [:jgilbert]]

(In reply to Joel Maher (:jmaher) from comment #5)
> oh great, I will figure out how to get us to run a chrome test from the new
> gpu job so we can keep this as chrome!

It would be really cool if there were simply a mochitest flag for this!

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8749180 [details]
MozReview Request: Bug 1269872 - convert webgl related crashtests to mochitest. r=jgilbert

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/50797/diff/1-2/

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

:jgilbert, this is a set of crashtests (5 total, from 3 directories) that require webgl to run- different than the mochitest-chrome test I asked for review on earlier.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

right now we are looking at 3 different suites that need to run for this:
* mochitest-plain
* mochitest-chrome
* mochitest-browser-chrome

Looking at how mochitest-other is defined to support multiple mochitest types:
https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#604

we will need to do this for a new job type 'mochitest-gpu' and 'mochitest-gpu-e10s':
https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#651

and of course, firefox 49+.  what starts to make my head spin is all the references to mochitest-gl:
https://dxr.mozilla.org/build-central/search?q=mochitest-gl&redirect=true&case=true

and what do we do about android?  I believe we wouldn't need to change android since mochitest-chrome (this specific test anyway) and devtools are not run on there?

on top of that we have in-tree changes to make (all references to mochitest-gl):
https://dxr.mozilla.org/mozilla-central/search?q=mochitest-gl&redirect=false&case=true

I think once we get the xpcshell test figured out and the crashtests to a r+ state, we can work on the scheduling in more seriousness.

Comment 10

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8749180 [details]
MozReview Request: Bug 1269872 - convert webgl related crashtests to mochitest. r=jgilbert

https://reviewboard.mozilla.org/r/50797/#review47843

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8749180 [details]
MozReview Request: Bug 1269872 - convert webgl related crashtests to mochitest. r=jgilbert

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/50797/diff/2-3/

Comment 12

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/dc9136806be4

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/dc9136806be4

Comment 14

[Armen [:armenzg]]

Attachment: add win7 gpu & gpu-e10s jobs for opt and debug on try

Builders added:
+ Windows 7 32-bit try debug test mochitest-gpu
+ Windows 7 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 32-bit try opt test mochitest-gpu
+ Windows 7 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try debug test mochitest-gpu
+ Windows 7 VM 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try opt test mochitest-gpu
+ Windows 7 VM 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu-e10s

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8750800 [details] [diff] [review]
add win7 gpu & gpu-e10s jobs for opt and debug on try

Review of attachment 8750800 [details] [diff] [review]:
-----------------------------------------------------------------

I think this is close, don't we need all the job definitions in the different config/platforms as well:
https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#975

::: mozilla-tests/config.py
@@ +673,5 @@
> +        'use_mozharness': True,
> +        'script_path': 'scripts/desktop_unittest.py',
> +        'extra_args': ['--mochitest-suite',
> +                       'mochitest-plain,mochitest-chrome,mochitest-browser-chrome',
> +                       '--subsuite=gpu'],

I think we want --disable-e10s here, and not --e10s below

@@ +3134,5 @@
> +    for platform in ('win32',):
> +        platform_info = PLATFORMS.get(platform)
> +        for slave_platform in platform_info.get('slave_platforms'):
> +            if slave_platform in branch['platforms'][platform] and \
> +               slave_platform in ('win7_ix', 'win7_vm', 'win7_vm_gfx',):

technically this won't run in win7_vm, we can schedule it if we want.

Ideally this would run on all platforms, not just win7.

Comment 16

[Armen [:armenzg]]

Attachment: MozReview Request: Bug 1269872 - Add gpu jobs to try. r=jmaher

Review commit: https://reviewboard.mozilla.org/r/51667/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/51667/

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8750833 [details]
MozReview Request: Bug 1269872 - Add gpu jobs to try. r=jmaher

https://reviewboard.mozilla.org/r/51667/#review48457

::: mozilla-tests/config.py:677
(Diff revision 1)
> +    ('mochitest-gpu', {
> +        'use_mozharness': True,
> +        'script_path': 'scripts/desktop_unittest.py',
> +        'extra_args': ['--mochitest-suite',
> +                       'mochitest-plain,mochitest-chrome,mochitest-browser-chrome',
> +                       '--subsuite=gpu'],

I would prefer --disable-e10s for non-e10s and no options for e10s, that should be the default now for all harnesses.

Last question is mobile/android- this should be supported there- we don't have browser-chrome, but we do have mochitest-plain/mochitest-chrome- possibly a glc (gl-chrome) job for android only since the 'gl' job is already in 10 chunks.

Comment 18

[Armen [:armenzg]]

I forgot to add this with the lastest patch:
Builders added:
+ Rev4 MacOSX Snow Leopard 10.6 try debug test mochitest-gpu
+ Rev4 MacOSX Snow Leopard 10.6 try debug test mochitest-gpu-e10s
+ Rev4 MacOSX Snow Leopard 10.6 try opt test mochitest-gpu
+ Rev4 MacOSX Snow Leopard 10.6 try opt test mochitest-gpu-e10s
+ Rev7 MacOSX Yosemite 10.10.5 try debug test mochitest-gpu
+ Rev7 MacOSX Yosemite 10.10.5 try debug test mochitest-gpu-e10s
+ Rev7 MacOSX Yosemite 10.10.5 try opt test mochitest-gpu
+ Rev7 MacOSX Yosemite 10.10.5 try opt test mochitest-gpu-e10s
+ Ubuntu Code Coverage VM 12.04 x64 try opt test mochitest-gpu
+ Ubuntu Code Coverage VM 12.04 x64 try opt test mochitest-gpu-e10s
+ Ubuntu TSAN VM 12.04 x64 try opt test mochitest-gpu
+ Ubuntu TSAN VM 12.04 x64 try opt test mochitest-gpu-e10s
+ Ubuntu VM 12.04 try debug test mochitest-gpu
+ Ubuntu VM 12.04 try debug test mochitest-gpu-e10s
+ Ubuntu VM 12.04 try opt test mochitest-gpu
+ Ubuntu VM 12.04 try opt test mochitest-gpu-e10s
+ Ubuntu VM 12.04 x64 try opt test mochitest-gpu
+ Ubuntu VM 12.04 x64 try opt test mochitest-gpu-e10s
+ Windows 10 64-bit try debug test mochitest-gpu
+ Windows 10 64-bit try debug test mochitest-gpu-e10s
+ Windows 10 64-bit try opt test mochitest-gpu
+ Windows 10 64-bit try opt test mochitest-gpu-e10s
+ Windows 7 32-bit try debug test mochitest-gpu
+ Windows 7 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 32-bit try opt test mochitest-gpu
+ Windows 7 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try debug test mochitest-gpu
+ Windows 7 VM 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try opt test mochitest-gpu
+ Windows 7 VM 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu-e10s
+ Windows 8 64-bit try debug test mochitest-gpu
+ Windows 8 64-bit try debug test mochitest-gpu-e10s
+ Windows 8 64-bit try opt test mochitest-gpu
+ Windows 8 64-bit try opt test mochitest-gpu-e10s
+ Windows XP 32-bit try debug test mochitest-gpu
+ Windows XP 32-bit try debug test mochitest-gpu-e10s
+ Windows XP 32-bit try opt test mochitest-gpu
+ Windows XP 32-bit try opt test mochitest-gpu-e10s

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

I thought we were not doing this for codecoverage and tsan according to the patch?

Comment 20

[James Graham [:jgraham]]

Note that this patch as written won't work with treeherder because it will generate multiple error summary files from a single job type.

Comment 21

[Armen [:armenzg]]

For Android, do you want all platforms? I assume there's no e10s there.

Comment 22

[Joel Maher ( :jmaher ) (UTC -8)]

all instances where we already have the 'gl' job family running- and you are correct, we have no e10s on android.

Comment 23

[Armen [:armenzg]]

Comment on attachment 8750833 [details]
MozReview Request: Bug 1269872 - Add gpu jobs to try. r=jmaher

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/51667/diff/1-2/

Comment 24

[Armen [:armenzg]]

This was way more painful than I wished!

Builders added:
+ Android 4.3 armv7 API 15+ try debug test gl-chrome
+ Android 4.3 armv7 API 15+ try opt test gl-chrome
+ Rev4 MacOSX Snow Leopard 10.6 try debug test mochitest-gpu
+ Rev4 MacOSX Snow Leopard 10.6 try debug test mochitest-gpu-e10s
+ Rev4 MacOSX Snow Leopard 10.6 try opt test mochitest-gpu
+ Rev4 MacOSX Snow Leopard 10.6 try opt test mochitest-gpu-e10s
+ Rev7 MacOSX Yosemite 10.10.5 try debug test mochitest-gpu
+ Rev7 MacOSX Yosemite 10.10.5 try debug test mochitest-gpu-e10s
+ Rev7 MacOSX Yosemite 10.10.5 try opt test mochitest-gpu
+ Rev7 MacOSX Yosemite 10.10.5 try opt test mochitest-gpu-e10s
+ Ubuntu VM 12.04 try debug test mochitest-gpu
+ Ubuntu VM 12.04 try debug test mochitest-gpu-e10s
+ Ubuntu VM 12.04 try opt test mochitest-gpu
+ Ubuntu VM 12.04 try opt test mochitest-gpu-e10s
+ Ubuntu VM 12.04 x64 try opt test mochitest-gpu
+ Ubuntu VM 12.04 x64 try opt test mochitest-gpu-e10s
+ Windows 10 64-bit try debug test mochitest-gpu
+ Windows 10 64-bit try debug test mochitest-gpu-e10s
+ Windows 10 64-bit try opt test mochitest-gpu
+ Windows 10 64-bit try opt test mochitest-gpu-e10s
+ Windows 7 32-bit try debug test mochitest-gpu
+ Windows 7 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 32-bit try opt test mochitest-gpu
+ Windows 7 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try debug test mochitest-gpu
+ Windows 7 VM 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM 32-bit try opt test mochitest-gpu
+ Windows 7 VM 32-bit try opt test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try debug test mochitest-gpu-e10s
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu
+ Windows 7 VM-GFX 32-bit try opt test mochitest-gpu-e10s
+ Windows 8 64-bit try debug test mochitest-gpu
+ Windows 8 64-bit try debug test mochitest-gpu-e10s
+ Windows 8 64-bit try opt test mochitest-gpu
+ Windows 8 64-bit try opt test mochitest-gpu-e10s
+ Windows XP 32-bit try debug test mochitest-gpu
+ Windows XP 32-bit try debug test mochitest-gpu-e10s
+ Windows XP 32-bit try opt test mochitest-gpu
+ Windows XP 32-bit try opt test mochitest-gpu-e10s

Comment 25

[Armen [:armenzg]]

Attachment: MozReview Request: Rename mobile job to gpu

Review commit: https://reviewboard.mozilla.org/r/52205/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/52205/

Comment 26

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8750833 [details]
MozReview Request: Bug 1269872 - Add gpu jobs to try. r=jmaher

https://reviewboard.mozilla.org/r/51667/#review49153

::: mozilla-tests/config.py:676
(Diff revision 2)
> +MOCHITEST_GPU = [
> +    ('mochitest-gpu', {
> +        'use_mozharness': True,
> +        'script_path': 'scripts/desktop_unittest.py',
> +        'extra_args': ['--mochitest-suite',
> +                       'mochitest-plain,mochitest-chrome,mochitest-browser-chrome',

the only thing is that browser-chrome is the main harness, but we need to run devtools- I think this will be fine as devtools is just --subsuite=devtools.  so I am going to assume all is good and we can validate on try.

Comment 27

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8751771 [details]
MozReview Request: Rename mobile job to gpu

https://reviewboard.mozilla.org/r/52205/#review49155

::: mozilla-tests/mobile_config.py:2692
(Diff revision 1)
>          'script_maxtime': 14400,
>          },
>      ),
>  ]
>  
> -ANDROID_4_3_GL_CHROME = [
> +ANDROID_4_3_GPU = [

gpu-chrome

Comment 28

[Armen [:armenzg]]

https://hg.mozilla.org/build/buildbot-configs/rev/75e85663341f77e4b2e079a199fab1bad977d99e
Bug 1269872 - Add gpu jobs to try. r=jmaher

Comment 29

[Joel Maher ( :jmaher ) (UTC -8)]

I just realized the android chrome test we have for gpu is |skip-if = os == android|:
https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/chrome/chrome.ini#5

which means we can remove the android job changes!

Comment 30

[Joel Maher ( :jmaher ) (UTC -8)]

more likely we can just name it android_4_3_gpu, and then have it only run the --subsuite=gpu

Comment 31

[Armen [:armenzg]]

https://hg.mozilla.org/build/buildbot-configs/rev/d99a2e77873ebaffac1c9b4b4b83d787606590eb
Bug 1269872 - Remove mochitest-chrome from Android gpu jobs. r=jmaher

Comment 32

[Joel Maher ( :jmaher ) (UTC -8)]

this isn't working on try:
http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-567bc7ccc8c5a82a919150fa84205d1257211c7c/try-win32/try_xp_ix_test-mochitest-gpu-bm110-tests1-windows-build0.txt.gz

'c:/mozilla-build/python27/python' '-u' 'scripts/scripts/desktop_unittest.py' '--cfg' 'unittests/win_unittest.py' '--mochitest-suite' 'mochitest-plain,mochitest-chrome,mochitest-browser-chrome' '--subsuite=gpu' '--disable-e10s' '--blob-upload-branch' 'try' '--download-symbols' 'ondemand'
...
Usage: desktop_unittest.py [options]

desktop_unittest.py: error: no such option: --subsuite
program finished with exit code 2
elapsedTime=0.437000


possibly we need in-tree configs to adjust, or to rename things.

Comment 33

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/d99a2e77873e

Comment 34

[Armen [:armenzg]]

For reference, the jobs are hidden:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=567bc7ccc8c5a82a919150fa84205d1257211c7c&exclusion_profile=false&filter-searchStr=gpu

Comment 35

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: fix up buildbot configs to not define subsuites

no builder differences, changes will be in tree in linux_unittests.py, etc.

Comment 36

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add mozharness configs and taskcluster configs for gpu/clipboard

Comment 37

[Armen [:armenzg]]

Comment on attachment 8753012 [details] [diff] [review]
add mozharness configs and taskcluster configs for gpu/clipboard

Review of attachment 8753012 [details] [diff] [review]:
-----------------------------------------------------------------

Use --disable-e10s for plain jobs and you can also extend the plain task from the e10s one.

Comment 38

[Joel Maher ( :jmaher ) (UTC -8)]

landed the buildbot stuff here:
https://hg.mozilla.org/build/buildbot-configs/rev/41ec8f02fedf

Comment 39

[Chris AtLee [:catlee]]

In production: http://hg.mozilla.org/build/buildbot-configs/rev/a3cc07d009c5

Comment 40

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: change --disable-e10s to --e10s

ok, as these configs touch all branches, we should have the buildbot hardcoded stuff as --e10s and leave the hack in mozharness to deal with the specific harness flags.

Comment 41

[Joel Maher ( :jmaher ) (UTC -8)]

landed:
https://hg.mozilla.org/build/buildbot-configs/rev/10b8e6c3c984

this will get picked up in the next reconfig.

Comment 42

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/10b8e6c3c984

Comment 43

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: use androidarm_4_3.py not androidarm.py for mobile tests

Comment 44

[Joel Maher ( :jmaher ) (UTC -8)]

getting closer:
https://hg.mozilla.org/build/buildbot-configs/rev/8aa5b6e8e7c1

Comment 45

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: remove builders of gpu/clipboard that we wouldn't run in production

Comment 46

[Joel Maher ( :jmaher ) (UTC -8)]

landed the changes to reduce e10s jobs for the try push:
https://hg.mozilla.org/build/buildbot-configs/rev/800f339e74f1

Comment 47

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/8aa5b6e8e7c1

Comment 48

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/800f339e74f1

Comment 49

[Joel Maher ( :jmaher ) (UTC -8)]

next steps for this bug to get it live:
1) get this running on ASAN (asan is opt only, no need for debug) e10s and non-e10s. this requires yet another buildbot-configs patch
2) get green on android (I have some mozharness changes which I made, waiting on try to open)
3) land the mozharness changes
4) ensure gpu/cl jobs are hidden on all trees (should be done already, just double check)
5) create/land buildbot configs patch to run on all branches, excluding winxp opt/debug e10s*, win8 debug e10s.  On Ash all winxp/win8 needs to run.
6) deploy buildbot-configs patch
7) land patch to adjust manifests (and add taskcluster jobs) in-tree
8) once merged around, unhide gpu/cl jobs
9) verify all is green
10) hide 'gl' jobs on all trees
11) land in-tree manifest changes for subsuite=webgl, to be subsuite=gpu, remove mozharness related bits
12) let that merge around
13) create buildbot patch to increase android gpu chunks to 10 and remove 'gl' jobs from desktop and android for branch 49+
14) remove gl jobs exception from treeherder
15) resolve bug

Comment 50

[Chris AtLee [:catlee]]

Attachment: MozReview Request: Bug 1269872 - Create GPU/clipboard jobs for ASAN r=rail

Review commit: https://reviewboard.mozilla.org/r/54274/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/54274/

Comment 51

[Rail Aliiev [:rail]]

Comment on attachment 8754846 [details]
MozReview Request: Bug 1269872 - Create GPU/clipboard jobs for ASAN r=rail

https://reviewboard.mozilla.org/r/54274/#review50952

Comment 52

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/1ad733905a5b

Comment 53

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: the asan configs need defintions in the platform type

this is simple and should get us running the test properly.

Comment 54

[Joel Maher ( :jmaher ) (UTC -8)]

thanks Callek for the review, landed in the repo:
https://hg.mozilla.org/build/buildbot-configs/rev/c2db2be3fd78

Comment 55

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/c2db2be3fd78

Comment 56

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: enable gpu/clipboard on 49+ branches (and ash)

this changes a lot of builders!

Comment 57

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: buildbot builder differences for running gpu/clipboard on 49+

Ryan, a sanity check from you and verification of the ash jobs!

Comment 58

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: in-tree mozharness changes to get gpu/clipboard jobs running

these changes will allow us to run once we have subsuite definitions in place.

Comment 59

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8755604 [details]
buildbot builder differences for running gpu/clipboard on 49+

Looks sane to me!

Comment 60

[Joel Maher ( :jmaher ) (UTC -8)]

landed buildbot configs change:
https://hg.mozilla.org/build/buildbot-configs/rev/7d5af12079a6

Comment 61

[Armen [:armenzg]]

Comment on attachment 8755607 [details] [diff] [review]
in-tree mozharness changes to get gpu/clipboard jobs running

Review of attachment 8755607 [details] [diff] [review]:
-----------------------------------------------------------------

So sad so many changes are needed!

Comment 62

[Geoff Brown [:gbrown]]

Comment on attachment 8755607 [details] [diff] [review]
in-tree mozharness changes to get gpu/clipboard jobs running

Review of attachment 8755607 [details] [diff] [review]:
-----------------------------------------------------------------

(I didn't intentionally change :armenzg's review there - sorry!)

Comment 63

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/1355a79b416c

Comment 64

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/7d5af12079a6

Comment 65

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: manifest changes to .ini files gpu job

all but test_peerConnection_captureStream_canvas_webgl.html is in this patch, awaiting word from the media team- luckily the mochitest-media job is not on the short list to migrate over to the VM's.

Comment 66

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8756276 [details] [diff] [review]
manifest changes to .ini files gpu job

Review of attachment 8756276 [details] [diff] [review]:
-----------------------------------------------------------------

r=me under the assumption that this is green on Try and the understanding that the various |subsuite = webgl| annotations are intended to be left alone for the time-being.

Comment 67

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8756276 [details] [diff] [review]
manifest changes to .ini files gpu job

Review of attachment 8756276 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/bindings/test/mochitest.ini
@@ +36,5 @@
>  [test_exceptionThrowing.html]
>  [test_exception_messages.html]
>  [test_forOf.html]
>  [test_integers.html]
> +subsuite = gpu

Do we want this in gpu? It likely is because of WebGL entrypoints/typedefs, but there shouldn't be any actual GPU calls. (or at least I shouldn't think so!)[1]

@@ +53,5 @@
>  skip-if = debug == false
>  [test_usvstring.html]
>  skip-if = debug == false
>  [test_sequence_wrapping.html]
> +subsuite = gpu

[1]

Comment 68

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/1355a79b416c

Comment 69

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b3b9c70d53be

Comment 70

[Joel Maher ( :jmaher ) (UTC -8)]

thanks for the reviews, I have left test_integer.html out of the patch for now.

I will closely monitor these tests and call out any other tests which need issues.  The only two on my radar are:
dom/media/tests/mochitests/test_peerConnection_captureStream_canvas_webgl.html has an issue on android as it is moved from mochitest-media -> mochitest-gpu.
dom/bindings/test/test_integer.html uses experimental-webgl, it might need to be picked up in a future cleanup.

Comment 71

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b3b9c70d53be

Comment 72

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add taskcluster configs for gpu/clipboard

as seen on try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f98e8e9eaf437128c3e152ebaae507654eea680a

Comment 73

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/605b0b7d5fed

Comment 74

[Joel Maher ( :jmaher ) (UTC -8)]

from comment 49, the remaining items are:
0) hide 'gl' jobs on all trees
1) land in-tree manifest changes for subsuite=webgl, to be subsuite=gpu, remove mozharness related bits
2) let that merge around
3) create buildbot patch to increase android gpu chunks to 10 and remove 'gl' jobs from desktop and android for branch 49+
4) remove gl jobs exception from treeherder
5) resolve bug


I will look at that next week- in addition the changes for these 2 tests if needed:
dom/media/tests/mochitests/test_peerConnection_captureStream_canvas_webgl.html has an issue on android as it is moved from mochitest-media -> mochitest-gpu.
dom/bindings/test/test_integer.html uses experimental-webgl, it might need to be picked up in a future cleanup.

Comment 75

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/605b0b7d5fed

Comment 76

[Armen [:armenzg]]

What's left in here?
Can we assume that will be fixed on TC land? I see a gpu job green on my try pushes.

Comment 77

[Joel Maher ( :jmaher ) (UTC -8)]

oh, this is done

Comment 78

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.