1296086 - Allow GL layers with a software GL implementation using m3.large instances on some TaskCluster suites

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect)

Description

[Andrew Comminos [:acomminos]]

When GL layers gets enabled (bug 594876), we'll need to manually enable software GL layers using the environment variable MOZ_LAYERS_ALLOW_SOFTWARE_GL added in bug 1295757.

I propose adding a new transitory property to TaskCluster suites, "allow-software-gl-layers", set to true by default. This will upgrade the instance to a m3.large AWS instances (required for reasonable GL layers performance on a software GL implementation, see bug 1281241) and add a flag to mozharness to allow software layers acceleration. On suites that are not very green with this config, allow-software-gl-layers may be set to false to fallback to the old config- basic composition plus the default AWS instances.

Comment 1

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

Review commit: https://reviewboard.mozilla.org/r/72382/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72382/

Comment 2

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review commit: https://reviewboard.mozilla.org/r/72380/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72380/

Comment 3

[Andrew Comminos [:acomminos]]

These patches expect the patches from bugs 1295757 and 1281241 to be applied first. Devtools is less green with GL layers, so let's disallow software GL there for now.

Comment 4

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/1-2/

Comment 5

[Andrew Comminos [:acomminos]]

Comment on attachment 8782150 [details]
Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72382/diff/1-2/

Comment 6

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/2-3/

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782150 [details]
Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72382/#review70220

this looks great

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

https://reviewboard.mozilla.org/r/72380/#review70222

::: taskcluster/ci/desktop-test/tests.yml:239
(Diff revision 3)
>          by-test-platform:
>              # Bug 1242986: linux64/debug mochitest-devtools-chrome e10s is not greened up yet
>              linux64/debug: false
>              default: both
> +    # Bug 1296086: high number of intermittents observed with software GL and large instances
> +    allow-software-gl-layers: false

this will change for all platforms- so the mozharness changes will set the env variable by default for all platforms- this only affects linux, right?  then this will set software-gl-layers:false for all platforms.

we will probably need to differentiate between ASAN and linux32 here as we roll this out- maybe an all or none strategy for a test type is the way to go.

::: taskcluster/taskgraph/transforms/tests/desktop_test.py:97
(Diff revision 3)
> +                logger.warning('Allowed software GL layers on a legacy instance, expect timeouts.')
> +
> +            test['mozharness'].setdefault('extra-options', [])\
> +                              .append("--allow-software-gl-layers")
> +
> +        yield test

while I am not a fan of this, it makes sense given the fact that we intend for all tests to run in this fashion on linux64.

Can we put a comment that indicates to remove this when bug xxxxxxx is resolved?

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

https://reviewboard.mozilla.org/r/72380/#review70232

::: taskcluster/taskgraph/transforms/tests/desktop_test.py:92
(Diff revision 3)
> +    for test in tests:
> +        allow = get_keyed_by(item=test, field='allow-software-gl-layers',
> +                             item_name=test['test-name'])
> +        if allow:
> +            if test['instance-size'] == 'legacy':
> +                logger.warning('Allowed software GL layers on a legacy instance, expect timeouts.')

This should probably just fail.  I doubt anyone will see warnings in the decision task unless they happen to look at those logs for some other reason.  Also, a link to a bug # would be good.

::: taskcluster/taskgraph/transforms/tests/test_description.py:90
(Diff revision 3)
>      # on the platform)
>      Required('loopback-audio', default=False): bool,
>      Required('loopback-video', default=False): bool,
>  
> +    # Whether the test can run using a software GL implementation on Linux
> +    # using the GL compositor. Upgrades to a large instance to avoid timeouts.

This comment isn't quite accurate anymore

Comment 10

[Andrew Comminos [:acomminos]]

Here's a push with GL layers and m3.large instances for all;

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fb08dde910816aedc05bfbdca0d1fbbb888e7c52

I'll be making all high volume layers acceleration related intermittents block this bug.

Comment 11

[Andrew Comminos [:acomminos]]

Comment on attachment 8782150 [details]
Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72382/diff/2-3/

Comment 12

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/3-4/

Comment 13

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/4-5/

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

https://reviewboard.mozilla.org/r/72380/#review70382

thanks for the cleanup

Comment 15

[Andrew Comminos [:acomminos]]

Thanks. I just noticed that we'll also need to add the --allow-software-gl-layers flag to marionette; I'll quickly take care of that.

Comment 16

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add a marionette flag to allow GL composition on a software GL implementation.

Review commit: https://reviewboard.mozilla.org/r/72712/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72712/

Comment 17

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add a web-platform-tests flag to allow GL composition on a software GL implementation.

Review commit: https://reviewboard.mozilla.org/r/72714/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72714/

Comment 18

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add a firefox media tests flag to allow GL composition on a software GL implementation.

Review commit: https://reviewboard.mozilla.org/r/72716/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72716/

Comment 19

[Andrew Comminos [:acomminos]]

Attachment: Bug 1296086 - Add a firefox functional tests flag to allow GL composition on a software GL implementation.

Review commit: https://reviewboard.mozilla.org/r/72718/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72718/

Comment 20

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/5-6/

Comment 21

[Andrew Comminos [:acomminos]]

Comment on attachment 8782150 [details]
Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72382/diff/3-4/

Comment 22

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782609 [details]
Bug 1296086 - Add a marionette flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72712/#review70404

Comment 23

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782610 [details]
Bug 1296086 - Add a web-platform-tests flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72714/#review70406

Comment 24

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782611 [details]
Bug 1296086 - Add a firefox media tests flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72716/#review70408

Comment 25

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782612 [details]
Bug 1296086 - Add a firefox functional tests flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72718/#review70410

Comment 26

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8782612 [details]
Bug 1296086 - Add a firefox functional tests flag to allow GL composition on a software GL implementation.

https://reviewboard.mozilla.org/r/72718/#review70412

Comment 27

[Joel Maher ( :jmaher ) (UTC -8)]

we might want to standardize the talos.py script as well- not sure if we will be doing this for performance tests.

Comment 28

[Andrew Comminos [:acomminos]]

(In reply to Joel Maher ( :jmaher) from comment #27)
> we might want to standardize the talos.py script as well- not sure if we
> will be doing this for performance tests.

We shouldn't, I don't think- the performance using software GL isn't significant as we disable layers acceleration on it in production. Dedicated GPUs will automagically use layers acceleration once it's flipped in in bug 594876.

Comment 29

[Andrew Comminos [:acomminos]]

Comment on attachment 8782151 [details]
Bug 1296086 - Add property to TaskCluster to allow software GL layers.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72380/diff/6-7/

Comment 30

[Andrew Comminos [:acomminos]]

Comment on attachment 8782150 [details]
Bug 1296086 - Add a mozharness flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72382/diff/4-5/

Comment 31

[Andrew Comminos [:acomminos]]

Comment on attachment 8782609 [details]
Bug 1296086 - Add a marionette flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72712/diff/1-2/

Comment 32

[Andrew Comminos [:acomminos]]

Comment on attachment 8782610 [details]
Bug 1296086 - Add a web-platform-tests flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72714/diff/1-2/

Comment 33

[Andrew Comminos [:acomminos]]

Comment on attachment 8782611 [details]
Bug 1296086 - Add a firefox media tests flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72716/diff/1-2/

Comment 34

[Andrew Comminos [:acomminos]]

Comment on attachment 8782612 [details]
Bug 1296086 - Add a firefox functional tests flag to allow GL composition on a software GL implementation.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72718/diff/1-2/

Comment 35

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9b55d21dd152
Add property to TaskCluster to allow software GL layers. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/a54b3ed2232f
Add a mozharness flag to allow GL composition on a software GL implementation. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/bef08c5c1348
Add a marionette flag to allow GL composition on a software GL implementation. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/f5120a79b034
Add a web-platform-tests flag to allow GL composition on a software GL implementation. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/9c167f078b80
Add a firefox media tests flag to allow GL composition on a software GL implementation. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/e7b45868aa75
Add a firefox functional tests flag to allow GL composition on a software GL implementation. r=jmaher

Comment 36

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/9b55d21dd152
https://hg.mozilla.org/mozilla-central/rev/a54b3ed2232f
https://hg.mozilla.org/mozilla-central/rev/bef08c5c1348
https://hg.mozilla.org/mozilla-central/rev/f5120a79b034
https://hg.mozilla.org/mozilla-central/rev/9c167f078b80
https://hg.mozilla.org/mozilla-central/rev/e7b45868aa75

Comment 37

[Joel Maher ( :jmaher ) (UTC -8)]

I wanted to follow up here to see what work remains- I believe it is just greening up remaining jobs on try server?

Comment 38

[Joel Maher ( :jmaher ) (UTC -8)]

we now run all tests on c3.large (multi-core, non m1.medium) or larger, can we resolve this?

Comment 39

[Joel Maher ( :jmaher ) (UTC -8)]

I believe most if not all of these instances in-tree:
http://searchfox.org/mozilla-central/search?q=allow-software-gl-layers&path=taskcluster%2Fci%2Ftest%2F*.yml

of allow-software-gl-layers:False, were on suites that we ran on m1.medium.

:acomminos, do weigh in if you thing we should be trying to allow software gl layers or if there is other work to do here.

Comment 40

[Joel Maher ( :jmaher ) (UTC -8)]

following up here, should we be running software-gl-layers?

Comment 41

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:davidb, maybe it's time to close this bug?

Comment 42

[David Bolter [:davidb] (NeedInfo me for attention)]

I'm not sure we need this bug open. Jessie can you ask around?

Comment 43

[Jeff Muizelaar [:jrmuizel]]

We haven't been looking at doing gl layers on Linux and probably will just try to turn on WebRender instead. If we need to re-open this we can.