1552563 - Investigate if possible to eliminate starting compiz for linux tests

Status: RESOLVED FIXED

(Testing :: General, enhancement)

Description

[Edwin Takahashi (:egao | infrequent contributor)]

Problem

Currently, compiz window manager is started for all tests, even if it may not be required.

This is typically not a problem, except compiz adds overhead for both scenarios:

• when starting up
• after initialization, where compiz throws a cryptic error about restacking windows

In the case of linux tests on CI, it is the latter which proves to be more of a problem. The restacking error typically consumes ~900ms of docker container time [1]:

[task 2019-05-16T23:58:44.889Z] 23:58:44     INFO - retry: Calling _download_file with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/G-ssOgl5QSOk-yd_XL1QPA/artifacts/public/build/target.test_packages.json', 'file_name': u'/builds/worker/workspace/build/target.test_packages.json'}, attempt #1
[task 2019-05-16T23:58:47.359Z] compiz (core) - Warn: Attempted to restack relative to 0x1600006 which is not a child of the root window or a window compiz owns
[task 2019-05-16T23:58:56.191Z] 23:58:56     INFO - Downloaded 1253 bytes.

[1]example taken from https://taskcluster-artifacts.net/MBdTFXRhQE-phCgU7Rltig/0/public/logs/live_backing.log

As proof of concept, I have run web-platform-tests without starting compiz, and the expected outcome is still produced in https://taskcluster-artifacts.net/cqc29MBuRFq2lBvDDTrguQ/0/public/logs/live_backing.log.

Objective

Only start compiz if absolutely necessary for test suite.

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

I had what I thought was a fairly good and elegant attempt at preventing this compiz error in this push:
https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=pending%2Crunning%2Csuperseded%2Cusercancel%2Cretry%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&classifiedState=unclassified&group_state=expanded&revision=9a15447e319eb34f081fbd36ece77e3440035d8f&searchStr=linux64%2Fopt&selectedJob=246988518

However, turns out there's a problem with parsing mozharness_args, since not all suites pass in arguments that is necessary for this code section to work:

    declare -a REQUIRES_COMPIZ=("mochitest-plain"
                                "mochitest-browser-chrome-chunked"
                                "mochitest-devtools-chrome-chunked"
                                "mochitest-chrome"
                                "mochitest-a11y"
                                "wdspec"
                                )

    # only start compiz (which is time consuming) if it's actually required.
    for test_suite in "${REQUIRES_COMPIZ[@]}"; do
	if [[ "${@}" == *$test_suite* ]]; then
            # start compiz for our window manager
            compiz 2>&1 &
            break;
        fi
    done

Examples of tests that do not pass in mozharness_args in a nice format are web-platform-tests, firefox-ui-functional-remote and marionette. These are not caught by the whitelist REQUIRES_COMPIZ and thus compiz is not started, and tests fail.

This approach (of detecting in shell script) thus won't work.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1552563 - do not run compiz on linux tests unless needed by tests

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

In the patch posted, it was decided the best way to implement this whitelisting mechanism was through environment variable prior to test-linux.sh is called.

There are several environment variables that appear to denote if windowing manager is necessary, but they do not pertain to compiz specifically, so a new variable NEED_COMPIZ was created at mozharness_test.py.

The whitelist must be managed manually, which is a downside of this implementation.

Given that compiz on linux platforms eat up anywhere between 3-18 seconds due to the restack error, not starting compiz will potentially save us an amount comparable to that of the 15 second sleep removal from test-linux.sh.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/52e365586577
do not run compiz on linux tests unless needed by tests r=jmaher

Comment 5

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1552563 - remove mochitest-media from whitelist not requiring compiz

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aeff5661bcab
Backed out changeset 52e365586577 for failing test_getUserMedia_basicScreenshare.html on a CLOSED TREE
https://hg.mozilla.org/integration/autoland/rev/2196d50bfb2d
remove mochitest-media from whitelist not requiring compiz CLOSED TREE

Comment 7

[Edwin Takahashi (:egao | infrequent contributor)]

Original change caused permafail on mochitest-media-2 on linux platforms.

Comment 8

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/2196d50bfb2d