1546100 - Remove hardcoded e10s=False for mochitest-chrome/a11y

Status: RESOLVED FIXED

(Testing :: Mochitest, task)

Description

[Andrew Halberstadt [:ahal]]

In bug 1544816 I "turned on" e10s for mochitest-chrome and mochitest-a11y in the task configuration. But I forgot that there is a hardcoded e10s=False in the mochitest harness.

This line is confusing and should be removed, the task configuration should be the source of truth.

Unfortunately it looks like a11y and chrome can't run with e10s enabled, so if we remove this hack people's local commands will start to fail. I propose that instead of silently changing the value of "e10s", we fail fast if we try to run a11y/chrome with e10s enabled (with an error that hints at --disable-e10s).

Comment 1

[Andrew Halberstadt [:ahal]]

That other bug also broke a tryselect test (which didn't run on the push). I'll use this bug to clean that up as well.

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546100 - [tryselect] Fix tools/tryselect/test/test_fuzzy.py, r?jmaher

This was regressed by bug 1544816 but the test never ran on the push that regressed.
This patch also updates the 'files-changed' for the tryselect task.

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546100 - Turn off e10s for mochitest-a11y/chrome in task configuration (Backs out effa55bd84bb and 2f12958d4e8c), r?jmaher

Turns out these suites were hardcoded to be non-e10s in the mochitest harness.
So while it looked like they were working with e10s in treeherder, they were
actually still running with it disabled.

Turning e10s on causes both suites to permafail due to timeouts.

Depends on D28386

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/643d7793369e
[tryselect] Fix tools/tryselect/test/test_fuzzy.py, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/8cef878e7aa6
Turn off e10s for mochitest-a11y/chrome in task configuration (Backs out effa55bd84bb and 2f12958d4e8c), r=jmaher

Comment 5

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/643d7793369e
https://hg.mozilla.org/mozilla-central/rev/8cef878e7aa6

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546100 - [mochitest] Use kwargs from moztest.resolve in mach mochitest, r?jmaher

These days 'moztest.resolve' is becoming the defacto place to define test harness
related metadata. Currently |mach test| uses it to find kwargs that need to get
set for mochitest commands, but |mach mochitest| does not. Instead, it grabs this
metadata from 'mochitest_options.py'.

This patch makes the kwargs used consistent between both |mach test| and |mach
mochitest|.

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1546100 - [mochitest] Error out when running 'a11y' or 'chrome' with e10s, r?jmaher

Previously we would silently change the value of "e10s" from False to True.
This can cause confusion and lead people to falsely think mochitest-chrome/a11y
work with e10s (they do not).

Now we explicitly error out in this case. This might be slightly less
convenient for the developer (e.g they might need to re-run the command), but
the downside of needing to rerun a test command is less than the risk of
misunderstanding what is being tested.

Note: when running |mach test| or |mach mochitest| on a directory that contains
both chrome/a11y and another suite, we'll still do the right thing and
implicitly set "e10s=False".

Depends on D28537

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e47482b63074
[mochitest] Use kwargs from moztest.resolve in mach mochitest, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/ce1c928fb0cd
[mochitest] Error out when running 'a11y' or 'chrome' with e10s, r=jmaher

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/e47482b63074
https://hg.mozilla.org/mozilla-central/rev/ce1c928fb0cd