1391350 - [meta] tracking bug for issues related to running tests in non-e10s mode

Status: RESOLVED FIXED

(Testing :: General, enhancement, P3)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

we are turning on:
crashtest
jsreftest
mochitest-plain
mochitest-browser-chrome
mochitest-clipboard
mochitest-devtools-chrome
mochitest-gpu
mochitest-webgl
reftest
reftest-no-accel
web-platform-tests
web-platform-tests-reftests

Ideally we will be able to for each suite we turned on:
* inspect which tests are unique to non-e10s and follow up with owners to determine what is required to make the tests work in e10s
* assess coverage and determine if there is large areas of code that is non-e10s specific which is only covered by running in non-e10s mode (note: other tests might cover this, android might, or e10s tests might cover it as well)
* file bugs to fix things
* when all things are fixed, only run in e10s mode.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

in addition we currently run some of our jsdebugger based code coverage on linux64 opt builds and we specifically only run the tests in non-e10s mode- this is for:
mochitest-plain
mochitest-browser-chrome
mochitest-devtools

I know this isn't web-platform-tests, nor reftest; but our coverage is not as dire as it would seem.  Android does run reftest, so any web-platform-tests that are exercising code in non-e10s mode or non-windows have a chance of lacking coverage.

Comment 3

[Julian Descottes [:jdescottes]]

Geoff, has there been a change here? I recently found a devtools test skipped in e10s that was no longer passing, so I tried to understand why CI didn't catch this. It looks like almost no tests are running in non-e10s now? Looking at https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=dd386b5b9fa7f5cd6dc4bbbfa0503b3eb2969af5&filter-tier=1&filter-tier=2&filter-tier=3 for instance.

Note that I'm not asking for non-e10s tests to be re-enabled, but I'd like to confirm that this is the expected state, so that we can prioritize enabling the ~60 devtools tests still skipped in e10s correctly.

Comment 4

[Geoff Brown [:gbrown]]

I'm not aware of a change.

I expect to see most tests running in a non-e10s configuration on linux32/debug only (and Android) and I see that in recent mozilla-central pushes for plain mochitests, plain reftests, wpt, etc. -- but I do not see mochitest-devtools-chrome included there.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=linux32%20debug

I looked back about 6 weeks and saw no relevant scheduling changes.

I am a little surprised we are not running linux32/debug mochitest-devtools-chrome (e10s or not) and I don't immediately see how that is configured. Having a look...

Comment 5

[Geoff Brown [:gbrown]]

Recall the non-e10s testing issue initially came up in bug 1386689 and was discussed in https://groups.google.com/forum/#!topic/mozilla.dev.platform/8gS5pOaLw0s ... although lots has changed since then.

> I am a little surprised we are not running linux32/debug mochitest-devtools-chrome (e10s or not)

Of course it turns out I wrote the patch to do that!

https://hg.mozilla.org/mozilla-central/rev/4bea6ef3d9de (bug 1328915).

So in that sense, this is the expected state: We intentionally do not run linux32/debug mochitest-devtools-chrome, and we intentionally run most non-e10s tests only on linux32/debug.


I haven't read through all the related history. I think it is possible that we did not explicitly decide to stop running non-e10s devtools -- but that's where we are.

Comment 6

[Julian Descottes [:jdescottes]]

As discussed on IRC, I think this is fine for DevTools, we are making progress on Bug 1387330