Closed
Bug 1433276
Opened 7 years ago
Closed 7 years ago
adjust non-e10s test scheduling
Categories
(Testing :: General, enhancement)
Testing
General
Tracking
(firefox60 fixed)
RESOLVED
FIXED
mozilla60
Tracking | Status | |
---|---|---|
firefox60 | --- | fixed |
People
(Reporter: jmaher, Assigned: jmaher)
References
(Depends on 1 open bug)
Details
Attachments
(1 file)
43.86 KB,
patch
|
ahal
:
review+
|
Details | Diff | Splinter Review |
we have retained non-e10s test scheduling for a few reasons:
1) android still ships with non-e10s by default
2) not all desktop tests run on android, for example web-platform-tests which have unique service worker code
3) users can easily turn off e10s to use addons or adjust their browser (not a good reason, but still valid)
We chose windows7-debug originally as this provided coverage of assertions and debug information to developers while gaining coverage on windows. Now that we are many months past the initial decision we are looking at what remains for non-e10s coverage and the top 3 reasons still exist.
After some discussion we find that linux32-debug will give us great coverage. Ideally in coming months #2 will not be an issue and we can then make a decision that if we have enough tests running on android (geckoview in the future) that we can be confident our browser is stable then we could be closer to making a decision to not run non-e10s on desktop builds.
Assignee | ||
Comment 1•7 years ago
|
||
this is:
mochitest-plain
mochitest-browser-chrome
mochitest-clipboard
mochitest-devtools-chrome
mochitest-gpu
mochitest-webgl
reftest
reftest-no-accel
crashtest
jsreftest
web-platform-tests
web-platform-tests-reftests
I wonder if we need:
jsreftest
reftest-no-accel
mochitest-devtools-chrome
mochitest-browser-chrome
:snorp, a small attempt at reducing the tests by 4 types- what do you think?
Flags: needinfo?(snorp)
Yeah, I think excluding those should be fine.
Flags: needinfo?(snorp)
Assignee | ||
Comment 3•7 years ago
|
||
one last question- should this be on linux32 (stylo by default) or linux32-stylo-disabled ?
Assignee | ||
Comment 4•7 years ago
|
||
this patch will work for linux32/stylo- if we need stylo-disabled instead, I can adjust it quickly.
The test failures are ones that were failing in the same way on win7-debug non-e10s
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Comment 5•7 years ago
|
||
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3)
> one last question- should this be on linux32 (stylo by default) or
> linux32-stylo-disabled ?
Just testing linux32 with Stylo default is adequate. We don't need to test non-e10s stylo-disabled because Stylo is already enabled by default on Android in Nightly.
Assignee | ||
Updated•7 years ago
|
Attachment #8945806 -
Flags: review?(ahalberstadt)
Comment 6•7 years ago
|
||
Comment on attachment 8945806 [details] [diff] [review]
move non-e10s testing to linux32-debug
Review of attachment 8945806 [details] [diff] [review]:
-----------------------------------------------------------------
It was hard to see what changed in the reftest.list, splinter kind of munged things up. Also, aren't there windows annotations in the web-platform manifests that should be removed? (I'm not familiar with these files)
The taskcluster changes look good though.
Attachment #8945806 -
Flags: review?(ahalberstadt) → review+
Assignee | ||
Comment 7•7 years ago
|
||
the web-platform-tests are really hard to edit- it seems as though we have every possibly config there and sometimes old ones we haven't ran for a long time. I would like to clean them up..i.e. if this is 80% of our cases, default to "timeout" or "crash" or "...", then exceptions for the passes.
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9d96681c74f2
adjust non-e10s test scheduling. r=ahal
Comment 9•7 years ago
|
||
:jmaher, starting with this push, an almost perma failure starts to occur on dom/serviceworkers/test/test_workerupdatefoundevent.html
Could you please take a look?
Treeherder: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=runnable&filter-resultStatus=success&fromchange=4637ab31ae110a8747761ea00814463c5855b107&filter-searchStr=b5df8d5c2639fef31a716cd37d027da3f49f9643
Log: https://treeherder.mozilla.org/logviewer.html#?job_id=158871318&repo=mozilla-inbound&lineNumber=2146
Flags: needinfo?(jmaher)
Assignee | ||
Comment 10•7 years ago
|
||
I landed a quick fix to disable this one test. Looking at the log:
[task 2018-01-27T16:00:39.317Z] 16:00:39 INFO - GECKO(2223) | ++DOCSHELL 0xcc70b800 == 7 [pid = 2223] [id = {ac16d669-f467-4f37-8180-b51cf69a634a}]
[task 2018-01-27T16:00:39.319Z] 16:00:39 INFO - GECKO(2223) | ++DOMWINDOW == 15 (0xccac5160) [pid = 2223] [serial = 847] [outer = (nil)]
[task 2018-01-27T16:00:39.322Z] 16:00:39 INFO - GECKO(2223) | ++DOMWINDOW == 16 (0xcc70c000) [pid = 2223] [serial = 848] [outer = 0xccac5160]
[task 2018-01-27T16:00:39.481Z] 16:00:39 INFO - GECKO(2223) | [2223, Main Thread] WARNING: FIXME: Not implemented!: file /builds/worker/workspace/build/src/dom/serviceworkers/ServiceWorkerRegistration.cpp, line 1031
[task 2018-01-27T16:01:00.474Z] 16:01:00 INFO - GECKO(2223) | --DOMWINDOW == 15 (0xcc626400) [pid = 2223] [serial = 845] [outer = (nil)] [url = http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html]
[task 2018-01-27T16:01:09.304Z] 16:01:09 INFO - GECKO(2223) | JavaScript error: http://mochi.test:8888/tests/dom/serviceworkers/test/worker_updatefoundevent.js, line 9: ReferenceError: reject is not defined
[task 2018-01-27T16:01:44.966Z] 16:01:44 INFO - GECKO(2223) | --DOMWINDOW == 14 (0xd7195800) [pid = 2223] [serial = 844] [outer = (nil)] [url = http://mochi.test:8888/tests/dom/serviceworkers/test/test_worker_reference_gc_timeout.html]
[task 2018-01-27T16:06:06.121Z] 16:06:06 INFO - TEST-INFO | started process screentopng
[task 2018-01-27T16:06:06.125Z] 16:06:06 INFO - /builds/worker/workspace/build/tests/bin/screentopng: error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory
[task 2018-01-27T16:06:06.127Z] 16:06:06 INFO - TEST-INFO | screentopng: exit 127
[task 2018-01-27T16:06:06.128Z] 16:06:06 INFO - TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_workerupdatefoundevent.html | Test timed out.
I see:
JavaScript error: http://mochi.test:8888/tests/dom/serviceworkers/test/worker_updatefoundevent.js, line 9: ReferenceError: reject is not defined
the source is here:
https://searchfox.org/mozilla-central/source/dom/workers/test/serviceworkers/worker_updatefoundevent.js#9
:bkelly- I see you wrote the code that calls reject()- we have been discussing where to run non-e10s tests and linux32/debug seemed like the most appropriate place given current code conditions, etc.- could you look into fixing this test?
Flags: needinfo?(jmaher) → needinfo?(bkelly)
Comment 11•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/9d96681c74f2
https://hg.mozilla.org/mozilla-central/rev/1d32e08b9e14
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox60:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Comment 12•7 years ago
|
||
Redirect NI to Marion to find someone to look at comment 10. The code looks just busted there. It should be using throw() instead of the undefined reject(). Its unclear to me if the test will pass if we fix that, though.
Flags: needinfo?(bkelly) → needinfo?(mdaly)
Comment 13•7 years ago
|
||
:catalin, can you take a look at this please?
Flags: needinfo?(mdaly) → needinfo?(catalin.badea392)
You need to log in
before you can comment on or make changes to this bug.
Description
•