Closed
Bug 1170144
Opened 9 years ago
Closed 6 years ago
investigate run-by-dir cycle times as the harness/browser startup/shutdown takes a long time
Categories
(Testing :: Mochitest, defect)
Testing
Mochitest
Tracking
(firefox41 affected)
RESOLVED
INACTIVE
Tracking | Status | |
---|---|---|
firefox41 | --- | affected |
People
(Reporter: jmaher, Unassigned)
References
Details
we run browser-chrome/devtools tests in --run-by-dir mode. This is great for test isolation. While working towards bringing this to other tests (mochitest-plain/mochitest-chrome), I see we are doubling our runtimes. running per dir gives us much more advantages than not (think simplicity for chunking, parallelization, test isolation). In this bug we should investigate and implement methods for now to make the harness startup/shutdown faster. We can do stuff in parallel (not run the tests, but the rest of the work), or just optimize our path for simplicity.
Comment 1•9 years ago
|
||
We talked about this a bit in PDX. I think the first step would be to instrument things to find out what exactly is taking the most time between runs. If it's mostly browser startup then obviously that's a tricky thing to deal with. We'd have to start a browser in the background while the previous chunk is still running and then only start running tests in it when the previous tests finish. If we're spending a lot of time doing non-startup things like creating the profile then we can certainly cut that down without complicating things much.
Reporter | ||
Comment 2•9 years ago
|
||
for example in mochitest-other with run-by-dir we launch and terminate the browser 96 times: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d20914fd2b64 without it, we do it 3 times. This takes us from 75 to 105 minutes- adding a leak, timeout, etc. we cross the 120 minute threshold. 93 extra jobs adds 30 minutes, otherwise we add 20 seconds for startup/shutdown. possibly we could batch directories with <4 test files together to reduce the number of chunks we run.
Reporter | ||
Updated•6 years ago
|
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → INACTIVE
You need to log in
before you can comment on or make changes to this bug.
Description
•