1603477 - windows 10 ccov mochitest-devtools sometimes exceeds its max-run-time

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Geoff Brown [:gbrown]]

In bug 1589796 there are some 150 minute task timeouts for windows 10 ccov mochitest-devtools:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=279763408&repo=mozilla-central&lineNumber=20350
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=279990182&repo=mozilla-central&lineNumber=17801
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=280118169&repo=mozilla-central&lineNumber=22256

Successful runs of chunk 16 usually run in 140+ minutes:
https://treeherder.mozilla.org/logviewer.html#?job_id=280856780&repo=mozilla-central

Other chunks run much faster -- some in as little as 30 minutes.

The problematic chunk runs only one manifest (devtools\client\webconsole\test\browser\browser.ini), so cannot be broken up further: additional chunks will not help, and changes to the scheduling algorithm will not help.

Non-ccov platforms run the problematic manifest much faster. Windows 10/debug runs the webconsole manifest in about 40 minutes.

Comment 1

[Geoff Brown [:gbrown]]

I will increase the max-run-time to give us some breathing room, but I am very concerned: 150 minutes is a long time to wait for a task to complete, and a lot of test time in general. Also, I just increased the test time for these tasks in October (bug 1588161), when these tests were running in 120 minutes.

:nchevobbe - We could probably schedule these tests better if they weren't all in one manifest. Could devtools\client\webconsole\test\browser\browser.ini be broken into 2 or more manifests? Also, any ideas for making these run faster?
:marco - Any ideas for making these run faster on Windows ccov? How would you feel about disabling this manifest on ccov?

Comment 2

[Marco Castelluccio [:marco]]

We have recently switched to opt builds for Linux, and we are going to switch to opt builds at some point (hopefully soon) for Windows too. This should help a lot.

Comment 3

[Marco Castelluccio [:marco]]

Bug 1457392 is tracking the switch to opt.

Comment 4

[Geoff Brown [:gbrown]]

Attachment: Bug 1603477 - Increase max-run-time for windows ccov devtools; r=

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/03ca0b24c449
Increase max-run-time for windows ccov devtools; r=marco

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/03ca0b24c449

Comment 7

[Nicolas Chevobbe [:nchevobbe]]

I can split this in multiple manifest for sure. Do we have other places where we do that? Do we have a rule of thumb on how to do such thing, how to name the files, … ?

Comment 8

[Geoff Brown [:gbrown]]

That's great, thanks!

Sorry, I don't have much guidance. I know of some examples of multiple manifests in one directory:

https://searchfox.org/mozilla-central/source/toolkit/components/search/tests/xpcshell/searchconfigs/xpcshell-1.ini (-2, -3, -4.ini)
https://searchfox.org/mozilla-central/source/dom/webgpu/mochitest/mochitest-no-pref.ini (mochitest.ini)

but I think those have generally been motivated by other considerations.

If you can think of some logical grouping you could also move a group of tests to a new sub-directory.

Comment 9

[Nicolas Chevobbe [:nchevobbe]]

Okay, I'll try to come up with something in Bug 1603765.