1242502 - Bump mochitest time outs

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Armen [:armenzg]]

From https://bugzilla.mozilla.org/show_bug.cgi?id=1204281#c106

Comment 1

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=327e38114df3

Comment 2

[Armen [:armenzg]]

Hi ahal, I believe I switched to --chunk-by-runtime properly, however, I don't see the chunks being closer in times than before:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d342f527e10f&filter-searchStr=mochitest&group_state=expanded

Notice that I've also switched to 15 chunks for TC jobs.

From buildbot:
/builds/slave/test/build/venv/bin/python -u /builds/slave/test/build/tests/mochitest/runtests.py --total-chunks 8 --this-chunk 8 --appname=/builds/slave/test/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=/builds/slave/test/build/symbols --certificate-path=tests/certs --setpref=webgl.force-enabled=true --quiet --log-raw=/builds/slave/test/build/blobber_upload_dir/plain-chunked_raw.log --log-errorsummary=/builds/slave/test/build/blobber_upload_dir/plain-chunked_errorsummary.log --use-test-media-devices --screenshot-on-fail --chunk-by-runtime

From TaskCluster:
/home/worker/workspace/build/venv/bin/python -u /home/worker/workspace/build/tests/mochitest/runtests.py --total-chunks 15 --this-chunk 6 --appname=/home/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/F28CEvn3Q3qSBuHZTqV9yg/artifacts/public/build/target.crashreporter-symbols.zip --certificate-path=tests/certs --setpref=webgl.force-enabled=true --quiet --log-raw=/home/worker/workspace/build/blobber_upload_dir/plain-chunked_raw.log --log-errorsummary=/home/worker/workspace/build/blobber_upload_dir/plain-chunked_errorsummary.log --use-test-media-devices --screenshot-on-fail --chunk-by-runtime

Comment 3

[Armen [:armenzg]]

Attachment: MozReview Request: Bug 1242502 - Bump TC Linux64 debug mochitest (plain and e10s) timeout from 60 to 90 minutes. DONTBUILD. r=jmaher

We were seeing too many time outs.

Review commit: https://reviewboard.mozilla.org/r/32331/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/32331/

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8711824 [details]
MozReview Request: Bug 1242502 - Bump TC Linux64 debug mochitest (plain and e10s) timeout from 60 to 90 minutes. DONTBUILD. r=jmaher

https://reviewboard.mozilla.org/r/32331/#review28989

awesome

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/76271213e893

Comment 6

[Andrew Halberstadt [:ahal]]

I suspect it is turned on. They look closer than before to me, at least for buildbot linux opt. On central I saw the following job times:
30 67 31 26 38

In your try push, I saw:
22 58 27 21 24

That obviously isn't perfect, but it's an improvement. I bet that 58 is from a really large test manifest, the algorithm won't split up any manifests. You'd probably also get better results by changing the PERCENTILE [1] to something lower. Doing this comes at the cost of larger runtimes file sizes. Try lowering that number to 0, and re-generate the files and do a new try push. You should see a better result and that way you'll know it's enabled.

[1] https://dxr.mozilla.org/mozilla-central/source/testing/runtimes/writeruntimes.py#12

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/76271213e893

Comment 8

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ea4dfe1cc337

Comment 9

[Armen [:armenzg]]

Thanks ahal.
I was focusing on the longest running task and didn't realized that it was actually working.

I'm trying now with 0.01 percentile.

Comment 10

[Armen [:armenzg]]

Comment on attachment 8711824 [details]
MozReview Request: Bug 1242502 - Bump TC Linux64 debug mochitest (plain and e10s) timeout from 60 to 90 minutes. DONTBUILD. r=jmaher

https://reviewboard.mozilla.org/r/32331/#review29205

This has already landed. I'm just -reclosing the Mozreview request.

Comment 11

[Armen [:armenzg]]

Moving to bug 1243080.