1500177 - linux64/ccov web-platform-tests-e10s-4 (or e10s-1) frequently exceeds task max-run-time

Status: RESOLVED FIXED

(Testing :: General, defect, P1)

Description

[Geoff Brown [:gbrown]]

Bug 1411358 shows multiple recent cases of linux64/ccov wpt-4 timing out after 7200 seconds (120 minutes). Recent green runs of this task run in 110+ minutes. Other linux64/ccov wpt chunks run in 60 - 90 minutes, so adding more chunks may not be a good option.

https://treeherder.mozilla.org/logviewer.html#?job_id=206335195&repo=mozilla-central
https://treeherder.mozilla.org/logviewer.html#?job_id=206200103&repo=mozilla-central
https://treeherder.mozilla.org/logviewer.html#?job_id=205188757&repo=mozilla-central

Comment 1

[Geoff Brown [:gbrown]]

Here are the tests that took >10000 ms in a recent run of wpt-4:

10055	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1989.html?15001-16000 
10295	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc_5601.html?16001-17000 
10308	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-han.html?22001-23000 
10352	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-korean.html?15001-16000 
10363	/shadow-dom/untriaged/html-elements-in-shadow-trees/inert-html-elements/test-001.html 
10477	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc5601.html?14001-15000 
10615	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-han.html?19001-20000 
10666	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-iso-ir-149.html?14001-15000 
10894	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc5601.html?16001-17000 
10911	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1987.html?13001-14000 
11234	/encoding/legacy-mb-korean/euc-kr/euckr-encode-form.html?15001-16000 
13956	/worklets/paint-worklet-referrer.https.html 
14642	/worklets/layout-worklet-referrer.https.html 
15363	/worklets/audio-worklet-referrer.https.html 
16130	/worklets/animation-worklet-referrer.https.html 
26864	/referrer-policy/css-integration/svg/presentation-attribute.html 
29878	/referrer-policy/css-integration/svg/processing-instruction.html 
29925	/referrer-policy/css-integration/svg/internal-stylesheet.html 
29972	/referrer-policy/css-integration/svg/inline-style.html 
30211	/referrer-policy/css-integration/svg/external-stylesheet.html 
59598	/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html

Comment 2

[Geoff Brown [:gbrown]]

:jmaher - I'm not sure what to do about this one. Ideas? Or, who could investigate??

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I am unable to get runtimes for the tests, in general, I think we should disable the encoding tests, but I wanted to see what the runtimes were on different configs to see if this is osx specific or ccov specific or both.  :gbrown, does |./mach test-info| work for you?  It didn't for me after a few tries:
$ ./mach test-info testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html
===== testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html =====
Found testing\web-platform\tests\encoding\legacy-mb-korean\euc-kr\euckr-decode-ksc_5601.html in source control.
testing\web-platform\tests\encoding\legacy-mb-korean\euc-kr\euckr-decode-ksc_5601.html not found in any test manifest!
Querying ActiveData...
Unable to find matching records in ActiveData; using testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html!

Test results for testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html on mozilla-central,mozilla-inbound,autoland between 2018-10-19 and 2018-10-26
No test result data found.

Test durations for testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html on mozilla-central,mozilla-inbound,autoland between 2018-10-19 and 2018-10-26
No test durations found.

Test tasks for testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html on mozilla-central,mozilla-inbound,autoland between 2018-10-19 and 2018-10-26
No test tasks found.

Bugzilla quick search for 'testing\web-platform\tests\encoding\legacy-mb-korean\euc-kr\euckr-decode-ksc_5601.html,testing/web-platform/tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html':

Comment 4

[Geoff Brown [:gbrown]]

I think test-info is still broken -- :ekyle is working on bug 1500995.

Comment 5

[Geoff Brown [:gbrown]]

Timeouts recently shifted to chunk 1.

Comment 6

[Geoff Brown [:gbrown]]

Attachment: increase max-run-time from 2 hours to 3 hours

I also tried increasing the chunks to 24, but there was still 1 chunk taking about 2 hours.

Comment 7

[Marco Castelluccio [:marco]]

Comment on attachment 9023296 [details] [diff] [review]
increase max-run-time from 2 hours to 3 hours

Review of attachment 9023296 [details] [diff] [review]:
-----------------------------------------------------------------

We disabled those encoding tests for the Mac builds because they took too long.
I'm OK with increasing the max-run-time, but I would also be OK with disabling them for the time being.

Comment 8

[Geoff Brown [:gbrown]]

Thanks :marco. I'm looking for a quick win here so I'll go ahead with this patch.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/38eec54ce70b
Increase task max-run-time for linux64/ccov wpt; r=marco

Comment 10

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/38eec54ce70b