1622140 - Intermittent [taskcluster:error] crashtest Task aborted - max run time exceeded

Status: RESOLVED INCOMPLETE

(Testing :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=292947990&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LhTTc9v6S9uKYnY-Nz-dfA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LhTTc9v6S9uKYnY-Nz-dfA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2020-03-13T00:46:46.523Z] 00:46:46 INFO - JavaScript error: resource://reftest/ReftestFissionChild.jsm, line 50: TypeError: Window.getComputedStyle: Argument 1 is not an object.
[task 2020-03-13T00:58:36.186Z] 00:58:36 INFO - JavaScript error: resource://reftest/ReftestFissionChild.jsm, line 50: TypeError: Window.getComputedStyle: Argument 1 is not an object.
[task 2020-03-13T01:10:29.763Z] 01:10:29 INFO - JavaScript error: resource://reftest/ReftestFissionChild.jsm, line 50: TypeError: Window.getComputedStyle: Argument 1 is not an object.
[taskcluster:error] Aborting task...
[task 2020-03-13T01:15:38.905Z] 01:15:38 INFO - ###!!! [Parent][MessageChannel] Error: (msgtype=0x37012A,name=PContent::Msg_AsyncMessage) Channel error: cannot send/recv
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 2244 (child process of PID 2228) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 4708 (child process of PID 2228) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 5784 (child process of PID 2228) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 4584 (child process of PID 2228) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 5480 (child process of PID 2228) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 2228 (child process of PID 1732) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 1732 (child process of PID 1096) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 3992 (child process of PID 2412) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 1096 (child process of PID 2412) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 2412 (child process of PID 3536) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 3536 (child process of PID 2260) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z] SUCCESS: The process with PID 2260 (child process of PID 2368) has been terminated.
[taskcluster 2020-03-13T01:15:38.919Z]
[taskcluster 2020-03-13T01:15:38.920Z] === Task Finished ===
[taskcluster 2020-03-13T01:15:38.920Z] Task Duration: 1h0m0.2905201s
[taskcluster 2020-03-13T01:15:39.254Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/octet-stream" and expiry 2021-03-13T00:08:15.680Z
[taskcluster 2020-03-13T01:15:39.413Z] Uploading artifact public/test_info/crashtest_errorsummary.log from file build\blobber_upload_dir\crashtest_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2021-03-13T00:08:15.680Z
[taskcluster 2020-03-13T01:15:39.587Z] Uploading artifact public/test_info/crashtest_raw.log from file build\blobber_upload_dir\crashtest_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2021-03-13T00:08:15.680Z
[taskcluster 2020-03-13T01:15:40.555Z] Uploading redirect artifact public/logs/live.log to URL https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LhTTc9v6S9uKYnY-Nz-dfA/runs/0/artifacts/public/logs/live_backing.log with mime type "text/plain; charset=utf-8" and expiry 2021-03-13T00:08:15.680Z
[taskcluster:error] Task aborted - max run time exceeded

Comment 1

[Cosmin Sabou [:CosminS]]

These started showing up since Bug 1440212 landed here: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C7%2Cshippable%2Copt%2Creftests%2Ctest-windows7-32-shippable%2Fopt-crashtest-e10s%2Cr%28c%29&fromchange=4bd0143d7cbe6e8271e36f449de2673e70d4b9b4&tochange=cb1605f13d5f56252b6dc89e2934f4c4aedac298&group_state=expanded&selectedJob=292949518
Normally a green job would take aroun 16-20 mins to complete: https://treeherder.mozilla.org/logviewer.html#?job_id=292947309&repo=autoland
But these on windows time out at 60 mins: https://treeherder.mozilla.org/logviewer.html#?job_id=292949518&repo=autoland
https://treeherder.mozilla.org/logviewer.html#?job_id=292947990&repo=autoland
https://treeherder.mozilla.org/logviewer.html#?job_id=292947966&repo=autoland
There were also these TVs on that push: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&searchStr=tv&revision=7cfecff122d2b0c90251b00eda8c1f82d28e27ad&selectedJob=292905403
Boris, is this something you can look into? If not, maybe Geoff? Thank you.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Hmm. So

        let style = this.contentWindow.getComputedStyle(msg.data.fromBrowsingContext.embedderElement);

is the line throwing these exceptions:

[task 2020-03-13T00:38:14.916Z] 00:38:14     INFO - JavaScript error: resource://reftest/ReftestFissionChild.jsm, line 50: TypeError: Window.getComputedStyle: Argument 1 is not an object.

I don't know whether those are related to anything in particular or not. but it looks like the test basically finishes in about 31 minutes, then ends up with that exception every 3-10 minutes for a bit, then gets aborted, right?

Thinking about this, but maybe Nika also has ideas?

It might be worth backing out part 2 of bug 1440212 for now; it's possible that the change to get the element from the frameloader there is affecting this.

Comment 3

[Cosmin Sabou [:CosminS]]

I went on and backed out part two of bug 1440212 until this is figured out, thanks Boris for looking into it.
Backout link: https://hg.mozilla.org/integration/autoland/rev/c690583844c68e59838084beb9af9f09ff01282b

Comment 4

[Intermittent Failures Robot]

6 failures in 5336 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6

Platform breakdown:

• windows7-32-shippable: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1622140&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 5

[Andrew Halberstadt [:ahal] (PTO Until Apr 3rd)]

Clearing stale needinfos in my triage components, please re-flag if still needed.

Comment 6

[Release mgmt bot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.