Closed Bug 1432430 Opened 6 years ago Closed 5 years ago

Intermittent tp6_facebook_heavy command timed out: 7200 seconds elapsed | Aborting task - max run time exceeded!

Categories

(Testing :: Talos, defect, P5)

Product:
Testing
Component:
Talos
Version:
Version 3
Type:
defect

Tracking

(firefox59 disabled, firefox60 disabled)

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox59 --- disabled
firefox60 --- disabled

People

(Reporter: dluca, Assigned: jmaher)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell unknown])

Attachments

(1 file)

disable h2 on osx for perma fail on timeout
781 bytes, patch
rwood
: review+
Details | Diff | Splinter Review 
+++ This bug was initially created as a clone of Bug #1428790 +++

https://treeherder.mozilla.org/logviewer.html#?job_id=157904426&repo=mozilla-inbound#L12306

TEST-START | tp6_facebook_heavy
22:29:40     INFO -  Initialising browser for tp6_facebook_heavy test...
22:29:54     INFO -  Local copy of 'simple' is fresh enough
22:29:54     INFO -  0 days old
22:29:54     INFO -  Cloning profile located at /Users/cltbld/.mozilla/profiles/simple
22:29:54     INFO -  => bookmarkbackups
22:29:54     INFO -  => browser-extension-data
22:29:54     INFO -  => browser-extension-data/screenshots@mozilla.org
22:29:54     INFO -  => cache2
22:29:54     INFO -  => cache2/doomed
22:29:54     INFO -  => cache2/entries
[taskcluster 2018-01-23T07:03:33.227Z] Aborting task - max run time exceeded!
Somebody should look into the history of this job and see if the time to run has been increasing. It's suspicious that it started failing on Windows, then Linux, and now OS X as well.
Summary: Intermittent command timed out: 7200 seconds elapsed running ['/tools/buildbot/bin/python', 'scripts/scripts/talos_script.py', '--suite', 'h2-e10s', '--add-option', '--webServer,localhost', '--branch-name', 'Mozilla-Inbound-Non-PGO', '--cfg' → Intermittent tp6_facebook_heavy command timed out: 7200 seconds elapsed running ['/tools/buildbot/bin/python', 'scripts/scripts/talos_script.py', '--suite', 'h2-e10s', '--add-option', '--webServer,localhost', '--branch-name',
Summary: Intermittent tp6_facebook_heavy command timed out: 7200 seconds elapsed running ['/tools/buildbot/bin/python', 'scripts/scripts/talos_script.py', '--suite', 'h2-e10s', '--add-option', '--webServer,localhost', '--branch-name', → Intermittent tp6_facebook_heavy command timed out: 7200 seconds elapsed | Aborting task - max run time exceeded!
This bug has failed 37 times in the last 7 days only on OS X 10 affecting opt build type.
Failing tests: opt-talos-tp6-e10, opt-talos-h2-e10s.

Lin to the latest log:https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=160022649&lineNumber=1419

Part of that log:
 INFO -  mozcrash Saved minidump as /Users/cltbld/tasks/task_1517546859/build/blobber_upload_dir/D8BF1E5B-57B4-45A7-938B-4041F9376B0A.dmp
22:18:42     INFO -  PROCESS-CRASH | tp6_google_heavy | application crashed [unknown top frame]
22:18:42     INFO -  Crash dump filename: /var/folders/fd/kt26_m457rl243vm27v2bssr00000w/T/tmpZq0gcF/profile/minidumps/D8BF1E5B-57B4-45A7-938B-4041F9376B0A.dmp
22:18:42     INFO -  stderr from minidump_stackwalk:
22:18:42     INFO -  2018-02-01 22:18:39: minidump.cc:4359: INFO: Minidump opened minidump /var/folders/fd/kt26_m457rl243vm27v2bssr00000w/T/tmpZq0gcF/profile/minidumps/D8BF1E5B-57B4-45A7-938B-4041F9376B0A.dmp
22:18:42     INFO -  2018-02-01 22:18:39: minidump.cc:4808: ERROR: ReadBytes: read 0/32
22:18:42     INFO -  2018-02-01 22:18:39: minidump.cc:4453: ERROR: Minidump cannot read header
22:18:42     INFO -  2018-02-01 22:18:39: stackwalk.cc:133: ERROR: Minidump /var/folders/fd/kt26_m457rl243vm27v2bssr00000w/T/tmpZq0gcF/profile/minidumps/D8BF1E5B-57B4-45A7-938B-4041F9376B0A.dmp could not be read
22:18:42     INFO -  2018-02-01 22:18:39: minidump.cc:4331: INFO: Minidump closing minidump
22:18:42     INFO -  minidump_stackwalk exited with return code 1
22:18:42     INFO -  TEST-UNEXPECTED-ERROR | tp6_google_heavy | Found crashes after test run, terminating test

:jmaher could you please take a look, because the test is still failing even if it is disabled.
Flags: needinfo?(jmaher)
oh, this is almost perma fail, lets disable on osx.
Flags: needinfo?(jmaher)
ok, finally off for osx.
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8947926 - Flags: review?(rwood)
Comment on attachment 8947926 [details] [diff] [review]
disable h2 on osx for perma fail on timeout

Review of attachment 8947926 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!
Attachment #8947926 - Flags: review?(rwood) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1149bbf594f5
disable h2 on osx for near perma fail timing out. r=rwood
Keywords: leave-open
In the last 7 days we have 38 failures.
They occur on macosx64-nightly (opt) and OS X 10.10 (opt and debug).
A recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=165557481
I think this is the nightly profiling jobs acting up- outside of that it looks like tp6, lets see what things look like with the profiling jobs working.
Flags: needinfo?(rwood)
Whiteboard: [stockwell disabled] → [stockwell needswork]
Ah, yes again the tp6 failure is the same issue as seen in Bug 1439979 - a problem with non-replay 404 requests. Follow along in that bug for progress. Once that's fixed it should remove several of these tp6 intermittents.
Flags: needinfo?(rwood)

we disabled the heavy profile tests due to many issues (lack of bugs found, too many intermittents, hard to maintain, runtime)

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: