Closed Bug 1372922 Opened 6 years ago Closed 6 years ago
Reftest logs suite
_start many times if all tests crash
59 bytes, text/x-review-board-request
In bug 1372668, emorley found an enormous 236MB log: https://queue.taskcluster.net/v1/task/DBYbDF8RR8CrQL9tREtDaA/runs/0/artifacts/public/test_info/reftest-no-accel_errorsummary.log We should figure out why that is happening and prevent it.
I'm still digging into that log, but looks like we cycle this: 1) suite-start 2) test-start 3) crash 4) go back to 1) That seemed to happen for every test. The crash stacks from 3) definitely increase the log size, but actually aren't the worst offender. When a second suite-start gets logged, we do this: https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozlog/mozlog/structuredlog.py#257 That 'data' we json.dumps in the message, contains every single reftest id (/abspath/to/file1 == /abspath/to/file2) which is huge (especially since it's not chunked). There are a bunch of things we can do to fix this: 1. Stop dumping full data (either cut off after a certain point, or don't dump test ids at all) when logging a second suite-start. 2. Fix reftest harness to not log suite-start again in case the first test crashes (need to dig in some more, but I assume that's what's going on). 3. Fix reftest harness to abort a test run after N crashes in a row.
Some more links for the job in question: * https://tools.taskcluster.net/task-inspector/#DBYbDF8RR8CrQL9tREtDaA/ * https://treeherder.mozilla.org/#/jobs?repo=try&revision=c1aea65e222b1b40e1db2701e0d8cfdfccd669dc&selectedJob=106463464 * https://queue.taskcluster.net/v1/task/DBYbDF8RR8CrQL9tREtDaA/runs/0/artifacts/public/logs/live_backing.log
Pretty sure the multiple suite_start's was happening because the first (and subsequent tests) all crashed. I have to dig into this some more, but suite_start should never be logged more than once. So I think it's safe to make this bug about fixing that.
Summary: Investigate enormous errorsummary.log generated from reftest-no-accel → Reftest logs suite_start many times if all tests crash
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
Attachment #8877670 - Flags: review?(shing.lyu) → review?(jmaher)
Comment on attachment 8877670 [details] Bug 1372922 - Don't log suite_start a second time in reftest if recovering from a crash, https://reviewboard.mozilla.org/r/149118/#review155680 oh, good find
Attachment #8877670 - Flags: review?(jmaher) → review+
Pushed by email@example.com: https://hg.mozilla.org/integration/autoland/rev/4bceea59d062 Don't log suite_start a second time in reftest if recovering from a crash, r=jmaher
You need to log in before you can comment on or make changes to this bug.