Closed Bug 1372922 Opened 5 years ago Closed 5 years ago

Reftest logs suite_start many times if all tests crash

Categories

(Testing :: Reftest, enhancement)

enhancement
Not set
normal

Tracking

(firefox56 fixed)

RESOLVED FIXED
mozilla56
Tracking Status
firefox56 --- fixed

People

(Reporter: ahal, Assigned: ahal)

References

Details

Attachments

(1 file)

In bug 1372668, emorley found an enormous 236MB log:
https://queue.taskcluster.net/v1/task/DBYbDF8RR8CrQL9tREtDaA/runs/0/artifacts/public/test_info/reftest-no-accel_errorsummary.log

We should figure out why that is happening and prevent it.
Blocks: 1347945
I'm still digging into that log, but looks like we cycle this:

1) suite-start
2) test-start
3) crash
4) go back to 1)

That seemed to happen for every test. The crash stacks from 3) definitely increase the log size, but actually aren't the worst offender. When a second suite-start gets logged, we do this:
https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozlog/mozlog/structuredlog.py#257

That 'data' we json.dumps in the message, contains every single reftest id (/abspath/to/file1 == /abspath/to/file2) which is huge (especially since it's not chunked). There are a bunch of things we can do to fix this:

1. Stop dumping full data (either cut off after a certain point, or don't dump test ids at all) when logging a second suite-start.
2. Fix reftest harness to not log suite-start again in case the first test crashes (need to dig in some more, but I assume that's what's going on).
3. Fix reftest harness to abort a test run after N crashes in a row.
Pretty sure the multiple suite_start's was happening because the first (and subsequent tests) all crashed. I have to dig into this some more, but suite_start should never be logged more than once. So I think it's safe to make this bug about fixing that.
Summary: Investigate enormous errorsummary.log generated from reftest-no-accel → Reftest logs suite_start many times if all tests crash
Depends on: 1344991
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
Attachment #8877670 - Flags: review?(shing.lyu) → review?(jmaher)
Comment on attachment 8877670 [details]
Bug 1372922 - Don't log suite_start a second time in reftest if recovering from a crash,

https://reviewboard.mozilla.org/r/149118/#review155680

oh, good find
Attachment #8877670 - Flags: review?(jmaher) → review+
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4bceea59d062
Don't log suite_start a second time in reftest if recovering from a crash, r=jmaher
https://hg.mozilla.org/mozilla-central/rev/4bceea59d062
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
You need to log in before you can comment on or make changes to this bug.