Bug 910614 (log-SnR)

[meta] Improve the SnR of the log error summaries by reducing spurious errors in the log

NEW
Unassigned

Status

5 years ago
a month ago

People

(Reporter: emorley, Unassigned)

Tracking

(Depends on: 2 bugs, {meta})

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

When a job runs, buildbot uses a number of criteria to determine whether it passed or failed (and if failed, with what state). TBPL imports this result & then parses the stdout using another set of regex [1] in order to generate a failure summary - which is what is displayed when clicking on the job in the main UI & is also used to generate the search terms for the bug suggestions for intermittent failures.

However, there are times when this summary's SnR is pretty poor:
1) Certain strings in the log are falsely matching against the TBPL regex, even though they weren't the real failure that caused the run to not succeed - distracting people from the real failure lines.
2) Some failure modes result in many garbage lines that don't add much to the understanding of the failure (eg return codes when we already have a stack, or logcat entries that are identical to the already output failure line).

This bug is to track reducing the cases of both #1 and #2 - and will primarily need changes to test harnesses/parts of automation as opposed to TBPL (since the regex is already about as good as we can get it).

[1] https://hg.mozilla.org/webtools/tbpl/file/tip/php/inc/GeneralErrorFilter.php
Depends on: 907650
Depends on: 813989
Depends on: 910389
Depends on: 910761
Depends on: 910830
Depends on: 913147
Depends on: 913641
Depends on: 914293
Depends on: 915260
Depends on: 915437
Depends on: 918981
Alias: TBPL-SnR
Depends on: 920191
Depends on: 927387
Depends on: 943474
Depends on: 944364
Depends on: 951181
Depends on: 951184
Depends on: 951187
Depends on: 982600
Depends on: 967318
Depends on: 991692
Depends on: 1006511
Depends on: 1017559
Depends on: 1019038
Depends on: 1026469
Depends on: 1027597
Depends on: 1028947
Depends on: 1032872
Depends on: 1047330
Depends on: 1048182
No longer depends on: 1032872
Depends on: 1053197
Depends on: 1059255
Depends on: 1063583
Depends on: 1078409
Product: Webtools → Tree Management
Summary: [meta] Improve the SnR of the TBPL annotated summary (mainly by reducing spurious errors in the log) → [meta] Improve the SnR of the TBPL/Treeherder annotated summary (mainly by reducing spurious errors in the log)
Depends on: 1091309
Depends on: 1123706
Depends on: 1139827
Alias: TBPL-SnR → log-SnR
Component: TBPL → General
Product: Tree Management → Testing
Summary: [meta] Improve the SnR of the TBPL/Treeherder annotated summary (mainly by reducing spurious errors in the log) → [meta] Improve the SnR of the log error summaries by reducing spurious errors in the log
Depends on: 1191514
Depends on: 1220941
Depends on: 1244746
See Also: → bug 778688
Keywords: meta
You need to log in before you can comment on or make changes to this bug.