Closed Bug 1059788 Opened 7 years ago Closed 7 years ago

Log parser doesn't truncate extreme length log lines when extracting errors & search terms

Categories

(Tree Management :: Treeherder, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: KWierso, Mentored)

References

()

Details

(Whiteboard: [good first bug])

Attachments

(1 file)

TBPL truncates extreme length log lines, when extracting the error lines from the log, ie:

https://hg.mozilla.org/webtools/tbpl/file/9cb4d4589a82/php/inc/LogParser.php#l41
    $maxLineLength = 500;
    $canTruncate = ($type != "tinderbox_print" && $type != "reftest");
...
      if ($canTruncate && strlen($matchedline) > $maxLineLength) {
        // Cap the line length to avoid hanging the browser in extreme cases.
        $matchedline = rtrim(substr($matchedline, 0, $maxLineLength)) . "... [exceeded max length]\n";
      }

In addition, the search term length is capped:

https://hg.mozilla.org/webtools/tbpl/file/9cb4d4589a82/php/inc/AnnotatedSummaryGenerator.php#l167
    // Searching for extremely long search terms is undesirable, since:
    // a) We end up spamming Bugzilla's REST API.
    // b) Bugzilla's max summary length is 256 characters, and once "Intermittent "
    // and platform/suite information is prefixed, there are even fewer characters
    // left for us to use for the failure string against which we need to match.
    // c) For long search terms, the additional length does little to prevent against
    // false positives, but means we're more susceptible to false negatives due to
    // run-to-run variances in the error messages (eg paths, process IDs).
    $searchTerm = trim(substr($searchTerm, 0, 100));

The former protects against UI hangs & helps with readability of failure summaries in the UI. The latter improves bug suggestions.
Blocks: 1057359
Mentor: cdawson
Whiteboard: [good first bug]
I think we may already do the first part of comment 0:
https://github.com/mozilla/treeherder-service/blob/0a70cf6bf9b0b1d246f6901d2ce3536445eb7f88/treeherder/log_parser/artifactbuilders.pyx#L35

Though we'll need to check whether the implementation there impacts the reftest analyser (TBPL excludes these failure lines from truncation - bug 1001441).
Whether this bug blocks or not comes down to whether treeherder can offer bug suggestions for these top-oranges that I suspect might only be caught with a reduced search term (need to double check):

bug 1051908
bug 1054456
bug 1059287
I don't know python, but I think this might possibly limit the search terms to 100 characters.
Attachment #8486789 - Flags: feedback?(emorley)
Attachment #8486789 - Flags: feedback?(cdawson)
Attachment #8486789 - Flags: feedback?(cdawson) → feedback+
Attachment #8486789 - Flags: feedback?(emorley) → feedback+
https://github.com/mozilla/treeherder-service/commit/e35938277af77166d813b4321b9eb036d910a1c4
Assignee: nobody → kwierso
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.