Closed Bug 1276486 Opened 5 years ago Closed 5 years ago

Add 'extra_highlights' field to the job object

Categories

(Tree Management :: Treeherder: API, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: ahal, Unassigned)

Details

It would be awesome if we could define a list of extra regexes to use to determine which log lines to highlight directly in the taskcluster configs.

Currently, jobs need to use ugly log formatters so that treeherder will recognize which lines to highlight. For example:
https://dxr.mozilla.org/mozilla-central/source/tools/lint/eslint-formatter.js

The default eslint formatter is much more readable, unfortunately we can't use it. This ability would also allow us to move off the mozlog tbplformatter without requiring treeherder<->mozlog integration (at least for taskcluster jobs).
I don't think we want to do this. 

This would need a bit of a rewrite of the log parser (and log viewer), plus is meaning we'd be storing error handling logic in the Treeherder DB rather than only in code -- which just seems pretty wrong at first glance.

To me, preferable solutions would be:
1) Submitters do their own error summarisation (in the future we see Treeherder not parsing logs at all)
2) Submitting the logs as structured logs (we really should be reducing the number of jobs that use unstructured logs, not increasing them).
3) Using mozharness (or is it mozlog) style prefixed logs (ie with the error level prefix), such that Treeherder's log parser doesn't need a hardcoded list of regex, but simply matches against the ERROR level lines
4) (As a last resort) Adding to Treeherder's existing regex as a one-off

Open to further reasoning though! ;-)
s/;-)/:-)/
(In reply to Ed Morley [:emorley] from comment #1)
> This would need a bit of a rewrite of the log parser (and log viewer), plus
> is meaning we'd be storing error handling logic in the Treeherder DB rather
> than only in code -- which just seems pretty wrong at first glance.

To clarify, my hope was to get taskcluster to pass this data in via API call, e.g here:
https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/tests/eslint-gecko.yml#36

I didn't think that would need to get stored in a DB anywhere, but I have 0 knowledge of treeherder's architecture :). I may have also horribly mis-worded the title. I was hoping this would be an easy fix, but if it's not then I guess we can close this..


> To me, preferable solutions would be:
> 1) Submitters do their own error summarisation (in the future we see
> Treeherder not parsing logs at all)

I'd love to be able to do this, but I didn't think it was possible. In fact, this is the problem I was trying to solve with this bug! Is there a way to do this now?


> 2) Submitting the logs as structured logs (we really should be reducing the
> number of jobs that use unstructured logs, not increasing them).

For the record, the jobs I'm thinking about are lint jobs which don't really make sense under the mozlog model (there is no concepts of "suite" or "test"). We could probably shoe-horn it in, but it would be kind of awkward. The lint jobs will be able to produce json-lines, but it will be a different structure than mozlog raw.
Could you describe in some more detail the user-facing behaviour you want to enable? I don't know what you mean by "highlight". Do you mean specify error lines that will get shown in the treeherder jobs panel, log viewer, or both?

It is indeed possible to do your own log parsing at the moment and submit your own error summary, but the procedure is undocumented and I'm not sure if I recommend it.
Flags: needinfo?(ahalberstadt)
Both I suppose, I sort of figured they used the same set of regexes. But yes, the blue "Failure summary" panel and the lines in the log viewer.
Flags: needinfo?(ahalberstadt)
I don't think we should do this for the reasons in comment 1.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.