1519162 - Remove the concept of log steps (TextLogStep)

Status: RESOLVED DUPLICATE of bug 1633041

(Tree Management :: Treeherder, enhancement, P3)

Description

[Ed Morley [:emorley]]

Currently when logs are parsed, the log parser looks for buildbot style step markers, and uses those to split up the logs into sections, that are stored in the TextLogStep model. Any errors are then associated with those steps.

However Taskcluster never made use of the steps (jobs create a dummy single unnamed step), so with buildbot's EOL they no longer contain useful information.

The UI for log steps was removed in bug 1450045, however we should remove the corresponding support from the log parser and models.

The models parts will involve making TextLogError foreign key on Job directly (with associated schema migrations/backfilling of FKs), followed by removal of TextLogStep.

However a good first step would be to just simplify the log parser (which should also help improve its performance - which is notable since it's the biggest compute consumer for background tasks).

Comment 1

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #0)

However a good first step would be to just simplify the log parser

eg removing most of:
https://github.com/mozilla/treeherder/blob/33921b52fd264236577cf8fc91d58f28f78c23a0/treeherder/log_parser/parsers.py#L40-L253

Comment 2

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #0)

However a good first step would be to just simplify the log parser (which should also help improve its performance - which is notable since it's the biggest compute consumer for background tasks).

Marking blocking bug 1407377 due to this.

Comment 3

[Sarah Clements [:sclements]]

All of the changes mentioned above will be covered in bug 1633041.