Consider prioritising log parsing for some repository groups over others

RESOLVED WONTFIX

Status

Tree Management
Treeherder: Data Ingestion
P4
normal
RESOLVED WONTFIX
3 years ago
3 years ago

People

(Reporter: emorley, Unassigned)

Tracking

Details

(Reporter)

Description

3 years ago
Broken out from bug 1074927.

Certain repositories are more affected by a backlog of log parsing jobs more than others. eg: a push to Try can wait an extra 20 mins for log parsing if needs be (ideally it won't have to, but in worst case) - whereas a delay of that length will cause problems on repositories where changes stack on top of each other.

If we do this, I would suggest making "development" and "release stabilisation" repositories top priority, and type "try" lowest.

We'll also need to implement the cut-off time idea from bug 1076761 comment 0, to ensure that the jobs don't get indefinitely postponed in extreme load conditions.
(Reporter)

Updated

3 years ago
Blocks: 1080757
(Reporter)

Comment 1

3 years ago
With the additional workers & also bug 1076769, the log parsing times should be less of an issue, so this can wait a bit longer (it will help with the longer term scaling as we add more repos/data sources).
Priority: P2 → P3
(Reporter)

Updated

3 years ago
No longer blocks: 1080757
Component: Treeherder → Treeherder: Data Ingestion
(Reporter)

Updated

3 years ago
Priority: P3 → P4
(Reporter)

Comment 2

3 years ago
With the de-emphasis of log parsing inside treeherder (vs non-buildbot submitters submitting already-parsed logs), I don't think we'll need/want to do this, since bug 1076761 is the quicker win, should we need it.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.