Consider increasing the number of processor worker processes

RESOLVED FIXED

Status

Tree Management
Treeherder: Data Ingestion
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: wlach, Assigned: wlach)

Tracking

Details

Attachments

(1 attachment)

Right now we're still seeing log backlogs due to bug 1296434. The root cause is described there, but it also highlights that fact that we're underutilizing our log processing workers. Based on my interpretation, we're not even using 50% of the CPU on our workers (under normal conditions). Let's experiment by doubling the number of workers and see if it improves things.
Created attachment 8791626 [details] [review]
[treeherder] wlach:1303055 > mozilla:master
Comment on attachment 8791626 [details] [review]
[treeherder] wlach:1303055 > mozilla:master

I am currently testing this patch on stage and it seems to be working well.
Attachment #8791626 - Flags: review?(emorley)

Updated

2 years ago
Attachment #8791626 - Flags: review?(emorley) → review+

Comment 3

2 years ago
Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/0300fb69bad7223585fd9e9c885b5a159b292221
Bug 1303055 - Increase the number of log processing workers (#1857)

This should increase throughput, especially in cases where we are backlogged
due to network latency.

Updated

2 years ago
Assignee: nobody → wlachance
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.