Closed Bug 1076774 Opened 5 years ago Closed 5 years ago

Consider segregating workers for more critical tasks (eg pushlog & job result ingestion)

Categories

(Tree Management :: Treeherder, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: mdoglio)

References

Details

Attachments

(1 file)

Saw an IRC comment about us possibly having tasks like the pushlog ingestion in another VM to increase reliability.

I don't have a particular gut feeling either way - happy to defer to others on whether we should do this - but filing this bug so we don't forget about the idea.

As for how we define "critical tasks" - I would definitely call pushlog ingestion critical, and following closely behind, job result ingestion. Log parsing is definitely lower down the list.
Summary: Consider segregating workers for more critical tasks (eg pushlog ingestion) → Consider segregating workers for more critical tasks (eg pushlog & job result ingestion)
Blocks: 1075799
Blocks: 1073015
Assignee: nobody → mdoglio
Status: NEW → ASSIGNED
I implemented the pushlog-related change in bug 1076778. I will track the buildapi part here
Attachment #8500412 - Flags: review?(cdawson)
Depends on: 1076778
Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/9555d428725a4eaf7cec24c65670a27b8ca3acd6
Bug 1076774 - use a separate worker for jobs ingestion

https://github.com/mozilla/treeherder-service/commit/7546d5d74a400542d653155377f125fe22e517b6
Merge pull request #241 from mozilla/bug-1076774-implement-buildapi-queue

Bug 1076774 - use a separate worker for jobs ingestion
Attachment #8500412 - Flags: review?(cdawson)
We have requested the VMs for the additional workers and changed the code where needed.
I will close this bug once everything is in place on production
Great - thank you :-)
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.