Use a gevent worker for pushlog ingestion

RESOLVED FIXED

Status

Tree Management
Treeherder
P1
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: mdoglio, Assigned: mdoglio)

Tracking

Details

Attachments

(2 attachments)

(Assignee)

Description

4 years ago
Created attachment 8498805 [details]
New relic stats for fetch_hg_pushlog

A celery worker can use a prefork model or an async model . the first one is usually better for cpu bound tasks while the second one is usually better for i/o bound tasks. We are currently using the first model for everything other than the log parser.
According to new relic the fetch hg pushlog task is spending 95% of the time on the request to hg.m.o on average, so it would be a good idea to test in on dev and see how it performs.
(Assignee)

Updated

4 years ago
Blocks: 1076750

Updated

4 years ago
Priority: -- → P1
(Assignee)

Updated

4 years ago
Assignee: nobody → mdoglio
Status: NEW → ASSIGNED
(Assignee)

Comment 1

4 years ago
Created attachment 8498891 [details] [review]
Github PR #236 on treeherder-service
Attachment #8498891 - Flags: review?(jeads)

Updated

4 years ago
Blocks: 1076774

Comment 2

4 years ago
Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/cfa29aaa656cdeed4602170ca1a1527a713c8ed4
Bug 1076778 - Use a gevent worker for pushlog ingestion

https://github.com/mozilla/treeherder-service/commit/893d89ce845ea5fecdd52d39ba6c885ed31142de
Merge pull request #236 from mozilla/bug-1076778-use-gevent-for-pushlog

Bug 1076778 - Use a gevent worker for pushlog ingestion

Updated

4 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Updated

4 years ago
Attachment #8498891 - Flags: review?(jeads)
You need to log in before you can comment on or make changes to this bug.