1076778 - Use a gevent worker for pushlog ingestion

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P1)

Description

[Mauro Doglio [:mdoglio]]

Attachment: New relic stats for fetch_hg_pushlog

A celery worker can use a prefork model or an async model . the first one is usually better for cpu bound tasks while the second one is usually better for i/o bound tasks. We are currently using the first model for everything other than the log parser.
According to new relic the fetch hg pushlog task is spending 95% of the time on the request to hg.m.o on average, so it would be a good idea to test in on dev and see how it performs.

Comment 1

[Mauro Doglio [:mdoglio]]

Attachment: Github PR #236 on treeherder-service

Comment 2

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/cfa29aaa656cdeed4602170ca1a1527a713c8ed4
Bug 1076778 - Use a gevent worker for pushlog ingestion

https://github.com/mozilla/treeherder-service/commit/893d89ce845ea5fecdd52d39ba6c885ed31142de
Merge pull request #236 from mozilla/bug-1076778-use-gevent-for-pushlog

Bug 1076778 - Use a gevent worker for pushlog ingestion