1272183 - Busy loop in autophonetreeherder

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Geoff Brown [:gbrown]]

The treeherder submission server loops, sleeping following failed submissions. But following successful submissions, there is no sleep, even if there are no jobs available, resulting in a tight loop, and high cpu use.

This is a regression from bug 1267531.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: sleep between calls to jobs.get_next_treeherder_job()

Currently serve_forever() calls jobs.get_next_treeherder_job() in a tight loop unless post_request() fails. Prior to bug 1267531, serve_forever() would sleep for retry_wait seconds between calls to jobs.get_next_treeherder_job().

In this patch, I'm sleeping for just 1 second between job retrieval attempts -- I think it's enough of a rest, keeps job submission snappy, and decouples avoiding-a-busy-loop from wait-after-failure (retry_wait).

This significantly reduces autophone cpu usage for me.

Comment 2

[Bob Clary [:bc] (inactive)]

Comment on attachment 8751798 [details] [diff] [review]
sleep between calls to jobs.get_next_treeherder_job()

Review of attachment 8751798 [details] [diff] [review]:
-----------------------------------------------------------------

r+. great! I'd noticed the cpu issue but hadn't looked into it yet. Thanks for fixing it already!

Comment 3

[Geoff Brown [:gbrown]]

https://github.com/mozilla/autophone/commit/df8051bc5ddb0421f99e65bf571c3435fdda60f3

Comment 4

[Bob Clary [:bc] (inactive)]

deployed 2016-05-12 14:00:00