897448 - release-runner should retry "make update" fabric actions

Status: RESOLVED DUPLICATE of bug 900693

(Release Engineering :: Release Automation: Other, defect)

Description

[Rail Aliiev [:rail]]

Attachment: update.diff

We hit this issue at least a couple of times when make update fails due to 500 errors from hg.m.o.

Attached in an untested patch.

I have 2 concerns so far:

1) I always use retry (attempt=1 for normal calls). I used this approach to reduce if/else blocks with boilerplate.

2) Maybe instead of modifying FabricHelper I should decorate action_update with @retriable?

Comment 1

[[e-mail] (:bhearsum)]

Comment on attachment 780352 [details] [diff] [review]
update.diff

Review of attachment 780352 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Rail Aliiev [:rail] from comment #0)
> 1) I always use retry (attempt=1 for normal calls). I used this approach to
> reduce if/else blocks with boilerplate.

I don't see anything wrong with this part.

> 2) Maybe instead of modifying FabricHelper I should decorate action_update
> with @retriable?

Will this allow the caller to change the # of retries? I think this is important, and your current patch doesn't allow for it either.

Another option here is to have the master's Makefile do the retries. I'm not sure I like that as much though...

feedback+ on this, but regardless of implementation I'd like it to be possible for the caller to change the # of retries.