972422 - Bump mock timeout

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Rail Aliiev [:rail]]

This happens for some mock commands:  

command timed out: 1200 seconds without output, attempting to kill
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1

remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: exceptions.RuntimeError: SIGKILL failed to kill process
]

Comment 1

[Rail Aliiev [:rail]]

Attachment: retry when mock times out

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 8375584 [details] [diff] [review]
retry when mock times out

Review of attachment 8375584 [details] [diff] [review]:
-----------------------------------------------------------------

I understand why we want these for MockInstall, as that's subject to network failures, but why do we want it for init or reset? If we start retrying things like that, we should start retrying purge builds, clobberer, etc. too, I think.

Comment 4

[Rail Aliiev [:rail]]

We talked about this offline and found that the error is not related to spot disconnects. The error comes from the slave when it tries to kill mock but unable to do so. This happens when you try to stop a build when mock install is running -- it won't stop. Probably it's related to the fact that mock is a suid program.

I'm  going to update the patch and just bump the timeuot.

Comment 5

[Rail Aliiev [:rail]]

Attachment: bump timeout

default is 1200 (from https://hg.mozilla.org/build/buildbotcustom/file/5d08b50a1531/steps/mock.py#l218)

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 8375787 [details] [diff] [review]
bump timeout

Review of attachment 8375787 [details] [diff] [review]:
-----------------------------------------------------------------

Might be worth bumping the default for MockInstall instead or in addition?

Comment 7

[Rail Aliiev [:rail]]

Comment on attachment 8375787 [details] [diff] [review]
bump timeout

https://hg.mozilla.org/build/buildbotcustom/rev/2b7e56065832

Comment 8

[Rail Aliiev [:rail]]

in production