Mozharness should release mozpool requests when it gives up waiting for the device to be ready

RESOLVED FIXED

Status

Release Engineering
Mozharness
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: Callek, Assigned: Callek)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

5 years ago
Created attachment 781385 [details] [diff] [review]
[mozharness] cancel the requests when not ready!

So, currently we issue mozpool requests, valid for 4 hours.

When mozpool takes a long time to get us a device, mozharness gives up. However mozpool still thinks the request-for-device is valid and will happily hold the device aside for the full 4 hours, even if it became ready after the fact.

This prevents future jobs from getting the device. And afaict the requests then stack up... We need to close the request when waiting-for-ready fails.

This won't fix any of the issues that cause the device to not be ready, but hopefully this will allow us to not have 100s of RETRIES when it happens.
Attachment #781385 - Flags: review?(aki)

Comment 1

5 years ago
Comment on attachment 781385 [details] [diff] [review]
[mozharness] cancel the requests when not ready!

>+        if fail_cb:
>+            fail_cb()

Maybe |if fail_cb and callable(fail_cb):|, but this is fine too.
Attachment #781385 - Flags: review?(aki) → review+
(Assignee)

Comment 2

5 years ago
Landed with |assert callable(fail_cb)| instead after IRC chat.

https://hg.mozilla.org/build/mozharness/rev/c1a932eb4239

Awaits mozharness merge
in production
Product: mozilla.org → Release Engineering

Comment 4

4 years ago
Can this be closed? Just looking at this since it's a dependency for one of my bugs.
(Assignee)

Comment 5

4 years ago
indeed I believe it is fixed. (sorry)
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED

Comment 6

4 years ago
yay:-) thanks Callek

Updated

4 years ago
Component: General Automation → Mozharness
You need to log in before you can comment on or make changes to this bug.