Closed
Bug 1022438
Opened 10 years ago
Closed 10 years ago
https://tbpl.mozilla.org/?tree=Try&rev=8fa5dc94a2d2 has more than 1500 pending jobs
Categories
(Release Engineering :: General, defect)
Tracking
(Not tracked)
RESOLVED
WORKSFORME
People
(Reporter: rail, Unassigned)
Details
Not sure what's going on there but this push generated more than 1500 pending b2g_emulator_vm try opt test marionette-webapi jobs
Comment 1•10 years ago
|
||
Funny story, I tried to trigger 999 jobs via the box on self-serve (taking advantage of a slow weekend and needing to collect some data), but somehow that got doubled to 1998. Not sure how it happened, FWIW. I definitely didn't double-submit.
Reporter | ||
Comment 2•10 years ago
|
||
So, it was expected, phew.
I still wonder whether we can do something more effective to figure out the root clause of the failures...
Severity: blocker → normal
Priority: P1 → --
Comment 3•10 years ago
|
||
It's the first time I've ever seen requested number of jobs doubled like that. Suppose it couldn't have been on a worse one either :P
This also exposes a need for a better way to selectively mass-cancel jobs in self-serve :)
Reporter | ||
Comment 4•10 years ago
|
||
FTR, we still have 916 jobs pending...
Reporter | ||
Comment 5•10 years ago
|
||
BTW, this also affects our usage of on-demand instances. We start all of them since we hit spot instance capacity (our own limit set in http://hg.mozilla.org/build/cloud-tools/file/acac9151bbb8/configs/watch_pending.cfg#l70).
Reporter | ||
Comment 6•10 years ago
|
||
I canceled all running jobs in self-serve since the push already completed way more than 999.
Comment 7•10 years ago
|
||
As I mentioned on IRC, it would have been nice if the running jobs had been left to complete instead of showing up as canceled and leaving me to have to sort through a mess of canceled jobs skewing the numbers I was trying to get.
Updated•10 years ago
|
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
Assignee | ||
Updated•7 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•