Determine root cause why a large number of jobs were triggered for win10 and win7 when enabling win10 for talos in bug 1366029. 10:38 AM <aselagea|buildduty> yeah. 10:39 AM <arr> CRITICAL Pending Jobs: 9290 on [t-w1064-ix] 10:39 AM <jmaher> we have 9000+ pending w10 jobs, odd 10:39 AM <arr> seems like that ramped up very fast 10:39 AM or we had a lot of old jobs in the database or... something 10:39 AM <jmaher> that seems suspicious 10:40 AM <aselagea|buildduty> no, all of them are waiting for <30 minutes in the queue 10:41 AM <arr> who just triggered 10K tests? 10:42 AM <jmaher> I suspect catlee's theory 10:43 AM <arr> jmaher: that the puppet patch didn't land? 10:44 AM <jmaher> arr: yes, it did yesterday 10:44 AM <arr> jmaher: which theory? 10:44 AM <jmaher> arr: old pending jobs are scheduled side note: this may be similar to the issue in bug 1223042
Created attachment 8873159 [details] Screen Shot 2017-05-31 at 3.31.35 PM.png high pending counts from this morning
Summary: determine root cause of sudden surge on high pending counts due while enabling w10 talos → determine root cause of sudden surge in pending counts after enabling w10 talos
Alin, did we find out any information on what the root cause might be here for the post-mortem?
Sadly, no. Something similar happened in bug https://bugzilla.mozilla.org/show_bug.cgi?id=1223042, but we couldn't find an answer in that case either.
I think too much time has passed to effectively determine the root cause.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.