We know the cause of both of these errors: * perspective_shutdown won't go away until we update the buildbot masters to a more recent version of buildbot (>= 0.8.4). This won't be happening any time soon AFAICT. * RuntimeError happens when users cancel their own jobs via self-serve. Not much we can do here. To keep the exceptions emails useful, let's whitelist these exceptions. Patch coming up.
Created attachment 559035 [details] [diff] [review] Ignore exceptions we can do nothing about (perspective_shutdown and RuntimeError) Tested successfully against some twistd.log files I downloaded from various masters.
Attachment #559035 - Flags: review?(catlee)
Comment on attachment 559035 [details] [diff] [review] Ignore exceptions we can do nothing about (perspective_shutdown and RuntimeError) https://hg.mozilla.org/build/tools/rev/d0a86616776e
Attachment #559035 - Flags: checked-in+
I've updated the tools checkout on all the masters. Will monitor exception emails this afternoon to make sure this is fixed.
Didn't get a flood of exception email at 11:00am PDT, but still got mail from preproduction-master for valid exceptions. Looks fixed.
Status: ASSIGNED → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.