The MPT outage over the weekend appears to have left lots of builds in a running state in the db when in fact they have finished. Presumably the DB was not responding at that point and the transaction timed out or was dropped. Upstream bug for that ? For example, http://production-master03.build.mozilla.org:8010/builders/OS%20X%2010.6.2%20mozilla-central%20build/builds/2359 is Start Fri Jan 14 23:00:31 2011 End Fri Jan 14 23:09:20 2011 Elapsed 8 mins, 48 secs with a retry final state due to the network connection dropping. In the db, there are two rows in the builds table (?!), both with builds.finish still NULL, and in buildrequests complete=0, results=NULL, complete_at=NULL. [SELECT * from builds,buildrequests WHERE builds.brid=buildrequests.id AND builds.number=2359 AND buildrequests.buildername="OS X 10.6.2 mozilla-central build" ORDER BY builds.start_time LIMIT 10;] The fix we need here is to clean up the db so that the list of running list is accurate.
Just one remaining zombie: Branch Revision Builder name Submitted at Running since Running for Master try 3e8c4b0f6e68 Rev3 Fedora 12 tryserver debug test mochitest-other 2011-01-14 14:18:28 2011-01-14 19:25:52 2 days, 5:06:49 buildbot-master2:8011 which I bet would go away if the master gets a restart like the others have had.
Looks ok now.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.