392701 - Bugzilla tinderboxen are gone or do not report recent checkins

Status: VERIFIED INVALID

(Infrastructure & Operations :: RelOps: General, task)

Description

[Frédéric Buclin]

At this exact time, http://tinderbox.mozilla.org/Bugzilla/ mentions none of the checkins we did yesterday, and boxes in http://tinderbox.mozilla.org/showbuilds.cgi?tree=Bugzilla3.0 (also 2.22 and 2.20) are all gone.

We are in the process of releasing Bugzilla for all these branches, and so we need to know if we regressed something. I reported this problem days ago on IRC, but it seems nobody tried to fix it yet.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

This looks like the usual thing for the static pages, which don't update when Tinderboxes stop reporting. The dynamically generated page
  http://tinderbox.mozilla.org/showbuilds.cgi?tree=Bugzilla
is usually more reliable.

Perhaps this is related to the landfill outage between 07:50 and 11:45 PDT on the 16th. Does your tinderbox auto-restart ?

Comment 2

[Reed Loden [:reed]]

[reed@landfill ~]$ ps auwwx | grep tinderbox
reed      3368  0.0  0.0  3668  612 pts/0    R+   11:11   0:00 grep tinderbox

[reed@landfill ~]$ uptime
 11:12:06 up 2 days, 6 min,  2 users,  load average: 0.06, 0.17, 0.21

Looks like somebody just didn't restart them. Please note that Bugzilla is not tier 1, so any bugs relating to the Bugzilla tinderboxen should not be filed as blocker/critical, as those severities page IT. Also, as the tinderboxen are run on a community-giving server, IT would probably not be the one to deal with them at all anyway.

Comment 3

[Max Kanat-Alexander]

Yes, I am the administrator of the Bugzilla tinderboxes. Any problems should be reported directly to me.

Comment 4

[Max Kanat-Alexander]

Oh, and I've restarted them now.