Closed Bug 122504 Opened 19 years ago Closed 19 years ago

tinderbox.mozilla.org not receiving/processing mail from build machines properly

Categories

(mozilla.org Graveyard :: Server Operations, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: darin.moz, Assigned: rkotalampi)

Details

tinderbox.mozilla.org not receiving/processing mail from build machines properly.

we're getting a lot of yellow boxes on tinderbox, which jj believes is a mail
problem.  that is, the tinderbox build machines are successfully completing
builds but the tinderbox pages are not reflecting the results... they are just
showing yellow boxes :-(
Looks like lounge is in heavy load and just queueing emails. Currently there's 
825 emails in queue - most going to tinderbox-daemon. Reason for loads: bunch 
of showbuild.pl processes.

maybe there was a change to the sidebar code that's causing people to constantly
hit the tinderbox page; is there a way to see what arguments are being passed to
showbuilds?
Now when the mail queue is seattled down load is gone... and I compared 
accesses to showbuilds.cgi and they looked normal (nothing alarming compared to 
yesterday).

Something in those 800+ (or whatever it was at peak) must have loaded the 
system badly. Unfortunately we don't have mail logs available to look further 
into this.

Assignee: daruszka → rko
Marking resolved.
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → FIXED
It seems like we are still getting these bursts of tinderbox emails from 
iPlanet/Sun from svbld to NSS tree.
bursts? how often? i thought we were only getting one build every
40 minutes or so.
I don't know how often but after this afternoon I've seen one more burst from 
iPlanet systems.
There was an issue with lounge's syslog (name resolution never hit "files") that
prevented mail logs to be saved. I have fixed it so next time we should be able
to make better analyzis of what happened and track the spammer down.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.