If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

[bouncer] Daily run of sentry must have (partially) failed

RESOLVED DUPLICATE of bug 716440

Status

mozilla.org Graveyard
Server Operations
--
critical
RESOLVED DUPLICATE of bug 716440
6 years ago
3 years ago

People

(Reporter: nthomas, Assigned: justdave)

Tracking

Details

(Reporter)

Description

6 years ago
These two bugs are examples where we had bogus locations were enabled in bouncer, for products where Check Now was not set:
 https://bugzilla.mozilla.org/show_bug.cgi?id=670110
 https://bugzilla.mozilla.org/show_bug.cgi?id=668682#c6

Please check what is happening with the sentry run which runs once a day.

To flesh out the 2nd case:
Was:
Product 				OS 			Available 
Thunderbird-3.1.10-Complete		win			5000	

10 mins after turning on Check Now:
Product 				OS 			Available 
Thunderbird-3.1.10-Complete		osx			75
Thunderbird-3.1.10-Complete		linux			75
Thunderbird-3.1.10-Complete		win			75
Thunderbird-3.1.10-Complete		solaris-sparc		75
Thunderbird-3.1.10-Complete		solaris-i386		75
Thunderbird-3.1.10-Complete		opensolaris-sparc	75
Thunderbird-3.1.10-Complete		opensolaris-i386	75

So not only did we not have all platforms, we have a mirror which didn't actually have the win files.
Blocks: 666342
Assignee: server-ops → justdave
So here's the current theory on this:

A while back we added a timeout to the sentry script so that if it took over 5 or 6 minutes to check a host (cumulative, across all of the files being checked), it failed out the host assuming it was too slow.

There was an exemption to this timeout added to the non-checknow version of the script which runs overnight.  The current theory is that this exemption is apparently not working correctly.

I haven't gotten a chance to debug it any yet, adding the details in case someone else gets to it before I do.
FWIW, this will be a code issue in the sentry scripts, which are in version control, so anyone could probably poke at it.
Is this still an issue?  what's the action needed to close this out?

Adding a couple of our coders to cc in case this still needs attention.  Please point out where "in version control" they are.  thx!
(Reporter)

Comment 4

5 years ago
The current issues with sentry's daily run are in bug 716440.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 716440
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.