mail.mozilla.org is down.

RESOLVED FIXED

Status

Infrastructure & Operations
Infrastructure: Other
--
blocker
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: mreavy, Assigned: justdave)

Tracking

Details

I'm the administrator for the media-private mailing list, and I can't access the mailing list to approve posts, etc.

I was reviewing the admin pages of media-private and then after clicking on a link I got the following error message:

"Bug in Mailman version 2.1.14
We're sorry, we hit a bug!
Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs."

Updated

6 years ago
Assignee: server-ops → server-ops-infra
Component: Server Operations → Server Operations: Infrastructure
QA Contact: phong → jdow
The network storage shelf storing the backend filesystem for the mailman server got taken offline for the datacenter move.  It's going to be down until further notice.  We're working on figuring out what to do about it.
(Reporter)

Comment 2

6 years ago
(In reply to Dave Miller [:justdave] from comment #1)
> The network storage shelf storing the backend filesystem for the mailman
> server got taken offline for the datacenter move.  It's going to be down
> until further notice.  We're working on figuring out what to do about it.

What does this mean for my list?  Can members still send email to media-private (and will the emails get archived), or does this just affect the administration of the list?  Thanks.
The mail server is effectively dead for the time being.  Inbound mail will be queued, and will get delivered after we get the server back online.  It's looking like we might have it back online within the next hour or so.
Severity: critical → normal
(In reply to Dave Miller [:justdave] from comment #3)
> The mail server is effectively dead for the time being.  Inbound mail will
> be queued, and will get delivered after we get the server back online.  It's
> looking like we might have it back online within the next hour or so.

That turned out to be a far-fetched idea at the time because too many things broke at once :|

*now* I expect it within an hour.
Severity: normal → critical
Summary: Can't access media-private admin pages (I'm the admin) → mail.mozilla.org is down.
And this is now a blocker because it's affecting mozilla.org email.
Assignee: server-ops-infra → justdave
Severity: critical → blocker
Group: infra
Been ramping up inbound mail to it via relays slowly just to make sure everything was working before unleashing the floodgates.  The queues have all been flushed and caught up now, and it looks like everything's working.  I just pushed the last DNS change to get the web interface to mailman working again, should propagate in 10 or 15 minutes.

Any additional issues that seem related either reopen or flag me down on IRC.  If something seems outright broken, reset assignee to default so it'll page someone.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Component: Server Operations: Infrastructure → Infrastructure: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.