1072971 - IMAP timeouts, HTTP ERROR 504 when accessing zimbra web UI

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Henry Langi [:hlangi]]

Getting reports of people getting a HTTP ERROR 504 when trying to access https://mail.mozilla.com

abr & jgruen are both on the following server: zmmbox1.mail.corp.phx1.mozilla.com

I'm on zmmbox2 and don't have any issues.

Comment 1

[Corey Shields [:cshields]]

Probably an issue contained to zmmbox1.  verified by graphs:

https://www.dropbox.com/s/wwc7qq3oglcxsmt/Screenshot%202014-09-25%2011.06.07.png?dl=0

Comment 2

[Guillaume Canavaggio [:guiom]]

david scravaglieri and john bernard are on zmmbox1.mail.corp.phx1.mozilla.com as well and experience the same 504 error

Comment 3

[David Scravaglieri [:scravag]]

Don't know if it helps but it happened after reseting my LDAP password

Comment 4

[Corey Shields [:cshields]]

Thanks David.  Shouldn't be related at all but never hurts to call it out.

Attempting to restart zimbra on zmmbox1 now.

Comment 5

[Guillaume Canavaggio [:guiom]]

David reports error 502 now with the Web UI

Comment 6

[Guillaume Canavaggio [:guiom]]

Nisha Malhan reports the same error (502) - she's on zmmbox1 too.

Comment 7

[Corey Shields [:cshields]]

Restarted and working, though I don't want to call the all clear yet.

Comment 9

[Ed Morley [:emorley]]

I can now connect via both IMAP and webmail, however I am missing some bugmails that should have been delivered during the downtime. Are these just backed up in a queue?

Comment 10

[Dave Miller [:justdave]]

Ed: more than likely.  If a mail delivery gets deferred it'll usually retry every 15 minutes to an hour afterwards, so if you don't have it within an hour I'd worry.

Based on the graphs, things appear to have initially gone sour around midnight, and went into a slow death spiral at that point.  Looking through the logs, we appear to have had a user using the Zimbra Outlook Connector about that time who appears to have triggered https://bugzilla.zimbra.com/show_bug.cgi?id=90726 (we've had tickets with Zimbra about this situation before).

Comment 11

[Dave Miller [:justdave]]

Our two previous support cases on this issue were 00130451 and 00132019

Comment 12

[Dave Miller [:justdave]]

Also pleased to report that within minutes of reporting to Zimbra that we got nailed by this again, the bug was moved to P1 priority and retargeted to the next upcoming minor release (it had been targeted to next major before).  They've definitely been getting on top of things much better since they separated from VMware.

Comment 13

[Dave Miller [:justdave]]

I guess I should have resolved this with comment 10...