Closed Bug 1079345 Opened 11 years ago Closed 11 years ago

Datazilla is down.

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: Usul, Unassigned)

Details

PROBLEM(CRITICAL): datazilla-zlb.vips.scl3.mozilla.com/https - datazilla.mozilla.org Info: CRITICAL - Socket timeout after 10 seconds Date: 10-07-2014 08:52:31 Runbook doesn't really exists.
[6:18pm] • Usul goes a files a bug [6:19pm] cyliang: Usul: And now a whole bunch of errors that look like someone trying to do an attack (trying to grab apache logs,etc.) cyliang: Warning: Truncated incorrect DOUBLE value: '../../../../../../../../../../etc/httpd/logs/error.log' [6:21pm] cyliang: Warning: Truncated incorrect DOUBLE value: '/etc/passwd' [6:21pm] cyliang: Warning: Truncated incorrect DOUBLE value: '1386962191/../../../../../../../../../../boot.ini'
Assignee: server-ops-webops → nobody
Component: WebOps: Other → Security Assurance: Operations
Product: Infrastructure & Operations → mozilla.org
QA Contact: nmaul
64.213.68.131 and 78.97.7.6 blacklisted based on Bro logs. Both IPs has been detected as triggering lots of HTTP errors.
Datazilla went back up per Usul
Also banned 1412699011.980081 - - - - - - - - - MozillaHTTPErrors::Excessive_HTTP_Errors_Attacker Excessive HTTP errors for requests from 89.78.216.176 628 in 1.0 hr, eps: 0 89.78.216.176 - nsm3-eth4-5 Notice::ACTION_LOG 86400.000000 F - - - - - 1412699011.980081 - - - - - - - - - MozillaHTTPErrors::Excessive_HTTP_Errors_Attacker Excessive HTTP errors for requests from 91.74.229.16 570 in 1.0 hr, eps: 0 91.74.229.16 - nsm3-eth4-5 Notice::ACTION_LOG 86400.000000 F - - - - -
Probably an automated scan reaching it's max? the urls above look like common attack patterns in most every scanner (nikto, w3af, etc) Some current settings/logs on the server look like they may not withstand it: datazilla1.webapp.scl3.mozilla.com /etc/httpd/mozilla/generic.conf MaxClients 130 error log: [Tue Oct 07 09:09:10 2014] [error] server reached MaxClients setting, consider raising the MaxClients setting
Assignee: nobody → server-ops
Component: Security Assurance: Operations → Server Operations
QA Contact: shyam
datazilla is backup, banns are in place. Closing for now.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.