1237829 - AhrefsBot/5.0 causing wikimo/generic cluster slowdown

Status: RESOLVED WORKSFORME

(Infrastructure & Operations :: MOC: Problems, task)

Description

[Salvador Espinoza [:sal]]

reported in #moc 

17:59 <stephend> wiki.mozilla.org is having perf issues; known?
18:00 <stephend> looks like a bot is hammering it? https://rpm.newrelic.com/accounts/263620/applications/2207193/traced_errors/29da31-2abe3d10-b4e2-11e5-8436-c81f66b8ceca

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

5:59 PM <stephend> heya sal
5:59 PM <stephend> wiki.mozilla.org is having perf issues; known?
6:00 PM <@nagios-scl3> New Sysadmin OnDuty is jlaz
6:00 PM @nagios-scl3 set the topic: on duty sysadmin: jlaz |Happy 2016 everybody
6:00 PM <@sal> stephend: hey, nothing know
6:00 PM <@ashish> main page loads fine
6:00 PM <@ashish> and quick
6:01 PM <stephend> looks like a bot is hammering it? https://rpm.newrelic.com/accounts/263620/applications/2207193/traced_errors/29da31-2abe3d10-b4e2-11e5-8436-c81f66b8ceca
6:01 PM <stephend> I wouldn't have noticed, except our mozilla.org tests fail to connect to it within 10 seconds (since it's in the footer)
6:02 PM <stephend> 17:58:31 E   ConnectionError: HTTPSConnectionPool(host='wiki.mozilla.org', port=443): Max retries exceeded with url: /Webdev/GetInvolved/mozilla.org (Caused by ReadTimeoutError("HTTPSConnectionPool(host='wiki.mozilla.org', port=443): Read timed out. (read timeout=10)",))
6:03 PM <stephend> but yeah, seems snappy to me now, too
6:03 PM — stephend re-runs tests
6:04 PM <stephend> yep, better: http://imgur.com/TaADEAP

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

To be clear(er) - this was from last night: Wednesday, January 6th.  All times are PM and PDT.

Comment 3

[Ryan C [:ryanc] (UTC-4)]

Anything left with this?

Comment 4

[Ryan C [:ryanc] (UTC-4)]

Reopen again if necessary.