1202758 - [stage] Crash reports are missing style sheets

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Matt Brandt [:mbrandt]]

Attachment: qa_screenshot_of_error.png

Several of the resource files are returning HTTP 404 errors

Steps to reproduce:
0) Goto https://crash-stats.allizom.org/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-09-08&signature=shutdownhang+|+WaitForSingleObjectEx+|+WaitForSingleObject+|+PR_Wait+|+mozilla::ReentrantMonitor::Wait+|+nsThread::ProcessNextEvent+|+NS_ProcessNextEvent+|+mozilla::layers::CompositorParent::ShutDown&version=Firefox:43.0a1

Comment 1

[Peter Bengtsson [:peterbe]]

The problem was that crash-stats.allizom.org WAS load balanced on two different instances. 

$ host crash-stats.allizom.org
crash-stats.allizom.org is an alias for crash-stats.mocotoolsstaging.net.
crash-stats.mocotoolsstaging.net has address 54.213.30.86
crash-stats.mocotoolsstaging.net has address 52.11.1.154

I should have noticed, when I signed into the AWS console, I didn't search for "stage-socorroweb" but clicked on the first one I saw by manually scanning the list. I see now that one of them is terminated. 

JP, care to resolve this with some note about what went wrong and what you did to prevent it from repeating?

Comment 2

[Matt Brandt [:mbrandt]]

Bumping to QA verified .. issue is resolved. Thank you jp and peterbe for remedying this.

Comment 3

[JP Schneider [:jp]]

A deploy likely failed sometime this past week and it kept the extra node in.