635098 - Determine requirements for going full throttle on Firefox crashes

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[[:jberkus] Josh Berkus]

Based on earlier testing, we're actually not that far away from being able to cope with full processing of Firefox crashes.  We need to determine what we actually need to improve, implement, or add in order to be able to process all crashes, including:

* monitor/processor requirements
* hbase requirements
* postgresql requirements

Comment 1

[Laura Thomson :laura]

Let's review in Q2.

Comment 2

[[:jberkus] Josh Berkus]

By request:

Currently we accumulate around 12GB/week of data in PostgreSQL.  Full throttle would increase this to approximately 100GB/week.  This would have the following drawbacks:

1. Unless we dramatically increase the storage available to Master01 (as in add several terabytes), we would only be able to store 6 weeks of data.

2. Partitions of that size would mean that only around 2 weeks of data would be cached in memory.  This would mean that reporting queries would start to have response times equivalent to the old servers (slow, timeouts).

Looking at it this way, it seems like we cannot really consider full throttle without eliminating the "reports" table.

Comment 3

[Laura Thomson :laura]

I owe you dinner.   We'll revisit this in 2012.

Comment 4

[[:jberkus] Josh Berkus]

The desire to go full-throttle seems to have vanished.  Can we close this bug?

Comment 5

[Robert Kaiser]

(In reply to [:jberkus] Josh Berkus from comment #4)
> The desire to go full-throttle seems to have vanished.

IMHO, only because we know that we have some limitation that doesn't make it viable right now.

Comment 6

[[:jberkus] Josh Berkus]

Kairo,

Well, it's up to you.  However, if we have no plans to take any action on this bug, we should close it "WONTFIX".

Comment 7

[Lonnen :lonnen]

we're not going ahead with this plan. as requested...