1122013 - [stage][regression] Signature reports do not load

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Matt Brandt [:mbrandt]]

Attachment: qa_bug_stage.png

Signature reports appear to not be loading on stage.

Steps to reproduce:
1. goto https://crash-stats.allizom.org/report/list?product=Firefox&range_value=7&range_unit=days&date=2015-01-15&signature=OOM+|+small&version=Firefox%3A34.0#tab-reports
2. select any crash

Expected:
Report loads

Actual:
"Crash Not Found" - https://crash-stats.allizom.org/report/index/93c7e92a-2068-4c20-9922-765032150108

Comment 1

[Selena Deckelmann :selenamarie :selena]

The crashes on the report page appeared to be from 1/8, a week ago.

I was able to load a report from the last 3 days: https://crash-stats.allizom.org/report/index/716a2478-0753-4d6e-bf73-956d62150112

Maybe :rhelmer can shed some light on this?

Comment 2

[Robert Helmer [:rhelmer]]

(In reply to Selena Deckelmann :selenamarie :selena from comment #1)
> The crashes on the report page appeared to be from 1/8, a week ago.
> 
> I was able to load a report from the last 3 days:
> https://crash-stats.allizom.org/report/index/716a2478-0753-4d6e-bf73-
> 956d62150112
> 
> Maybe :rhelmer can shed some light on this?

The production Postgres DB is copied to stage on a weekly basis, but the prod and stage S3 buckets are not synched.

I'm not sure it's worth the cost and time to sync the prod/stage S3 buckets. Do you think we should keep doing the prod->stage sync thing, or just have them be independent?

Comment 3

[Matt Brandt [:mbrandt]]

I'm mainly interested in ensuring the test coverage is good -- is there a good middle ground for keeping this data sync'd? Perhaps updating the test to only look at older reports?

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Matt Brandt [:mbrandt] from comment #3)
> I'm mainly interested in ensuring the test coverage is good -- is there a
> good middle ground for keeping this data sync'd? Perhaps updating the test
> to only look at older reports?

Disabling the postgres sync and just depending on submitted crashes would do this, but it's a lot less convenient.

Let me see what the cost to do a full sync would be.

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #4)
> (In reply to Matt Brandt [:mbrandt] from comment #3)
> > I'm mainly interested in ensuring the test coverage is good -- is there a
> > good middle ground for keeping this data sync'd? Perhaps updating the test
> > to only look at older reports?
> 
> Disabling the postgres sync and just depending on submitted crashes would do
> this, but it's a lot less convenient.
> 
> Let me see what the cost to do a full sync would be.

The current problem is that we're syncing prod PG -> stage PG, but not syncing S3 also (or ES for that matter, which will come up as we test supersearch more deeply)

Per the meeting we just had (https://etherpad.mozilla.org/2015-q1-socorro-coverage-needs) let's leave PHX as-is, and solve this as we stand up AWS staging this quarter:

* rebuild stage on AWS without and backend synching
** e.g. S3, PG, ES should all be standalone
** submitting crashes from prod to stage collector is OK

Comment 6

[Justin Potts [:justinpotts]]

Fixed; verified through automation in test_top_crasher_reports_tab_has_uuid_report.