1173466 - backfill AWS ES for the past 6 months

Status: RESOLVED FIXED

(Socorro :: Infra, task)

Description

[Robert Helmer [:rhelmer]]

We need to backfill AWS ES from S3, redacting the json_dump. Lars has created a copy_raw_and_processed, just need to test and run it.

Comment 1

[Robert Helmer [:rhelmer]]

ES backfill is complete, but comparing the document count for each index to PHX, and socorro201504 through socorro201512 look a little too low.

Not sure if the problem is we didn't queue the right messages, messages were lost (I don't think this was it), those crashes couldn't be processed, or something else.

Going to try backfilling one of those weeks and seeing if that helps.

Comment 2

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #1)
> ES backfill is complete, but comparing the document count for each index to
> PHX, and socorro201504 through socorro201512 look a little too low.
> 
> Not sure if the problem is we didn't queue the right messages, messages were
> lost (I don't think this was it), those crashes couldn't be processed, or
> something else.
> 
> Going to try backfilling one of those weeks and seeing if that helps.

OK backfilling one of these weeks helps. We're live now, going to continue this in the background.