696722 - Create a script to import crashes into ElasticSearch

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

To be able to test Socorro with ElasticSearch we need data in ES. We will consequently need a way to easily fill an ES instance with crashes. 

The goal here is to create a script that pulls data for a day from crash-analysis.m.c and then pulls individual crashes from crash-stats directly, to then index them in an ES instance.

Comment 1

[[DEACTIVATED] Adrian Gaudebert]

Here is my current state of work: [https://github.com/AdrianGaudebert/socorro/compare/master...696722-script-import-es]

CCing rhelmer for review. Also a few questions: should I put that script there (scripts/)? What do you think about the way I generate the dump file (a simple tar with a bunch of json files)?

Comment 2

[[DEACTIVATED] Adrian Gaudebert]

Suggestions from rhelmer: add a parameter to pass a URL and load crashes from that URL instead of prod; add a parameter to pass a local csv file.

Comment 3

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/598991241eeebc55e01856f4d9bc6fa8761ca1b9
Merge pull request #449 from AdrianGaudebert/696722-script-import-es

Fixes bug 696722 - Added a script to export crashes from PostgreSQL and ...