Closed Bug 945757 Opened 12 years ago Closed 8 years ago

Improve the elasticsearch backfill / reindex app

Categories

(Socorro :: Backend, task)

Product:
Socorro
Component:
Backend
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: adrian, Unassigned)

Details

The script we are using to do backfilling / reindexing into elasticsearch could get some love and be more robust. Some ideas: * add logging about current status * improve failure resilience (it should never stop before the backfill is entirely done) * make sources more configurable. Currently the script can only be used for reindexing (taking crashes from ES to put them back into ES). It should be able to take crashes from any of our crash storages. Any other suggestion is welcome.
Great ideas! I recommend using a queue for setting up the list of what needs to be backfilled rather than a count or a list internal to the app. This eliminates the requirement that the app itself be super resilient. Instead, you can rely on the queue to keep track of whether or not you've got more work to do.
Priority: -- → P3
Assignee: adrian → nobody
Priority: P3 → --
That script doesn't exist anymore.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.