Closed Bug 1378512 Opened 7 years ago Closed 7 years ago

Index creation is broken in elasticsearch 5.3

Categories

(Socorro :: Processor, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: miles, Assigned: adrian)

Details

Since we upgraded the processor to work with our elasticsearch 5.3 cluster, it has been failing to process crashes. In meeting, adrian said that each time the processor attempts to process a crash, it attempts to create the index in elasticsearch that that crash should be put into.

We are now in week 27 of the Socorro calendar, and the processor has thus far failed to create a new index. Will determined that [0] occurs in the part of the processor's code that creates new indices. So this bug appears to block all processing.

Will also hypothesized that the code that does index creation in the processor was maybe not updated correspondingly for 5.3.

For some more general info, see the migration status Gdoc [1].

[0] https://sentry.prod.mozaws.net/operations/socorro-stage/issues/619003/
[1] https://docs.google.com/a/mozilla.com/document/d/1dtlZW2xaKy_jHFg9fg0VX2ddpH1eTC3OoWgoRP4WdfM/edit?usp=sharing
Throwing this your way Adrian.
Flags: needinfo?(adrian)
I think I know what's going on. I'm guessing you migrated the `socorro` index from the current cluster to the new one? If so, that failure makes sense: the content of the `socorro` index has to be wildly different between the old and new clusters, because they contain data used to generate the mapping of new data indices. 

The solution to this problem is simple: delete the `socorro` index from the new cluster, then run `scripts/setup_supersearch_app.py` from any box that has the correct code and configuration. It will put the right data into the `socorro` index, and then processing should start working again.

I am going to do that now.
Flags: needinfo?(adrian)
There's a problem with the mapping that ES creates by default for the `socorro` index, that causes a conflict with some of the data we want to have in there. I am currently working on creating a better mapping for that index, that should solve our issues.
Assignee: nobody → adrian
This is resolved.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.