Closed Bug 1393650 Opened 7 years ago Closed 4 years ago

[es5 migration] mappings being overwritten during reindexing

Categories

(Socorro :: Infra, task, P3)

Tracking

(Not tracked)

RESOLVED INACTIVE

People

(Reporter: miles, Unassigned)

References

(Blocks 1 open bug)

Details

Even though we set index.mapper.dynamic [1] => false at the index level, this was being overridden by several instances of the mapping parameter dynamic [2] being set to true in our mappings.

I say overwritten because we showed that they start out correct:
I created a fresh index (testindex-unsoiled) and then curled it [3]

[1] https://www.elastic.co/guide/en/elasticsearch/reference/5.3/dynamic-mapping.html
[2] https://www.elastic.co/guide/en/elasticsearch/reference/5.3/dynamic.html
[3] https://pastebin.mozilla.org/9030546
I created an index 'testindex-no-dynamic' using an index creation payload with all 'dynamic' mapping parameters removed [1], and after reindexing socorro201720 into that test index, it looks as though the mappings specified at index creation were mostly (entirely?) preserved [2].

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1322630#c46
[2] https://pastebin.mozilla.org/9030559 (note: used jq to filter to just processed_crash mappings)
I'm curious about two things:

1. Is the only type change we're seeing keyword -> text? Are there any other type changes? At this point, I think we write a script to figure this out reliably or someone does it meticulously by hand. I have vague memories of Peter diffing mappings--is that a script?

2. What happens to indexes that Socorro processor creates? Are those ok in regards to the fields we're concerned about?
Also, the reason this came up was that we were seeing fields we marked as keyword get turned into something else which then caused the error in bug #1393197 (Fielddata is disabled on text fields by default) when doing super search.
Making this a P2 to look into.
Priority: -- → P2

Bumping down to P3 until after we look into bug #1568601.

Priority: P2 → P3
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.