[es5 migration] mappings being overwritten during reindexing

NEW
Unassigned

Status

Socorro
Infra
2 months ago
2 months ago

People

(Reporter: miles, Unassigned)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 months ago
Even though we set index.mapper.dynamic [1] => false at the index level, this was being overridden by several instances of the mapping parameter dynamic [2] being set to true in our mappings.

I say overwritten because we showed that they start out correct:
I created a fresh index (testindex-unsoiled) and then curled it [3]

[1] https://www.elastic.co/guide/en/elasticsearch/reference/5.3/dynamic-mapping.html
[2] https://www.elastic.co/guide/en/elasticsearch/reference/5.3/dynamic.html
[3] https://pastebin.mozilla.org/9030546
(Reporter)

Comment 1

2 months ago
I created an index 'testindex-no-dynamic' using an index creation payload with all 'dynamic' mapping parameters removed [1], and after reindexing socorro201720 into that test index, it looks as though the mappings specified at index creation were mostly (entirely?) preserved [2].

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1322630#c46
[2] https://pastebin.mozilla.org/9030559 (note: used jq to filter to just processed_crash mappings)
I'm curious about two things:

1. Is the only type change we're seeing keyword -> text? Are there any other type changes? At this point, I think we write a script to figure this out reliably or someone does it meticulously by hand. I have vague memories of Peter diffing mappings--is that a script?

2. What happens to indexes that Socorro processor creates? Are those ok in regards to the fields we're concerned about?
Also, the reason this came up was that we were seeing fields we marked as keyword get turned into something else which then caused the error in bug #1393197 (Fielddata is disabled on text fields by default) when doing super search.
You need to log in before you can comment on or make changes to this bug.