1503591 - research way to introduce new fields in elasticsearch

Status: REOPENED

(Socorro :: General, enhancement, P3)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Currently, if we do a deploy that adds a new field to Elasticsearch crash documents, usage of that field is "broken" until a new index is created the following week. Further, reprocessing older crash reports has no effect--the field id still busted for them.

It'd be a lot smoother if we had a way to add new fields to existing mappings. This makes it possible to rename fields and introduce new fields for which we have tons of data.

Maybe a script we could run before a deploy across all the existing indices?

This bug covers researching that.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Brian and I both found this:

https://www.elastic.co/guide/en/elasticsearch/reference/1.4/indices-put-mapping.html

I'm going to look into whether that works first.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 4681: bug 1503591: tweak create_recent_indices to have a dry run mode

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/cdead61134b4f335470c71e51f337bf39e6182a5
bug 1503591: tweak create_recent_indices to have a dry run mode

This also fixes clear_recent_indices_app to be a little less clunky in what
it's doing in regards to the args passed in. And it fixes run_update_data.sh
which was running crontabber and apps wrong.

https://github.com/mozilla-services/socorro/commit/cfcc13d2135fd85c45522209073e45d0f6307fcd
Merge pull request #4681 from willkg/1503591-show-mapping

bug 1503591: tweak create_recent_indices to have a dry run mode

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Mark as enhancement.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Bumping to P3--not going to do this any time soon.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

I made some changes to ES crash storage so it's a lot better at not saving data to indexes which have mappings that don't include those fields. So the consequences of adding new fields is much less problematic--we don't get the brokenness we used to have.

I'm going to mark this as WONTFIX.

Comment 7

[Bianca Danforth [:bdanforth]]

I'm going to reopen this bug, as it came up again all these years later!

Edit: Quoting myself from my comment in Bug 1988674:

In order to have these older crash reports show up with these new fields in SuperSearch, we would need some kind of migration script to update all past indexes' mappings with the new fields. Then after such a migration, reprocessing old crashes would work as expected.

Comment 8

[Bianca Danforth [:bdanforth]]

Got burned by this again on Bug 2007193; I added it to our shortlist of backlog tickets to fix hopefully this year sometime: https://mozilla-hub.atlassian.net/browse/CRINGE-61.

Comment 9

[Bianca Danforth [:bdanforth]]

This is why we only create a new index once per week -- it's lazily created if it doesn't already exist each time a crash document is saved to Elasticsearch based on how we name the index, which uses this name "socorro%Y%W". Since the time %W is based on is date_processed and that's in UTC, this currently occurs at midnight UTC on Mondays.