733193 - Create and store ElasticSearch's schema in Socorro

Status: RESOLVED FIXED

(Socorro :: Database, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

Attachment: crash_reports mapping

Before we start using ElasticSearch in production, we want to make sure the schema we will use for crash reports is correct and robust. Updating it will most likely be a pain, so we want to avoid it as much as possible. 

Attached is the most recent schema I have.

Comment 1

[[:jberkus] Josh Berkus]

This schema seems to be missing a LOT of fields.  Am I not understanding it correctly?

Comment 2

[[DEACTIVATED] Adrian Gaudebert]

(In reply to [:jberkus] Josh Berkus from comment #1)
> This schema seems to be missing a LOT of fields.  Am I not understanding it
> correctly?

ElasticSearch automatically updates the schema when we index a document containing a new field. I don't think we want to have all fields here, but only specific fields where we want a special mapping (or a mapping different from what ES would guess from the value of that field).

Comment 3

[[DEACTIVATED] Adrian Gaudebert]

Attachment: ES schema with timezone in dates

As we updated Socorro to support timezones, and moved to ISO 8601 datetimes, we need to update ES schema. The one attached has a date_format of "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ". I also removed all the duplicate date_formats to keep only the default one.

Comment 4

[Laura Thomson :laura]

Ping?

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

Schema is here: https://github.com/mozilla/SocorroSearchService/blob/master/conf/socorro_mapping.json

And accessible in Socorro via a submodule, see https://github.com/mozilla/socorro/blob/master/.gitmodules