1375245 - MozDef data producers producing inconsistently typed data in details.future and details.current

Status: RESOLVED FIXED

(Enterprise Information Security Graveyard :: MozDef, task)

Description

[Gene Wood [:gene]]

It appears that there is/are data producer(s) which are putting events into mozdef with a field and differently typed data for that field.

The two fields that show this in the events-weekly index, implying that within the last week, conflicting typed data like this has been put into mozdef, are :

details.current
details.future

I'd recommend
* Determining the producers causing this and if it is multiple producers colliding with the same field name, have it changed, or if one producer producing differently typed data, have it fixed
* Setup a monitor to detect when producers publish inconsistently typed data into mozdef

You can see these fields by going to kibana

Settings... Indices...events-weekly
Sort by Indexed
Look for fields of type "conflict"

Comment 1

[Alicia Smith [:phrozyn] (use NEEDINFO)]

Do you have a link to an example event?

We encountered this before, but the field is not valuable to us, so I didn't fix it.
It would require reindexing of data to do so.

Comment 2

[Gene Wood [:gene]]

> Do you have a link to an example event?

No, I just encountered the report of this condition in kibana. Steps to reproduce are in Comment 0

Comment 3

[Alicia Smith [:phrozyn] (use NEEDINFO)]

yeah, I  understand.

https://bugzilla.mozilla.org/show_bug.cgi?id=1333906

Is the original bug regarding this

Comment 4

[Gene Wood [:gene]]

Out of curiosity, what was the resolution of this?