add search ability to hb data view



3 years ago
3 years ago


(Reporter: willkg, Unassigned)



(Whiteboard: u=gregg c=heartbeat p= s=)

Bug #1108604 fixed the hb data view to be better for Gregg's purposes. The one outstanding item was the ability to search specific "fields".

> 3.  Search:
> I wish a design to erect
> to allow more nuanced "select"
> but I don't want to ask
> and increase your task
> it has lots of side effects
> One nice well-intentioned note
> That I don't quite know how to emote
> don't mean to whine
> Django orm operator style is fine
> to filter to records with votes
> (i.e., rating>=0  rating==8   or (in json:  thing.subthing=="a"
> or tokenized:   field[OP][val], all in quote?  break it up `shlex` style?

I don't know how to do that offhand since mysql doesn't know anything about JSON blobs. If we were allowed to index everything in ES, then we could do this with ES, but we're not, so we can't. I don't know what other options there are.

This bug covers figuring out how to implement some kind of search and doing it.
Blocks: 1092280
I think there are three possibilities here:

1. switch to postgres
2. index all the HB data in elasticsearch
3. come up with some search syntax that lets us do some kinds of searches (strings, substrings, ...) that might work a bit here

I think option 1 is a no-go. Cheng and crew need to stick with mysql else they'd have to rewrite all the things.

Option 2 is a no-go since we avoid indexing PII. We might be ok with changing that policy, but that'd require some serious thought and analysis first.

Option 3 is possible, but I'm not sure we could come up with a syntax that's sufficiently helpful. Seems like it's better to hope we get a live replica of the db and then you can do whatever you want with the replica.

Given that, I'm going to push this off for now but I think ultimately we should WONTFIX this.
Whiteboard: u=gregg c=heartbeat p= s=input.2015q1 → u=gregg c=heartbeat p= s=
I'm unblocking the heartbeat v2 tracker because of comment #1.
No longer blocks: 1092280
This is hard and we pushed it off and we're all still alive, so I'm voting to WONTFIX this.
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.