680166 - Using release_channel in search with ElasticSearch

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

The way we retrieve crashes by versions has changed in Socorro, so search has to follow. This change was done for Postgres in bug 679182. 

Not sure yet how to do it, some data that we have in postgres are not in ES, so we may have to hack a bit.

Comment 1

[Robert Kaiser]

Note that we'd still like to have possibilities open to search for the more raw data behind things. But you're right, the default should be to search using the new "display version" scheme, which might need some translation into actual behind-the-scenes data.
I guess you'll need to get the display-version-to-real-filters conversion data from our place in Postgres and then use those filters in the actual ES query.

Comment 2

[[DEACTIVATED] Adrian Gaudebert]

After thinking and talking to lonnen about how to solve this, we came with three solutions:

1. having the version_string added to all crashes, so I can just use a basic filter. By far the best solution for me, but is it doable? I opened bug 680608 to ask for it. 

2. having an API to call to tell me all I need to know about a version_string (from 6.0b2, tells me that it is version 6.0, channel beta, buildid xxxxx). It seems a bug is open for this, and it's just waiting to be landed. Can't find that bug though. 

3. using the postgres tables to get the data I need (instead of the API of 2.). By far the worst choice as it implies to use Postgres inside a code that is supposed to use ElasticSearch only, and goes against the initial reason of the search API (being able to use ES *or* PG).

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to Adrian Gaudebert [:adrian] from comment #2)
> 2. having an API to call to tell me all I need to know about a
> version_string (from 6.0b2, tells me that it is version 6.0, channel beta,
> buildid xxxxx). It seems a bug is open for this, and it's just waiting to be
> landed. Can't find that bug though. 

I don't have the bugs handy, but I know both releng and metrics have been looking into providing this as a service.

Maybe simplest for Socorro to provide a middleware service for this in the meantime (backed by postgres, but it'll just be an HTTP call from the ES point-of-view)? 

Could replace it with something more generic in the future.

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

Attachment: Use channels in ElasticSearch

This patch: 

* adds a new service called versions_info, sending back data about a version string (see documentation) ;
* uses that service in ES to filter the crashes by channel and build ids when needed.

Comment 5

[K Lars Lohn [:lars] [:klohn]]

Comment on attachment 555874 [details] [diff] [review]
Use channels in ElasticSearch

A more generic solution would be far more desirable, and I agree with rehelmer, the implementation can be deferred.  The time constraints being what they are, I'll avoid on a long missive and r+ this.  

Please file a separate bug for the refactoring of this, assign it to yourself, and set it for the next milestone.  This intra-dependency of APIs coupled with interdependency of implementations plays right into our refactoring conversation from IRC.  I'll expound on the topic in the new bug when I return from vacation on Monday.

Comment 6

[[DEACTIVATED] Adrian Gaudebert]

Landed on trunk with commit r3484: http://code.google.com/p/socorro/source/detail?r=3484