625947 - [Backend] Add faceted search to flightdeck

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Add-on Builder, defect, P1)

Description

[Wil Clouser [:clouserw]]

The second half of bug 625942 is to add a bunch of juicy stuff to the search.  Bullet points include:

>> # of forks or third-party dependents on an add-on or library (who's reusing it)
>> all add-ons or libraries that share a given dependency
>> size of the add-on or library
>> AMO review status

If any of these seem unreasonable technically, let's chat.

Comment 1

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

> >> # of forks or third-party dependents on an add-on or library (who's reusing it)
> >> all add-ons or libraries that share a given dependency
> >> size of the add-on or library
> >> AMO review status

If these are accessible from the DB, it's easy.  If it's not its slightly more difficult - from talking to DBUC there's files that need to be searched on disk, so for that we'd need to build a spider, and use a search engine that had insert support (Sphinx 1.1 or something else).

-d

Comment 2

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

FYI, I'm looking into elastic search as webdev decided (along with metrics) that this might be the best tool for the job.

Comment 3

[Wil Clouser [:clouserw]]

What's the status of this?

Comment 4

[Daniel Buchner [:dbuc]]

Will release in 2 weeks during the 0.9.4 push.

Comment 5

[Daniel Buchner [:dbuc]]

The rest of the facets that we need added to the Builder's search system are as follows:

filter: author/owner
filter: dependencies
facet: number of forks
facet: most active (recency of last update indexed against number of total commits, accounting for add-on/library age)
facet: size of the add-on/library
facet: number of dependencies

Comment 6

[Wil Clouser [:clouserw]]

What's the news here, dave?

Comment 7

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

(In reply to comment #6)
> What's the news here, dave?

This might be a while - I've got to work on uplifting faceting code into it's own package - that's my task for this week.  I'll be on leave until June 1 or 2 at which point I will probably begin implementing this.

Comment 8

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

This doesn't need me anymore, the code in:

https://github.com/davedash/flightdeck/commits/656394-no-pagination-if

Makes facets easy to deal with if you look at the elasticutils docs:

http://elasticutils.rtfd.org/en/docs

Comment 9

[Sean McArthur [:seanmonstar]]

I read this and wonder at the difference of filters and facets, in your mind. I'm using "facets", to list the "filters" on the left side.

I see it as we can have filters, and sort-bys. Sory by Keyword Match, Most Active, Most Copies, etc.