781177 - [ia] Use ES "more like this" for related documents

Status: RESOLVED FIXED

(support.mozilla.org :: Knowledge Base Software, task, P2)

Description

[Ricky Rosario [:rrosario, :r1cky]]

ElasticSearch has a nice feature for finding similar documents:
http://www.elasticsearch.org/guide/reference/api/more-like-this.html

We should use this and delete our cron job with the huge SQL statement that uses tags. We could convert this to use topics and products but it will be much easier to just use elastic search.

Elasticutils doesn't support this yet but I filed an issue:
https://github.com/mozilla/elasticutils/issues/53

Comment 1

[Ricky Rosario [:rrosario, :r1cky]]

Putting this into Q3 milestone for now. We'll add to a sprint once elasticutils gets this feature landed.

Comment 2

[Ricky Rosario [:rrosario, :r1cky]]

pyes.ES implements morelikethis(). We may be able to just call to that directly.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

My vote is add a method to search.models.SearchMixin for morelikethis and use the same args as the pyes ES morelikethis. It would call get_es() to get an ES and call morelikethis on that.

Then in wiki.models.Document, create a "related_documents" or a "morelikethis" that calls SearchMixin.morelikethis with the specifics (e.g. the fields, the instance id, ...).

In elasticutils, if/when we ditch pyes, we'll continue to have an ES and we'll make sure it has a morelikethis going forward.

Comment 4

[Ricky Rosario [:rrosario, :r1cky]]

I think this will be 3pts. I know I will run into random issues.

Comment 5

[Ricky Rosario [:rrosario, :r1cky]]

This landed on master:
https://github.com/mozilla/kitsune/commit/39841f51379da3aaa2d511f5b065dc7b98ae1d41

I'll deploy to prod on Monday.

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

This landed this morning in prod.