894649 - Install Polish analyzer on SUMO elasticsearch boxes

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Community Platform, task, P4)

Description

[Michael Cooper [:mythmon]]

+++ This bug was initially created as a clone of Bug #888015 +++

SUMO is going to turn on l10n aware searches soon, and having support for Polish would be swell. AMO did this a week or two ago in Bug #888015, and we want the same thing.

The plugin is here:
https://github.com/elasticsearch/elasticsearch-analysis-stempel

AMO used version 1.2 to match up with ES version 0.20.x. Assuming we are still on 0.20.x, this is the right version to install:

$ plugin -install elasticsearch/elasticsearch-analysis-stempel/1.2.0

Comment 1

[Daniel Maher [:phrawzty]]

Hello,

The package has been deployed to the ES servers which are used by SUMO (dev, stage, and prod).  I am unclear as to whether the cluster needs to be restarted for this plugin to become available.  Could you please test it and let me know if the plugin is available - if not, I'll schedule a restart.

Comment 2

[Michael Cooper [:mythmon]]

Thanks Daniel, it looks like it didn't take effect (at least on dev), so it lookms a restart is needed.

$ curl 'elasticsearch-zlb.dev.vlan81.phx.mozilla.com:9200/_analyze?analyzer=polish&pretty=1' -d 'Pretend this is in Polish'
{
  "error" ElasticSearchIllegalArgumentException[failed to find analyzer [polish]]",
  "status": 400
}

Comment 3

[Daniel Maher [:phrawzty]]

I've restarted the nodes in the dev cluster, and the module is now active, as noted in the logs (for example) :
[2013-07-23 14:06:19,445][INFO ][plugins                  ] [elasticsearch2_dev_phx1] loaded [analysis-stempel], sites [bigdesk, head, paramedic]

That said, the curl you noted in comment 2 still returns the 400.  It may be worth noting that the same request performed against the AMO prod cluster returns a 400 as well (according to :jason).

Is that test string correct ?

Comment 4

[Michael Cooper [:mythmon]]

Apologies, I didn't actually test the command locally before sending it here. Apparently the Polish plugin doesn't work the way I expected. This command should work (based on local testing)

$ curl 'elasticsearch-zlb.dev.vlan81.phx.mozilla.com:9200/sumo_sumo-20130701/_analyze?analyzer=polish&pretty=1' -d 'Pretend this is in Polish'
{
  "tokens" : [ {
    "token" : "pretend",
    "start_offset" : 0,
    "end_offset" : 7,
    "type" : "<ALPHANUM>",
    "position" : 1
  }, {
    "token" : "this",
    "start_offset" : 8,
    "end_offset" : 12,
    "type" : "<ALPHANUM>",
    "position" : 2
  }, {
    "token" : "is",
    "start_offset" : 13,
    "end_offset" : 15,
    "type" : "<ALPHANUM>",
    "position" : 3
  }, {
    "token" : "polishić",
    "start_offset" : 16,
    "end_offset" : 22,
    "type" : "<ALPHANUM>",
    "position" : 4
  } ]
}

In particular, and index must be specified, which confuses me, but whatever. Does this work for you?

Comment 5

[Daniel Maher [:phrawzty]]

Dev looks good.  I'll roll through the necessary restarts on prod shortly.

Comment 6

[Daniel Maher [:phrawzty]]

Prod is now complete.