1133055 - Elasticsearch 1.3.8 update

Status: RESOLVED FIXED

(Infrastructure & Operations :: IT-Managed Tools, task)

Description

[Rob Hudson [:robhudson]]

MDN is using an older 1.3.x version of Elasticsearch. A new update came out that we should upgrade to (or configure groovy scripting as recommended).

https://www.elasticsearch.org/blog/elasticsearch-1-4-3-and-1-3-8-released/

Comment 1

[C. Liang [:cyliang]]

jezdez: Do you have a preference for upgrading or for turning off groovy?  

The only piece of input that I have is that, the last time I tested it, the knapsack plugin was very version dependent and is likely to break if there is an upgrade.  On the other hand, I haven't set up any index backups using it, so it wouldn't be a terrible loss. =)

Comment 2

[C. Liang [:cyliang]]

9:12:43 AM cyliang1: Oh, I was going to ask — did you want me to try disabling groovy scripting support on Elasticsearch or did you want me to try  upgrading it?
9:12:50 AM jezdez: oh, hmm
9:12:56 AM jezdez: yeah, sorry for not getting back to you about it
9:13:00 AM jezdez: in short, we don’t use groovy
9:13:04 AM jezdez: so get rid of it

Comment 3

[C. Liang [:cyliang]]

Groovy scripting has been disabled on the stage MDN ES cluster.  If you have an opportunity to run any specific test against this cluster, that would be great.  Otherwise, I'll plan on updating the production cluster early next week.


I tested scripting by the following request:

curl -XGET "http://localhost:9200/mdnprod-main_index/_search?q=*&pretty" -d' { "query": { "function_score": { "script_score": { "script": "Math.pow(doc[\"id\"].value, 2)", "lang": "groovy" } } } }'

On the MDN stage nodes, this returns an error. [1]  In the production cluster, it does not. [2]

[1] {
  "error" : "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[5U6iuYhfSCypt2_8c7rk2A][mdnstage-main_index][0]: RemoteTransportException[[developer-elasticsearch2.stage.scl3][inet[/10.22.81.172:9300]][search/phase/query]]; nested:
[...]
ParsingException[[mdnstage-main_index] script_score the script could not be loaded]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }]",
  "status" : 400
}

[2]  {
  "took" : 1550,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 29639,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "mdnprod-main_index",
      "_type" : "wiki_document",
      "_id" : "13215",
      "_score" : 1.0,
[...]
}

Comment 4

[C. Liang [:cyliang]]

Groovy scripting has been disabled on the prod MDN ES cluster.  Flag me down if this turns out to have unfortunate consequences.