1498931 - [tracker] Update MDN to ElasticSearch 6.x by March 2019

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Search, enhancement, P2)

Description

[John Whitlock [:jwhitlock]]

MDN is currently using ElasticSearch 5.6.10, which is scheduled for end-of-life on March 11, 2019, when ES 7.0 is released (https://www.elastic.co/support/eol). At that point, the last 6.x release will become the new long-term support release, supported for 12 to 18 months.

The 2.x update (bug 1357891) only required updating the server, and the 1.x client libraries continued to work.

The 5.x update (bug 1464093) required updating the client libraries (1.x to 2.x to 5.x), our ElasticSearch DSL code (models for documents in search), and the server. Site search was unavailable during the deployment and upgrade, which otherwise went smoothly.

I expect the 6.x upgrade to be more like the 5.x upgrade, requiring a simultaneous update of code, libraries, DSL models, and the server, with site search unavailable for a few hours.

Comment 1

[Peter Bengtsson [:peterbe]]

https://bugzilla.mozilla.org/show_bug.cgi?id=1544482 is about making our existing Python code work with a ES 6.x version. We need two follow-up bugs on the OPs side. One about setting up a new "instance" on elastic.co and one about executing a reindex in Stage/Production once it's possible.

Comment 2

[Peter Bengtsson [:peterbe]]

Note-to-self; Need to figure out what this means:

[2019-04-17T20:43:32,216][WARN ][o.e.d.i.m.AllFieldMapper ] [Ugt1VmQ] [_all] is deprecated in 6.0+ and will be removed in 7.0. As a replacement, you can use [copy_to] on mapping fields to create your own catch all field.

Comment 3

[Ryan Johnson (:rjohnson)]

This is complete. We are using Elasticsearch 6.7.1 instances on both stage and prod, and the search index has been re-created on both stage and prod. The search index took about 11 minutes to create on prod (which has 10 celery workers).