888502 - Add locale fallback for languages unsupported by Elasticsearch

Status: RESOLVED FIXED

(Marketplace Graveyard :: Search, defect, P2)

Description

[Rob Hudson [:robhudson]]

We're using the Elasticsearch list of language analyzers to decide which localized strings to store in ES. But some of the languages the Marketplace supports aren't supported by ES. We should at least store these and retrieve them so results have their appropriate localized strings.

Comment 1

[Rob Hudson [:robhudson]]

This may be becoming more relevant as more locales are being added lately.

E.g.:
https://github.com/mozilla/zamboni/commit/8ca054d3
https://github.com/mozilla/zamboni/commit/0d86b659

Without fixing this, what this means is apps that do support Greek, e.g., and a Greek user searches for the app, the app will still be returned using English because that's all we have in Elasticsearch. (See https://github.com/mozilla/zamboni/blob/master/mkt/webapps/utils.py#L139)

Search will also not perform as well for these languages.

It may be possible to group these into related languages and add plugins that work well for those groups.

We can also separate the storage/retrieval vs the search (stemming/stop words/etc.). Storage/retrieval should absolutely work and we shouldn't return empty strings. Search may work well enough using our defaults.

Comment 2

[Rob Hudson [:robhudson]]

This was done in commit:
https://github.com/mozilla/zamboni/commit/232e410d