1385892 - Advanced Top Contributors dashboard returns 500

Status: NEW

(support.mozilla.org :: General, enhancement, P2)

Description

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Thanks to the legendary mythmon we had a new but hidden advanced Community Hub Top Contributors dashboard to track contributions thru the whole community. It had date & locale filtering so we could find the top contributors in various locales.

It has to be visited directly since it is not linked in the site:
https://support.mozilla.org/en-US/community/top-contributors/questions/new

It appears some recent change in Kitsune before or around July 24th broke this tool. I believe it used React. So maybe something related to that was removed? Or maybe a repeat of Bug 1166021?

This tool is also used by our admins Rachel, Madalina & Michal to check up on contributors across all locales in more detail. So getting it back would be much appreciated. Thanks!

Comment 1

[Safwan Rahman (:safwan)]

Giorgos, Can you please provide the tracelog from Sentry?

Comment 2

[Giorgos Logiotatidis [:giorgos]]

RequestError: TransportError(400, {u'failed_shards': [{u'node': u'YvCjsoEdTFeJboqUvEFTkw', u'index': u'sumo_sumo-metrics', u'reason': {u'reason': u'failed to parse search source. unknown search element [facets]', u'type': u'search_parse_exception', u'line': 1, u'col': 221}, u'shard': 0}], u'root_cause': [{u'reason': u'failed to parse search source. unknown search element [facets]', u'type': u'search_parse_exception', u'line': 1, u'col': 221}], u'grouped': True, u'reason': u'all shards failed', u'phase': u'query', u'type': u'search_phase_execution_exception'})
  File "django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "newrelic/hooks/framework_django.py", line 499, in wrapper
    return wrapped(*args, **kwargs)
  File "kitsune/community/views.py", line 204, in top_contributors_new
    contributors = api_endpoint().get_data(request)
  File "kitsune/community/api.py", line 215, in get_data
    creator_answer_counts = answer_query.facet_counts()['creator_id']['terms']
  File "elasticutils/__init__.py", line 1631, in facet_counts
    return _facet_counts(self._raw_facets().items())
  File "elasticutils/__init__.py", line 1619, in _raw_facets
    return self._do_search().response.get('facets', {})
  File "elasticutils/__init__.py", line 1432, in _do_search
    response = self.raw()
  File "elasticutils/__init__.py", line 1511, in raw
    **extra_search_kwargs)
  File "newrelic/hooks/datastore_elasticsearch.py", line 70, in _nr_wrapper_Elasticsearch_method_
    return wrapped(*args, **kwargs)
  File "elasticsearch/client/utils.py", line 68, in _wrapped
    return func(*args, params=params, **kwargs)
  File "elasticsearch/client/__init__.py", line 440, in search
    params=params, body=body)
  File "elasticsearch/transport.py", line 284, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "elasticsearch/connection/http_urllib3.py", line 55, in perform_request
    self._raise_error(response.status, raw_data)
  File "elasticsearch/connection/base.py", line 97, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)

Comment 3

[Giorgos Logiotatidis [:giorgos]]

https://sentry.prod.mozaws.net/operations/sumo/issues/618207/
https://sentry.prod.mozaws.net/operations/sumo/issues/628595/

Comment 4

[Giorgos Logiotatidis [:giorgos]]

This is related to the recent upgrade to ElasticSearch v2 and the lack of facets in this version:

Facets have been deprecated in favor of aggregations. To make these pages work we would have to upgrade our elasticsearch python libraries and re-write those pages to work with aggregations.

For some pages we've replaced facets with db searches and caching (see bug 1381436) but we didn't work on this advanced board.

Given the development state of Kitsune and the complications of those dashboards we need to evaluate their use and importance.

Patrick can you please provide insights on the use and importance of metrics dashboards and especially for

 - https://support.mozilla.org/en-US/community/top-contributors/questions/new
 - https://support.mozilla.org/en-US/community/top-contributors/l10n/new

which are broken at the moment. Given that it's required large amounts of development work to get those running again would you spend time on those?

Thanks!

Comment 5

[vesper]

We are not sure what the future of the community hub as such is (this is a part of it, as described by Noah), but this feature would be great to have, since it allows us to analyse community participation over custom periods of time and message people in bulk at the same time.

@Madalina - Since you worked closely on the Community Hub, what is your take on its future in SUMO/on Kitsune? Is this something we'd let Participation Systems figure out with access to our data or do you think it should be prioritized as one of the top fixes for Kitsune in case we get dev power?