Make OrangeFactor, logparser & mozautoeslib compatible with ElasticSearch 1.2

RESOLVED WONTFIX

Status

RESOLVED WONTFIX
4 years ago
a year ago

People

(Reporter: emorley, Unassigned)

Tracking

Details

Bug 1071044 / bug would like to update ElasticSearch from v0.9 to 1.2.
This has already occurred on es-dev, but not prod.

In bug 1121507 it was discovered OrangeFactor and/or the libraries it used needed changes to be compatible with v1.2.

Some things were fixed (bug 1121941, bug 1121956), however there was at least one issue left (and likely more, since we were only using ES 1.2 then for logparser submissions, and nothing else):

(In reply to Ed Morley [:edmorley] from bug 1121507 comment #9)
> (In reply to Ed Morley [:edmorley] from bug 1121507 comment #4)
> > There's also still the "ElasticSearchException: Unknown exception type"
> > (comment 2).
> 
> We're currently using pyes v0.16 (was v0.15 until bug 1121517 comment ), but
> v0.19+ gives much better output for this exception:
> https://github.com/aparo/pyes/commit/b61b4be45804abbec6fdd0f43e6127d3643e1e8e
> 
> However there are lots of breaking changes, so I've backported that fix by
> hand to
> logparser/lib/python2.6/site-packages/pyes-0.16.0-py2.6.egg/pyes/
> convert_errors.py temporarily.
> 
> This revealed:
> 
> Traceback (most recent call last):
>   File "/home/webtools/apps/logparser/src/logparser/logparser/savelogs.py",
> line 64, in parse
>     lp.parseFiles()
>   File "/home/webtools/apps/logparser/src/logparser/logparser/logparser.py",
> line 106, in parseFiles
>     self.postResultsToElasticSearch(testdata)
>   File "/home/webtools/apps/logparser/src/logparser/logparser/logparser.py",
> line 196, in postResultsToElasticSearch
>     self._post_testgroup_to_elasticsearch(data)
>   File "/home/webtools/apps/logparser/src/logparser/logparser/logparser.py",
> line 183, in _post_testgroup_to_elasticsearch
>     testgroup.submit()
>   File
> "/home/webtools/apps/logparser/src/mozautolog/mozautolog/esautolog.py", line
> 80, in submit
>     self._generate_testrun()
>   File
> "/home/webtools/apps/logparser/src/mozautolog/mozautolog/esautolog.py", line
> 58, in _generate_testrun
>     doc_type = [self.doc_type])
>   File
> "/home/webtools/apps/logparser/src/mozautoeslib/mozautoeslib/eslib.py", line
> 186, in query
>     doc_types=self.doc_type)
>   File
> "/home/webtools/apps/logparser/lib/python2.6/site-packages/pyes-0.16.0-py2.6.
> egg/pyes/es.py", line 893, in count
>     return self._query_call("_count", query, indexes, doc_types,
> **query_params)
>   File
> "/home/webtools/apps/logparser/lib/python2.6/site-packages/pyes-0.16.0-py2.6.
> egg/pyes/es.py", line 251, in _query_call
>     response = self._send_request('GET', path, body, querystring_args)
>   File
> "/home/webtools/apps/logparser/lib/python2.6/site-packages/pyes-0.16.0-py2.6.
> egg/pyes/es.py", line 223, in _send_request
>     raise_if_error(response.status, decoded)
>   File
> "/home/webtools/apps/logparser/lib/python2.6/site-packages/pyes-0.16.0-py2.6.
> egg/pyes/convert_errors.py", line 58, in raise_if_error
>     raise pyes.exceptions.ElasticSearchException(u"Unknown exception type:
> %d, %s" % (status, result), status, result)
> ElasticSearchException: Unknown exception type: 400, {u'count': 0,
> u'_shards': {u'successful': 0, u'failed': 5, u'total': 5, u'failures':
> [{u'index': u'logs', u'reason':
> u'BroadcastShardOperationFailedException[[logs][0] ]; nested:
> QueryParsingException[[logs] request does not support [bool]]; ', u'shard':
> 0}, {u'index': u'logs', u'reason':
> u'BroadcastShardOperationFailedException[[logs][2] ]; nested:
> RemoteTransportException[[elasticsearch2_dev_phx1][inet[/SNIP:9300]][count/
> s]]; nested: QueryParsingException[[logs] request does not support [bool]];
> ', u'shard': 2}, {u'index': u'logs', u'reason':
> u'BroadcastShardOperationFailedException[[logs][1] ]; nested:
> RemoteTransportException[[elasticsearch2_dev_phx1][inet[/SNIP:9300]][count/
> s]]; nested: QueryParsingException[[logs] request does not support [bool]];
> ', u'shard': 1}, {u'index': u'logs', u'reason':
> u'BroadcastShardOperationFailedException[[logs][4] ]; nested:
> QueryParsingException[[logs] request does not support [bool]]; ', u'shard':
> 4}, {u'index': u'logs', u'reason':
> u'BroadcastShardOperationFailedException[[logs][3] ]; nested:
> RemoteTransportException[[elasticsearch1_dev_phx1][inet[/SNIP:9300]][count/
> s]]; nested: QueryParsingException[[logs] request does not support [bool]];
> ', u'shard': 3}]}}
Depends on: 1121941, 1121956

Updated

3 years ago
No longer blocks: 1062342
See Also: → bug 1062342
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
Blocks: 1356410
You need to log in before you can comment on or make changes to this bug.