Deploy an http cache for elastic search on stage.

RESOLVED INCOMPLETE

Status

Cloud Services
Operations: Marketplace
RESOLVED INCOMPLETE
5 years ago
3 years ago

People

(Reporter: alexis, Assigned: jason)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
https://github.com/ametaireau/httpcache is the code of an HTTP proxy handling cache for elasticsearch. We now need to deploy this alongside each webhead, in order to test if it speed things-up or not.

This means:

- Starting the httpcache proxy with the appropriate ports and adresses (I would make it listen from the localhost:9200 to elasticsearch:9200. Note that it needs to be indicated the location of memcache as well.
- Changing the host and port in the configuration of zamboni

And that should be it. You also can specify it the address to send statsd information, which could be helpful to gather statistics as well.
(Assignee)

Updated

5 years ago
Assignee: server-ops → jthomas
(Assignee)

Comment 1

5 years ago
I have deployed httpcache on stage and listening on localhost:9200. I configured stage zamboni to to point to httpcache however it is causing search pages to error. I have reverted the settings change for now. Please ping me on irc to troubleshoot.

[root@web1.stage.addons.phx1 ~]# curl http://localhost:9200/_cluster/health?pretty=true
{
  "cluster_name" : "addons-stage-elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 35,
  "active_shards" : 70,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0


This is what I see in the logs after hitting https://addons.allizom.org/en-US/firefox/search/?q=computer&cat=1%2C0&appver=18.0&platform=mac
Jan 31 11:03:41 web1.stage.addons.phx1.mozilla.com: [][] z.es:ERROR {'filter': {'and': [{'term': {'app': 1}}, {'term': {'is_disabled': False}}, {'in': {'status': (8, 9, 4)}}, {'in': {'platform': (3, 1)}}, {'term': {'type': 1}}]}, 'query': {'bool': {'should': [{'text': {'summary': {'query': u'computer', 'boost': 0.80000000000000004, 'type': 'phrase'}}}, {'fuzzy': {'app_slug': {'boost': 2, 'prefix_length': 4, 'value': u'computer'}}}, {'text': {'name_english': {'query': u'computer', 'boost': 2.5, 'analyzer': 'english'}}}, {'prefix': {'app_slug': {'boost': 1.5, 'value': u'computer'}}}, {'text': {'app_slug': {'query': u'computer', 'boost': 4, 'type': 'phrase'}}}, {'text': {'authors': {'query': u'computer', 'boost': 4, 'type': 'phrase'}}}, {'text': {'summary_english': {'query': u'computer', 'boost': 0.59999999999999998, 'analyzer': 'english', 'type': 'phrase'}}}, {'fuzzy': {'name': {'boost': 2, 'prefix_length': 4, 'value': u'computer'}}}, {'prefix': {'slug': {'boost': 1.5, 'value': u'computer'}}}, {'fuzzy': {'authors': {'boost': 2, 'prefix_length': 4, 'value': u'computer'}}}, {'text': {'slug': {'query': u'computer', 'boost': 4, 'type': 'phrase'}}}, {'text': {'description_english': {'query': u'computer', 'boost': 0.10000000000000001, 'analyzer': 'english', 'type': 'phrase'}}}, {'fuzzy': {'slug': {'boost': 2, 'prefix_length': 4, 'value': u'computer'}}}, {'text': {'name': {'query': u'computer', 'boost': 4, 'type': 'phrase'}}}, {'prefix': {'name': {'boost': 1.5, 'value': u'computer'}}}, {'text': {'description': {'query': u'computer', 'boost': 0.29999999999999999, 'type': 'phrase'}}}, {'prefix': {'authors': {'boost': 1.5, 'value': u'computer'}}}]}}, 'facets': {'platforms': {'terms': {'field': 'platform'}}, 'appversions': {'terms': {'field': 'appversion.1.max'}}, 'categories': {'terms': {'field': 'category', 'size': 200}}, 'tags': {'terms': {'field': 'tag'}}}, 'size': 20, 'fields': ['id']} :/data/www/addons.allizom.org/zamboni/apps/amo/search.py:200
Jan 31 11:03:41 web1.stage.addons.phx1.mozilla.com: [][] z.es:ERROR Elasticsearch error:  :/data/www/addons.allizom.org/zamboni/apps/search/middleware.py:16

strace on the httpcache process:

accept(4, {sa_family=AF_INET, sin_port=htons(53267), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5
fcntl(5, F_GETFL)                       = 0x2 (flags O_RDWR)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
accept(4, 0x7fff47934c60, [16])         = -1 EAGAIN (Resource temporarily unavailable)
fcntl(5, F_GETFL)                       = 0x802 (flags O_RDWR|O_NONBLOCK)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
recvfrom(5, "GET /addons_stage/addons/_search HTTP/1.1\r\nHost: 127.0.0.1:9200\r\nAccept-Encoding: identity\r\nContent-Length: 1825\r\n\r\n{\"filter\": {\"and\": [{\"term\": {\"app\": 1}}, {\"term\": {\"is_disabled\": false}}, {\"in\": {\"status\": [8, 9, 4]}}, {\"in\": {\"platform\": [3, 1]}}, {\"term\": {\"type\": 1}}]}, \"query\": {\"bool\": {\"should\": [{\"text\": {\"summary\": {\"query\": \"computer\", \"boost\": 0.80000000000000004, \"type\": \"phrase\"}}}, {\"fuzzy\": {\"app_slug\": {\"boost\": 2, \"prefix_length\": 4, \"value\": \"computer\"}}}, {\"text\": {\"name_english\": {\"query\": \"computer\", \"boost\": 2.5, \"analyzer\": \"english\"}}}, {\"prefix\": {\"app_slug\": {\"boost\": 1.5, \"value\": \"computer\"}}}, {\"text\": {\"app_slug\": {\"query\": \"computer\", \"boost\": 4, \"type\": \"phrase\"}}}, {\"text\": {\"authors\": {\"query\": \"computer\", \"boost\": 4, \"type\": \"phrase\"}}}, {\"text\": {\"summary_english\": {\"query\": \"computer\", \"boost\": 0.59999999999999998, \"analyzer\": \"english\", \"type\": \"phrase\"}}}, {\"fuzzy\": {\"name\": {\"boost\": 2, \"prefix_length\": 4, \"value\": \"computer\"}}}, {\"prefix\": {\"slug\": {\"boost\": 1.5, \"v"..., 8192, 0, NULL, NULL) = 1941
getsockname(5, {sa_family=AF_INET, sin_port=htons(9200), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getpeername(5, {sa_family=AF_INET, sin_port=htons(53267), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
write(2, "<socket fileno=5 sock=127.0.0.1:9200 peer=127.0.0.1:53267>: Unexpected Content-Length\n", 86) = 86
sendto(5, "HTTP/1.0 400 Bad Request\r\nConnection: close\r\nContent-length: 0\r\n\r\n", 66, 0, NULL, 0) = 66
close(5)                                = 0
Status: NEW → ASSIGNED
(Assignee)

Comment 2

5 years ago
As per @alexis we will not be proceeding with this.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INCOMPLETE
Component: Server Operations: AMO Operations → Operations: Marketplace
Product: mozilla.org → Mozilla Services
You need to log in before you can comment on or make changes to this bug.