1441838 - Test Autopush Dev Server

Status: VERIFIED FIXED

(Cloud Services :: QA: General, defect)

Description

[Chris Hartjes [:grumpy][:chartjes]]

QA was asked to run through some tests of a new Rust-based autopush service that has been deployed to the existing dev environment

Comment 1

[Chris Hartjes [:grumpy][:chartjes]]

For URL checks, the 'status' end point works but the 'health' end point does not

Using the following values for the tests:

server_url = https://autopush-rust.dev.mozaws.net
push_server = wss://autopush-rust.dev.mozaws.net
router = dev.autopush.routerv2
storage = dev.autopush.storage
bugzilla_api = https://bugzilla-dev.allizom.org
bugzilla_user = [e-mail]
bugzilla_api_key = oelpF8y4nHYuYf3xOxll4XQXTJtPAkIwrs1BmHvO


 pytest -v --env=dev tests/test_urls.py
================================================== test session starts ==================================================
platform darwin -- Python 2.7.14, pytest-3.0.7, py-1.5.2, pluggy-0.4.0 -- /Users/chartjes/.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/bin/python2.7
cachedir: .cache
rootdir: /Users/chartjes/rpappalax/autopush-integration-tests, inifile: tox.ini
plugins: variables-1.6.1
collected 2 items

tests/test_urls.py::test_health_endpoint FAILED
tests/test_urls.py::test_status_endpoint PASSED
================================================ short test summary info ================================================
FAIL tests/test_urls.py::test_health_endpoint

======================================================= FAILURES ========================================================
_________________________________________________ test_health_endpoint __________________________________________________

api_version = '0.0.0', conf = <backports.configparser.ConfigParser object at 0x105ef3210>, env = 'dev'

    def test_health_endpoint(api_version, conf, env):
>       response = requests.get(conf.get(env, 'server_url') + "/health")

tests/test_urls.py:5:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/lib/python2.7/site-packages/requests/api.py:72: in get
    return request('get', url, params=params, **kwargs)
../../.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/lib/python2.7/site-packages/requests/api.py:58: in request
    return session.request(method=method, url=url, **kwargs)
../../.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/lib/python2.7/site-packages/requests/sessions.py:518: in request
    resp = self.send(prep, **send_kwargs)
../../.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/lib/python2.7/site-packages/requests/sessions.py:639: in send
    r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <requests.adapters.HTTPAdapter object at 0x105fb5f90>, request = <PreparedRequest [GET]>, stream = False
timeout = <requests.packages.urllib3.util.timeout.Timeout object at 0x10608d7d0>, verify = True, cert = None
proxies = OrderedDict([('http', 'http://127.0.0.1:8118'), ('https', 'http://127.0.0.1:8118')])

    def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
        """Sends PreparedRequest object. Returns Response object.

            :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
            :param stream: (optional) Whether to stream the request content.
            :param timeout: (optional) How long to wait for the server to send
                data before giving up, as a float, or a :ref:`(connect timeout,
                read timeout) <timeouts>` tuple.
            :type timeout: float or tuple or urllib3 Timeout object
            :param verify: (optional) Either a boolean, in which case it controls whether
                we verify the server's TLS certificate, or a string, in which case it
                must be a path to a CA bundle to use
            :param cert: (optional) Any user-provided SSL certificate to be trusted.
            :param proxies: (optional) The proxies dictionary to apply to the request.
            :rtype: requests.Response
            """

        conn = self.get_connection(request.url, proxies)

        self.cert_verify(conn, request.url, verify, cert)
        url = self.request_url(request, proxies)
        self.add_headers(request)

        chunked = not (request.body is None or 'Content-Length' in request.headers)

        if isinstance(timeout, tuple):
            try:
                connect, read = timeout
                timeout = TimeoutSauce(connect=connect, read=read)
            except ValueError as e:
                # this may raise a string formatting error.
                err = ("Invalid timeout {0}. Pass a (connect, read) "
                       "timeout tuple, or a single float to set "
                       "both timeouts to the same value".format(timeout))
                raise ValueError(err)
        elif isinstance(timeout, TimeoutSauce):
            pass
        else:
            timeout = TimeoutSauce(connect=timeout, read=timeout)

        try:
            if not chunked:
                resp = conn.urlopen(
                    method=request.method,
                    url=url,
                    body=request.body,
                    headers=request.headers,
                    redirect=False,
                    assert_same_host=False,
                    preload_content=False,
                    decode_content=False,
                    retries=self.max_retries,
                    timeout=timeout
                )

            # Send the request.
            else:
                if hasattr(conn, 'proxy_pool'):
                    conn = conn.proxy_pool

                low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)

                try:
                    low_conn.putrequest(request.method,
                                        url,
                                        skip_accept_encoding=True)

                    for header, value in request.headers.items():
                        low_conn.putheader(header, value)

                    low_conn.endheaders()

                    for i in request.body:
                        low_conn.send(hex(len(i))[2:].encode('utf-8'))
                        low_conn.send(b'\r\n')
                        low_conn.send(i)
                        low_conn.send(b'\r\n')
                    low_conn.send(b'0\r\n\r\n')

                    # Receive the response from the server
                    try:
                        # For Python 2.7+ versions, use buffering of HTTP
                        # responses
                        r = low_conn.getresponse(buffering=True)
                    except TypeError:
                        # For compatibility with Python 2.6 versions and back
                        r = low_conn.getresponse()

                    resp = HTTPResponse.from_httplib(
                        r,
                        pool=conn,
                        connection=low_conn,
                        preload_content=False,
                        decode_content=False
                    )
                except:
                    # If we hit any problems here, clean up the connection.
                    # Then, reraise so that we can handle the actual exception.
                    low_conn.close()
                    raise

        except (ProtocolError, socket.error) as err:
>           raise ConnectionError(err, request=request)
E           ConnectionError: ('Connection aborted.', BadStatusLine("''",))

../../.local/share/virtualenvs/autopush-integration-tests-_LJXrNHM/lib/python2.7/site-packages/requests/adapters.py:488: ConnectionError

Comment 2

[Chris Hartjes [:grumpy][:chartjes]]

DESKTOP E2E PUSH TEST

Install Docker, grab the image firefoxtesteng/autopush-e2e-test and then start the container using the command:

docker run -p 8201:8201 firefoxtesteng/autopush-e2e-test

Using a fresh profile for Firefox, go to to http://localhost:8201

* go to about:config and set dom.push.serverURL = wss://autopush-rust.dev.mozaws.net/

Test #1

* with 'service-worker.js' selected click on 'Register Service Worker' and see the messages:

registering service worker
registered service worker. scope: http://localhost:8201/

* alter the 'Title' and 'Body' fields in the Notifications Property form to unique values
* click on 'pop Notification' and verify the notification contains your expected Title and Body
* click on 'close Notification' and verify the notification is closed

Test #2

* click on 'subscribe to push'
* change values for 'repeat' and 'delay' to have at least 2 messages with at least 5 seconds between them
* click on 'XHR to webpush app-server' and verify the number of messages sent and the delay between them is accurate


Passes

Comment 3

[Chris Hartjes [:grumpy][:chartjes]]

The topic server testing as outlined at https://github.com/jrconlin/topics with dom.push.serverURL pointing at the dev instance do not work

2018-02-28T10:20:04-0500 [HTTPConnection,0,127.0.0.1] [http] 200 GET / (127.0.0.1) 11.45ms
2018-02-28T10:20:04-0500 [HTTPConnection,1,127.0.0.1] [http] 200 GET /style.css (127.0.0.1) 0.86ms
2018-02-28T10:20:04-0500 [HTTPConnection,2,127.0.0.1] [http] 200 GET /sw.js (127.0.0.1) 0.92ms
2018-02-28T10:20:04-0500 [HTTPConnection,3,127.0.0.1] [http] 404 GET /favicon.ico (127.0.0.1) 0.81ms
2018-02-28T10:20:09-0500 [push_server.handler.MainHandler#info] Unable to format event {'log_namespace': 'push_server.handler.MainHandler', 'log_level': <LogLevel=info>, 'log_logger': <Logger 'push_server.handler.MainHandler'>, 'log_time': 1519831209.017774, 'log_source': <push_server.handler.MainHandler object at 0x10703bfd0>, 'log_format': "Writing body: {u'keys': {u'p256dh': u'BMCU_NAZQ4mThZXYtlJ0VzMNYTeEolacgumdQ2JfrDRR3OUHGhl-33QRiXbYu_UtFBlAjuq90HWaJ_P0c-UyQek', u'auth': u'jPozIhzq6lHwN6ZuiuFRjQ'}, u'endpoint': u'https://updates-autopush.dev.mozaws.net/wpush/v1/gAAAAABalsioHccrfJyK7QF8wMQcMJRYFosjBX8fxfhTImBzlolF6nKOavruGkGZXJSRH129Idqye5R0XO2s7LkQX00tPxem2S8isMJH18zNqE-nidJcOiEoD7Rbp_nFzmINIqQvEeno'}"}: u"u'keys'"
2018-02-28T10:20:09-0500 [push_server.handler.MainHandler#info] Done!
2018-02-28T10:20:09-0500 [HTTPConnection,4,127.0.0.1] [http] 200 POST / (127.0.0.1) 2.06ms


topic_pusher --msg "Hello Chris"                                                                               
No valid subscription file found. Extra data: line 1 column 345 - line 1 column 347 (char 344 - 346)
Could not send message: local variable 'sub_info' referenced before assignment

Comment 4

[JR Conlin [:jrconlin,:jconlin]]

hmm, at first glance this looks like an issue with the topic caller. Give me a bit to work out what might be going on here.

Comment 5

[JR Conlin [:jrconlin,:jconlin]]

Odd. I'm unable to reproduce the bug locally.

Please check that:
1) there's a file named `creds.txt` located in the root of the topics directory,
2) that it contains the data:
```
{'keys': 
  {'p256dh': 'BMCU_NAZQ4mThZXYtlJ0VzMNYTeEolacgumdQ2JfrDRR3OUHGhl-33QRiXbYu_UtFBlAjuq90HWaJ_P0c-UyQek', 
   'auth': 'jPozIhzq6lHwN6ZuiuFRjQ'}, 
'endpoint': 'https://updates-autopush.dev.mozaws.net/wpush/v1/gAAAAABalsioHccrfJyK7QF8wMQcMJRYFosjBX8fxfhTImBzlolF6nKOavruGkGZXJSRH129Idqye5R0XO2s7LkQX00tPxem2S8isMJH18zNqE-nidJcOiEoD7Rbp_nFzmINIqQvEeno'}
```

I suspect that the `creds.txt` file is corrupted or incomplete since the pusher is unable to load it.

Comment 6

[Ben Bangert [:benbangert]]

We expect the health/status checks to look different. There is no health one, as the status is currently handling it. The status should return something acceptable for the status check though.

Comment 7

[Chris Hartjes [:grumpy][:chartjes]]

(In reply to JR Conlin [:jrconlin,:jconlin] from comment #5)
> Odd. I'm unable to reproduce the bug locally.
> 
> Please check that:
> 1) there's a file named `creds.txt` located in the root of the topics
> directory,
> 2) that it contains the data:
> ```
> {'keys': 
>   {'p256dh':
> 'BMCU_NAZQ4mThZXYtlJ0VzMNYTeEolacgumdQ2JfrDRR3OUHGhl-
> 33QRiXbYu_UtFBlAjuq90HWaJ_P0c-UyQek', 
>    'auth': 'jPozIhzq6lHwN6ZuiuFRjQ'}, 
> 'endpoint':
> 'https://updates-autopush.dev.mozaws.net/wpush/v1/
> gAAAAABalsioHccrfJyK7QF8wMQcMJRYFosjBX8fxfhTImBzlolF6nKOavruGkGZXJSRH129Idqye
> 5R0XO2s7LkQX00tPxem2S8isMJH18zNqE-nidJcOiEoD7Rbp_nFzmINIqQvEeno'}
> ```
> 
> I suspect that the `creds.txt` file is corrupted or incomplete since the
> pusher is unable to load it.

I do have that creds.txt file and it matches what you provided.

Comment 8

[Chris Hartjes [:grumpy][:chartjes]]

(In reply to Ben Bangert [:benbangert] from comment #6)
> We expect the health/status checks to look different. There is no health
> one, as the status is currently handling it. The status should return
> something acceptable for the status check though.

Once you've updated the "health" value for status, I will update my deployment tests to account for it.

Comment 9

[JR Conlin [:jrconlin,:jconlin]]

(In reply to Chris Hartjes [:grumpy][:chartjes] from comment #7)
> I do have that creds.txt file and it matches what you provided.

Odd. I just duplicated the creds.txt data on my local machine and it worked fine. (I'll note that I did have to change the single quote (') to double quotes (") for the JSON to load, but that would have displayed a different error.

The original error message indicates that there is an invalid character in your `creds.txt` file. Probably near the end of the file. Try editing that file and making sure that there's no extra character past the final '}'

If that fails, it may be worth deleting the creds.txt file and re-running the test to see if the problem persists.

Comment 10

[Chris Hartjes [:grumpy][:chartjes]]

After speaking with benbangert about the health check no longer being available but the status check being acceptable, I approve the deployment of the new Rust version to staging for the normal testing.