1444841 - Handle pingback error messages properly

Status: RESOLVED FIXED

(Conduit :: Lando, enhancement)

Description

[Israel Madueme [:imadueme]]

When transplant is unable to apply a patch successfully it responds with a very descriptive error message, most likely directly from the hg import output.

When transplant sends this information along in the pingback to lando-api, lando-api fails (crashes) when saving the landing info due to a database constraint error: the landing 'error' field to store this information is limited to 128 characters, but, errors from transplant apparently can be bigger than that. (https://github.com/mozilla-conduit/lando-api/blob/master/migrations/67151bb74080_add_error_message_and_result_to_landing.py#L21)

In addition to this problem, transplant seems to return a tuple of the (hg error code, hg error message, '', '') (unknown what those last two are, haven't seen them used). We would have to properly extract the message in order to display it. Currently we (try to) save this tuple as a string. Or perhaps clients should be responsible for extracting the message, which lando-ui does not do currently. https://bugzilla.mozilla.org/show_bug.cgi?id=1444840 might change this behavior, though.

Regardless, we must increase the error column size.

Exception:

The message in this case was: (255, "applying /tmp/tmpJccXwZ\nunable to find ''client.py'' for patching\n(use ''--prefix'' to apply patch relative to the current directory)\n1 out of 1 hunks FAILED -- saving rejects to file client.py.rej\nabort: patch failed to apply", '''')


lando-api.db_1  | 2018-03-12 09:29:41.843 UTC [28] ERROR:  value too long for type character varying(128)
lando-api.db_1  | 2018-03-12 09:29:41.843 UTC [28] STATEMENT:  UPDATE landings SET status='failed', error='(255, "applying /tmp/tmpJccXwZ\nunable to find ''client.py'' for patching\n(use ''--prefix'' to apply patch relative to the current directory)\n1 out of 1 hunks FAILED -- saving rejects to file client.py.rej\nabort: patch failed to apply", '''')', updated_at='2018-03-12T09:29:41.842989'::timestamp WHERE landings.id = 3
lando-api_1     | [2018-03-12 09:29:41,849] ERROR in app: Exception on /landings/update [POST]
lando-api_1     | Traceback (most recent call last):
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
lando-api_1     |     context)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
lando-api_1     |     cursor.execute(statement, parameters)
lando-api_1     | psycopg2.DataError: value too long for type character varying(128)
lando-api_1     |
lando-api_1     |
lando-api_1     | The above exception was the direct cause of the following exception:
lando-api_1     |
lando-api_1     | Traceback (most recent call last):
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1982, in wsgi_app
lando-api_1     |     response = self.full_dispatch_request()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1614, in full_dispatch_request
lando-api_1     |     rv = self.handle_user_exception(e)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1517, in handle_user_exception
lando-api_1     |     reraise(exc_type, exc_value, tb)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
lando-api_1     |     raise value
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
lando-api_1     |     rv = self.dispatch_request()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
lando-api_1     |     return self.view_functions[rule.endpoint](**req.view_args)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/connexion/decorators/decorator.py", line 66, in wrapper
lando-api_1     |     response = function(request)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/connexion/decorators/validation.py", line 122, in wrapper
lando-api_1     |     response = function(request)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/connexion/decorators/validation.py", line 293, in wrapper
lando-api_1     |     return function(request)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/connexion/decorators/decorator.py", line 42, in wrapper
lando-api_1     |     response = function(request)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/connexion/decorators/parameter.py", line 195, in wrapper
lando-api_1     |     return function(**kwargs)
lando-api_1     |   File "/app/landoapi/auth.py", line 585, in wrapped
lando-api_1     |     return f(*args, **kwargs)
lando-api_1     |   File "/app/landoapi/api/landings.py", line 216, in update
lando-api_1     |     landing.save()
lando-api_1     |   File "/app/landoapi/models/landing.py", line 203, in save
lando-api_1     |     return db.session.commit()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/scoping.py", line 157, in do
lando-api_1     |     return getattr(self.registry(), name)(*args, **kwargs)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 906, in commit
lando-api_1     |     self.transaction.commit()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
lando-api_1     |     self._prepare_impl()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
lando-api_1     |     self.session.flush()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2171, in flush
lando-api_1     |     self._flush(objects)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2291, in _flush
lando-api_1     |     transaction.rollback(_capture_exception=True)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
lando-api_1     |     compat.reraise(exc_type, exc_value, exc_tb)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
lando-api_1     |     raise value
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2255, in _flush
lando-api_1     |     flush_context.execute()
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
lando-api_1     |     rec.execute(self)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
lando-api_1     |     uow
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
lando-api_1     |     mapper, table, update)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 737, in _emit_update_statements
lando-api_1     |     execute(statement, multiparams)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 945, in execute
lando-api_1     |     return meth(self, multiparams, params)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
lando-api_1     |     return connection._execute_clauseelement(self, multiparams, params)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
lando-api_1     |     compiled_sql, distilled_params
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
lando-api_1     |     context)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
lando-api_1     |     exc_info
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
lando-api_1     |     reraise(type(exception), exception, tb=exc_tb, cause=cause)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
lando-api_1     |     raise value.with_traceback(tb)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
lando-api_1     |     context)
lando-api_1     |   File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
lando-api_1     |     cursor.execute(statement, parameters)
lando-api_1     | sqlalchemy.exc.DataError: (psycopg2.DataError) value too long for type character varying(128)
lando-api_1     |  [SQL: 'UPDATE landings SET status=%(status)s, error=%(error)s, updated_at=%(updated_at)s WHERE landings.id = %(landings_id)s'] [parameters: {'updated_at': datetime.datetime(2018, 3, 12, 9, 29, 41, 842989), 'status': 'failed', 'error': '(255, "applying /tmp/tmpJccXwZ\\nunable to find \'client.py\' for patching\\n(use \'--prefix\' to apply patch relative to the current directory)\\n1 out of 1 hunks FAILED -- saving rejects to file client.py.rej\\nabort: patch failed to apply", \'\')', 'landings_id': 3}]

Comment 1

[Mark Côté [:mcote]]

I'm tempted to punt on these, but you are right, they should have been fixed as part of an earlier story.  But since that story was a while ago, I'll just have them block the current one.

Comment 2

[Phabricator Automation]

Attachment: pingback: properly handle large pingbacks and landed status.

Comment 3

[Phabricator Automation]

Comment on attachment 8960338 [details]
pingback: properly handle large pingbacks and landed status.

Byron Jones ‹:glob› has approved the revision.

https://phabricator.services.mozilla.com/D775