Closed
Bug 1019847
Opened 10 years ago
Closed 10 years ago
Ship and deploy relengapi-0.2.0, relengapi-mapper-0.2.0
Categories
(Infrastructure & Operations :: RelOps: General, task)
Infrastructure & Operations
RelOps: General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: dustin, Assigned: pmoore)
References
Details
This will include, in particular: https://github.com/mozilla/build-relengapi/pull/40 https://github.com/mozilla/build-relengapi/pull/23 https://github.com/mozilla/build-relengapi/issues/8 (aka bug 998948) some of which are breaking changes, hence the 0.2.0. The mapper deployment will need to contain https://github.com/petemoore/mapper/pull/4 I'm happy to give it a different number than 0.2.0 if requested. This will block on the latter pull request being merged (pmoore?). Reassign to me after that, and let me know if it's OK to break staging at that point.
|
Reporter | |
Updated•10 years ago
|
|
Assignee | |
Comment 1•10 years ago
|
||
Working on this now. I've just released both mapper and relengapi. I agreed with your version number suggestions: it makes sense to move to a 3 part version number (0.2.0) for mapper, rather than just a two part like it was before (0.8) since i'd like to hit 1.0 only when we go live, and I like having two more variables, so we can indicate non-backwardly-compatible changes. Currently deploying into staging - need to update some pypi packages on pypi server... Downloading/unpacking SQLAlchemy>=0.9.4 (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Could not find a version that satisfies the requirement SQLAlchemy>=0.9.4 (from relengapi==0.2.0->-r requirements.txt (line 1)) (from versions: 0.6.4, 0.6.6, 0.7.9, 0.8.0b2, migrate-0.7.2) Cleaning up... No distributions matching the version for SQLAlchemy>=0.9.4 (from relengapi==0.2.0->-r requirements.txt (line 1)) Storing debug log for failure in /root/.pip/pip.log
|
|
Assignee | |
Comment 2•10 years ago
|
||
^^^ 1.0.0, not 1.0, of course!! ^^^
|
|
Assignee | |
Comment 3•10 years ago
|
||
Published SQLAlchemy-0.9.4.tar.gz on our pypi server
|
|
Assignee | |
Comment 4•10 years ago
|
||
Refreshed deployment with the new versions! https://api-pub-build.allizom.org/versions { "result": { "blueprints": { "auth": { "distribution": "relengapi", "version": "0.2.0" }, "base": { "distribution": "relengapi", "version": "0.2.0" }, "docs": { "distribution": "relengapi", "version": "0.2.0" }, "mapper": { "distribution": "relengapi-mapper", "version": "0.2.0" }, "tokenauth": { "distribution": "relengapi", "version": "0.2.0" } }, "distributions": { "amqp": { "project_name": "amqp", "version": "1.4.4" }, "anyjson": { "project_name": "anyjson", "version": "0.3.3" }, "argparse": { "project_name": "argparse", "version": "1.1" }, "billiard": { "project_name": "billiard", "version": "3.3.0.16" }, "blinker": { "project_name": "blinker", "version": "1.3" }, "celery": { "project_name": "celery", "version": "3.1.9" }, "concurrentloghandler": { "project_name": "ConcurrentLogHandler", "version": "0.9.1" }, "docutils": { "project_name": "docutils", "version": "0.11" }, "flask": { "project_name": "Flask", "version": "0.10.1" }, "flask-browserid": { "project_name": "Flask-BrowserID", "version": "0.0.4" }, "flask-login": { "project_name": "Flask-Login", "version": "0.2.10" }, "flask-oauthlib": { "project_name": "Flask-OAuthlib", "version": "0.4.3" }, "flask-principal": { "project_name": "Flask-Principal", "version": "0.4.0" }, "ipy": { "project_name": "IPy", "version": "0.81" }, "itsdangerous": { "project_name": "itsdangerous", "version": "0.23" }, "jinja2": { "project_name": "Jinja2", "version": "2.7.2" }, "kombu": { "project_name": "kombu", "version": "3.0.14" }, "markupsafe": { "project_name": "MarkupSafe", "version": "0.19" }, "mysql-python": { "project_name": "MySQL-python", "version": "1.2.5" }, "oauthlib": { "project_name": "oauthlib", "version": "0.6.1" }, "pip": { "project_name": "pip", "version": "1.5.4" }, "pygments": { "project_name": "Pygments", "version": "1.6" }, "python": { "project_name": "Python", "version": "2.7.3" }, "python-dateutil": { "project_name": "python-dateutil", "version": "2.2" }, "pytz": { "project_name": "pytz", "version": "2014.1" }, "relengapi": { "project_name": "relengapi", "version": "0.2.0" }, "relengapi-docs": { "project_name": "relengapi-docs", "version": "0.1.8" }, "relengapi-mapper": { "project_name": "relengapi-mapper", "version": "0.2.0" }, "requests": { "project_name": "requests", "version": "2.2.1" }, "setuptools": { "project_name": "setuptools", "version": "2.2" }, "six": { "project_name": "six", "version": "1.6.1" }, "sphinx": { "project_name": "Sphinx", "version": "1.2.2" }, "sqlalchemy": { "project_name": "SQLAlchemy", "version": "0.9.4" }, "werkzeug": { "project_name": "Werkzeug", "version": "0.9.4" }, "wrapt": { "project_name": "wrapt", "version": "1.8.0" }, "wsgiref": { "project_name": "wsgiref", "version": "0.1.2" } } } }
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
|
|
Assignee | |
Comment 5•10 years ago
|
||
2014-06-09 07:26:05,614 [relengapi.app] Exception on /mapper/build-tools/insert/ignoredups [POST]
Traceback (most recent call last):
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/relengapi/lib/permissions.py", line 74, in req
if not can(*permissions):
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/relengapi/lib/permissions.py", line 58, in can
return all(perm in current_user.permissions for perm in permissions)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/relengapi/lib/permissions.py", line 58, in <genexpr>
return all(perm in current_user.permissions for perm in permissions)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/werkzeug/local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/werkzeug/local.py", line 297, in _get_current_object
return self.__local()
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask_login.py", line 46, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask_login.py", line 794, in _get_user
current_app.login_manager._load_user()
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask_login.py", line 361, in _load_user
return self._load_from_request(request)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/flask_login.py", line 429, in _load_from_request
user = self.request_callback(request)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/relengapi/lib/auth/__init__.py", line 117, in _request_loader
u = loader(request)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/relengapi/blueprints/tokenauth/__init__.py", line 186, in token_loader
token_data = Token.query.filter_by(id=token_info['id']).first()
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2333, in first
ret = list(self[0:1])
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2200, in __getitem__
return list(res)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2404, in __iter__
return self._execute_and_instances(context)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2419, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 720, in execute
return meth(self, multiparams, params)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 317, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 817, in _execute_clauseelement
compiled_sql, distilled_params
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
context)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1108, in _handle_dbapi_exception
exc_info
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 185, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 940, in _execute_context
context)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 435, in do_execute
cursor.execute(statement, parameters)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/data/www/relengapi/virtualenv/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (OperationalError) (1054, "Unknown column 'auth_tokens._permissions' in 'field list'") 'SELECT auth_tokens.id AS auth_tokens_id, auth_tokens.description AS auth_tokens_description, auth_tokens._permissions AS auth_tokens__permissions \nFROM auth_tokens \nWHERE auth_tokens.id = %s \n LIMIT %s' (1, 1)Status: RESOLVED → REOPENED
Resolution: FIXED → ---
|
|
Assignee | |
Comment 6•10 years ago
|
||
This didn't seem to fix it, will investigate schema: [root@relengwebadm.private.scl3 relengapi]# cd /data/releng-stage/src/relengapi [root@relengwebadm.private.scl3 relengapi]# SITE_DIR=`cd $(dirname $0); pwd` [root@relengwebadm.private.scl3 relengapi]# SITE=relengapi [root@relengwebadm.private.scl3 relengapi]# CLUSTER=`echo $SITE_DIR | sed -e 's!^/data/\([^/]*\)/.*!\1!'` [root@relengwebadm.private.scl3 relengapi]# cd $SITE_DIR [root@relengwebadm.private.scl3 relengapi]# virtualenv/bin/pip install -U --no-index --find-links=http://pypi.pub.build.mozilla.org/pub/ -r requirements.txt Ignoring indexes: https://pypi.python.org/simple/ http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: relengapi==0.2.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: relengapi-mapper==0.2.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: MySQL-python==1.2.5 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 3)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: ConcurrentLogHandler==0.9.1 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 4)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Flask in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Flask-Login>=0.2.10 in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Flask-Browserid in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Sphinx in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: SQLAlchemy>=0.9.4 in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Celery in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: argparse in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: requests in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: wrapt in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: blinker in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: pytz==2014.1 in ./virtualenv/lib/python2.7/site-packages (from relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: IPy in ./virtualenv/lib/python2.7/site-packages (from relengapi-mapper==0.2.0->-r requirements.txt (line 2)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: python-dateutil in ./virtualenv/lib/python2.7/site-packages (from relengapi-mapper==0.2.0->-r requirements.txt (line 2)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Werkzeug>=0.7 in ./virtualenv/lib/python2.7/site-packages (from Flask->relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: Jinja2>=2.4 in ./virtualenv/lib/python2.7/site-packages (from Flask->relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: itsdangerous>=0.21 in ./virtualenv/lib/python2.7/site-packages (from Flask->relengapi==0.2.0->-r requirements.txt (line 1)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: six in ./virtualenv/lib/python2.7/site-packages (from python-dateutil->relengapi-mapper==0.2.0->-r requirements.txt (line 2)) http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available Requirement already up-to-date: markupsafe in ./virtualenv/lib/python2.7/site-packages (from Jinja2>=2.4->Flask->relengapi==0.2.0->-r requirements.txt (line 1)) Cleaning up... [root@relengwebadm.private.scl3 relengapi]# RELENGAPI_SETTINGS=$SITE_DIR/settings.py $SITE_DIR/virtualenv/bin/relengapi createdb 2014-06-09 07:25:20,313 registering blueprint mapper 2014-06-09 07:25:20,316 registering blueprint docs 2014-06-09 07:25:20,412 registering blueprint docs 2014-06-09 07:25:20,412 registering blueprint base 2014-06-09 07:25:20,413 registering blueprint tokenauth 2014-06-09 07:25:20,414 registering blueprint auth 2014-06-09 07:25:20,415 creating tables for database mapper 2014-06-09 07:25:20,524 creating tables for database relengapi [root@relengwebadm.private.scl3 relengapi]#
|
|
Assignee | |
Comment 7•10 years ago
|
||
[root@relengwebadm.private.scl3 relengapi]# mysqldump -u relengapi_stage -pXXXXXXXXXXX -h stage-rw-vip.db.scl3.mozilla.com --no-data relengapi_stage -- MySQL dump 10.13 Distrib 5.1.71, for redhat-linux-gnu (x86_64) -- -- Host: stage-rw-vip.db.scl3.mozilla.com Database: relengapi_stage -- ------------------------------------------------------ -- Server version 5.6.12-log /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!40101 SET NAMES utf8 */; /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */; /*!40103 SET TIME_ZONE='+00:00' */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; mysqldump: Couldn't execute 'SET OPTION SQL_QUOTE_SHOW_CREATE=1': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_QUOTE_SHOW_CREATE=1' at line 1 (1064)
|
|
Assignee | |
Comment 8•10 years ago
|
||
It looks like I might need to drop the tables and recreate the schema: [root@relengwebadm.private.scl3 relengapi]# mysql -u relengapi_stage -pXXXXXXXXXXX -h stage-rw-vip.db.scl3.mozilla.com relengapi_stage Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7121156 Server version: 5.6.12-log MySQL Community Server (GPL) Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show create table auth_tokens; +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | auth_tokens | CREATE TABLE `auth_tokens` ( `id` int(11) NOT NULL AUTO_INCREMENT, `description` text NOT NULL, `_actions` text NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 | +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> It looks like _actions column should be renamed to _permissions - here is a local sqlite3 database: pmoore@Elisandra:~/work/mapper master $ sqlite3 relengapi.db SQLite version 3.7.13 2012-07-17 17:46:21 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> .schema CREATE TABLE auth_tokens ( id INTEGER NOT NULL, description TEXT NOT NULL, _permissions TEXT NOT NULL, PRIMARY KEY (id) ); sqlite>
|
|
Assignee | |
Comment 9•10 years ago
|
||
mysql> alter table auth_tokens change _actions _permissions text NOT NULL; Query OK, 0 rows affected (0.04 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> Let's see if that worked!
|
|
Assignee | |
Comment 10•10 years ago
|
||
That seems to have fixed it.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
|
|
Reporter | |
Comment 11•10 years ago
|
||
Thanks for working on this! I also updated staging to use LDAP groups for authentication, which seems to be working (as far as I can see as a member of cn=team_relops!). However, https://github.com/mozilla/build-relengapi/issues/83
You need to log in
before you can comment on or make changes to this bug.
Description
•