Closed Bug 1023541 Opened 10 years ago Closed 10 years ago

Deploy relengapi 0.2.1 and mapper 0.2.1 into production https://api.pub.build.mozilla.org/

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: pmoore, Assigned: pmoore)

References

Details

This is required in order to complete bug 847640.

Then mapper will be available here:
https://api.pub.build.mozilla.org/mapper
Depends on: 1023555
Blocks: 1023576
Rolling this out now...
I've created utility script: https://github.com/petemoore/myscrapbook/blob/master/relengapi_put_requirements.sh for this.

However, a problem finding pytz on our pypi server at the moment - need to look into this. Seems to work ok on staging - but maybe was manually installed in the virtualenv there? I'll have a look and see...

Downloading/unpacking pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1))
  http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available
  Could not find a version that satisfies the requirement pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1)) (from versions: 2011d)
Cleaning up...
No distributions matching the version for pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1))
Fixed, by uploading pytz-2014.1.tar.bz2 using https://hg.mozilla.org/build/braindump/rev/1e2cd2c76d4f

Deployment of packages complete:
https://api.pub.build.mozilla.org/versions

Still need to configure mapper to use new prod schema, create the new schema, populate the projects table, create the relengapi auth tokens, and sanity check everything - keeping this bug open until all steps completed.
I've hit some problems setting up the new database - see https://bugzilla.mozilla.org/show_bug.cgi?id=1023555#c7
I think we're done here now?
Almost! I need to create database, and now I can do it from web heads, but not from admin node (relengwebadm) - still need to do that. Although I'm not quite sure where the virtualenv is on the webheads yet, so I might just tunnel to them from relengwebadm to run the createdb from there...

Should be done shortly!

P.S. I just this second wrote comment: https://bugzilla.mozilla.org/show_bug.cgi?id=1023555#c16
OK, the tunnel didn't work - no idea why, but anyway, I used the DDL SQL from the staging environment directly in a mysql session which I ran from web1.releng.webapp.scl3:



[root@web1.releng.webapp.scl3 bin]# mysql -u mapper -pXXXXXXXXXXXX -h generic-rw-zeus.db.scl3.mozilla.com mapper
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 14492045
Server version: 5.6.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE TABLE `projects` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `name` varchar(255) NOT NULL,
    ->   PRIMARY KEY (`id`),
    ->   UNIQUE KEY `name` (`name`)
    -> ) ENGINE=InnoDB AUTO_INCREMENT=59 DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> CREATE TABLE `hashes` (
    ->   `hg_changeset` varchar(40) NOT NULL,
    ->   `git_commit` varchar(40) NOT NULL,
    ->   `project_id` int(11) NOT NULL,
    ->   `date_added` int(11) NOT NULL,
    ->   UNIQUE KEY `project_id__hg_changeset` (`project_id`,`hg_changeset`),
    ->   UNIQUE KEY `project_id__git_commit` (`project_id`,`git_commit`),
    ->   KEY `project_id` (`project_id`),
    ->   KEY `hg_changeset` (`hg_changeset`),
    ->   KEY `project_id__date_added` (`project_id`,`date_added`),
    ->   KEY `git_commit` (`git_commit`),
    ->   CONSTRAINT `hashes_ibfk_1` FOREIGN KEY (`project_id`) REFERENCES `projects` (`id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.04 sec)

mysql> quit
Bye
[root@web1.releng.webapp.scl3 bin]# 


This should be equivalent to running relengapi createdb.
That said, I probably need to do the same for relengapi (that was only mapper). Maybe I'll do that in the morning using relengapi when bug 1026117 is ready...
And I can't really close this bug until I've also generated tokens in the new environment for mapper, and migrated the other settings across from settings.py and tested the environment is running ok, LDAP authentication is working etc - it shouldn't take long after the new flow is in place.

Thanks,
Pete
Bug 1026117 isn't ready yet, so I created from mysql client on web1.releng.webapp.scl3.mozilla.com:

[root@web1.releng.webapp.scl3 ~]# mysql -u relengapi -pXXXXXXXXXXXXXXXX -h generic-rw-zeus.db.scl3.mozilla.com relengapi
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 14536606
Server version: 5.6.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE TABLE `auth_tokens` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `description` text NOT NULL,
    ->   `_permissions` text NOT NULL,
    ->   PRIMARY KEY (`id`)
    -> ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.06 sec)

mysql> CREATE TABLE `oauth2_clients` (
    ->   `client_id` varchar(40) NOT NULL,
    ->   `client_secret` varchar(55) NOT NULL,
    ->   `_redirect_uris` text,
    ->   `_default_scopes` text,
    ->   PRIMARY KEY (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> CREATE TABLE `oauth2_grants` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `user_email` varchar(255) NOT NULL,
    ->   `client_id` varchar(40) NOT NULL,
    ->   `code` varchar(255) NOT NULL,
    ->   `redirect_uri` varchar(255) DEFAULT NULL,
    ->   `expires` datetime DEFAULT NULL,
    ->   `_scopes` text,
    ->   PRIMARY KEY (`id`),
    ->   KEY `client_id` (`client_id`),
    ->   KEY `ix_oauth2_grants_code` (`code`),
    ->   CONSTRAINT `oauth2_grants_ibfk_1` FOREIGN KEY (`client_id`) REFERENCES `oauth2_clients` (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.04 sec)

mysql> CREATE TABLE `oauth2_tokens` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `client_id` varchar(40) NOT NULL,
    ->   `user_email` varchar(255) NOT NULL,
    ->   `token_type` varchar(40) DEFAULT NULL,
    ->   `access_token` varchar(255) DEFAULT NULL,
    ->   `refresh_token` varchar(255) DEFAULT NULL,
    ->   `expires` datetime DEFAULT NULL,
    ->   `_scopes` text,
    ->   PRIMARY KEY (`id`),
    ->   UNIQUE KEY `access_token` (`access_token`),
    ->   UNIQUE KEY `refresh_token` (`refresh_token`),
    ->   KEY `client_id` (`client_id`),
    ->   CONSTRAINT `oauth2_tokens_ibfk_1` FOREIGN KEY (`client_id`) REFERENCES `oauth2_clients` (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> quit;
Bye
[root@web1.releng.webapp.scl3 ~]#
Ah, looks like all the settings have been migrated from staging to prod that were required to be - thanks Dustin!
OK, I think this is done! Closing!
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.