Deploy relengapi 0.2.1 and mapper 0.2.1 into production https://api.pub.build.mozilla.org/

RESOLVED FIXED

Status

Release Engineering
Tools
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: pmoore, Assigned: pmoore)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

3 years ago
This is required in order to complete bug 847640.

Then mapper will be available here:
https://api.pub.build.mozilla.org/mapper
(Assignee)

Updated

3 years ago
Depends on: 1023555
(Assignee)

Updated

3 years ago
Blocks: 1023576
(Assignee)

Comment 1

3 years ago
Rolling this out now...
(Assignee)

Comment 2

3 years ago
I've created utility script: https://github.com/petemoore/myscrapbook/blob/master/relengapi_put_requirements.sh for this.

However, a problem finding pytz on our pypi server at the moment - need to look into this. Seems to work ok on staging - but maybe was manually installed in the virtualenv there? I'll have a look and see...

Downloading/unpacking pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1))
  http://pypi.pub.build.mozilla.org/pub/ uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available
  Could not find a version that satisfies the requirement pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1)) (from versions: 2011d)
Cleaning up...
No distributions matching the version for pytz==2014.1 (from relengapi[ldap]==0.2.1->-r requirements.txt (line 1))
(Assignee)

Comment 3

3 years ago
Fixed, by uploading pytz-2014.1.tar.bz2 using https://hg.mozilla.org/build/braindump/rev/1e2cd2c76d4f

Deployment of packages complete:
https://api.pub.build.mozilla.org/versions

Still need to configure mapper to use new prod schema, create the new schema, populate the projects table, create the relengapi auth tokens, and sanity check everything - keeping this bug open until all steps completed.
(Assignee)

Comment 4

3 years ago
I've hit some problems setting up the new database - see https://bugzilla.mozilla.org/show_bug.cgi?id=1023555#c7
I think we're done here now?
(Assignee)

Comment 6

3 years ago
Almost! I need to create database, and now I can do it from web heads, but not from admin node (relengwebadm) - still need to do that. Although I'm not quite sure where the virtualenv is on the webheads yet, so I might just tunnel to them from relengwebadm to run the createdb from there...

Should be done shortly!

P.S. I just this second wrote comment: https://bugzilla.mozilla.org/show_bug.cgi?id=1023555#c16
(Assignee)

Comment 7

3 years ago
OK, the tunnel didn't work - no idea why, but anyway, I used the DDL SQL from the staging environment directly in a mysql session which I ran from web1.releng.webapp.scl3:



[root@web1.releng.webapp.scl3 bin]# mysql -u mapper -pXXXXXXXXXXXX -h generic-rw-zeus.db.scl3.mozilla.com mapper
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 14492045
Server version: 5.6.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE TABLE `projects` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `name` varchar(255) NOT NULL,
    ->   PRIMARY KEY (`id`),
    ->   UNIQUE KEY `name` (`name`)
    -> ) ENGINE=InnoDB AUTO_INCREMENT=59 DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> CREATE TABLE `hashes` (
    ->   `hg_changeset` varchar(40) NOT NULL,
    ->   `git_commit` varchar(40) NOT NULL,
    ->   `project_id` int(11) NOT NULL,
    ->   `date_added` int(11) NOT NULL,
    ->   UNIQUE KEY `project_id__hg_changeset` (`project_id`,`hg_changeset`),
    ->   UNIQUE KEY `project_id__git_commit` (`project_id`,`git_commit`),
    ->   KEY `project_id` (`project_id`),
    ->   KEY `hg_changeset` (`hg_changeset`),
    ->   KEY `project_id__date_added` (`project_id`,`date_added`),
    ->   KEY `git_commit` (`git_commit`),
    ->   CONSTRAINT `hashes_ibfk_1` FOREIGN KEY (`project_id`) REFERENCES `projects` (`id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.04 sec)

mysql> quit
Bye
[root@web1.releng.webapp.scl3 bin]# 


This should be equivalent to running relengapi createdb.
(Assignee)

Comment 8

3 years ago
That said, I probably need to do the same for relengapi (that was only mapper). Maybe I'll do that in the morning using relengapi when bug 1026117 is ready...
(Assignee)

Comment 9

3 years ago
And I can't really close this bug until I've also generated tokens in the new environment for mapper, and migrated the other settings across from settings.py and tested the environment is running ok, LDAP authentication is working etc - it shouldn't take long after the new flow is in place.

Thanks,
Pete
(Assignee)

Comment 10

3 years ago
Bug 1026117 isn't ready yet, so I created from mysql client on web1.releng.webapp.scl3.mozilla.com:

[root@web1.releng.webapp.scl3 ~]# mysql -u relengapi -pXXXXXXXXXXXXXXXX -h generic-rw-zeus.db.scl3.mozilla.com relengapi
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 14536606
Server version: 5.6.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE TABLE `auth_tokens` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `description` text NOT NULL,
    ->   `_permissions` text NOT NULL,
    ->   PRIMARY KEY (`id`)
    -> ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.06 sec)

mysql> CREATE TABLE `oauth2_clients` (
    ->   `client_id` varchar(40) NOT NULL,
    ->   `client_secret` varchar(55) NOT NULL,
    ->   `_redirect_uris` text,
    ->   `_default_scopes` text,
    ->   PRIMARY KEY (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> CREATE TABLE `oauth2_grants` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `user_email` varchar(255) NOT NULL,
    ->   `client_id` varchar(40) NOT NULL,
    ->   `code` varchar(255) NOT NULL,
    ->   `redirect_uri` varchar(255) DEFAULT NULL,
    ->   `expires` datetime DEFAULT NULL,
    ->   `_scopes` text,
    ->   PRIMARY KEY (`id`),
    ->   KEY `client_id` (`client_id`),
    ->   KEY `ix_oauth2_grants_code` (`code`),
    ->   CONSTRAINT `oauth2_grants_ibfk_1` FOREIGN KEY (`client_id`) REFERENCES `oauth2_clients` (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.04 sec)

mysql> CREATE TABLE `oauth2_tokens` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `client_id` varchar(40) NOT NULL,
    ->   `user_email` varchar(255) NOT NULL,
    ->   `token_type` varchar(40) DEFAULT NULL,
    ->   `access_token` varchar(255) DEFAULT NULL,
    ->   `refresh_token` varchar(255) DEFAULT NULL,
    ->   `expires` datetime DEFAULT NULL,
    ->   `_scopes` text,
    ->   PRIMARY KEY (`id`),
    ->   UNIQUE KEY `access_token` (`access_token`),
    ->   UNIQUE KEY `refresh_token` (`refresh_token`),
    ->   KEY `client_id` (`client_id`),
    ->   CONSTRAINT `oauth2_tokens_ibfk_1` FOREIGN KEY (`client_id`) REFERENCES `oauth2_clients` (`client_id`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> quit;
Bye
[root@web1.releng.webapp.scl3 ~]#
(Assignee)

Comment 11

3 years ago
Ah, looks like all the settings have been migrated from staging to prod that were required to be - thanks Dustin!
(Assignee)

Comment 12

3 years ago
OK, I think this is done! Closing!
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.