assign b-2008 and try-2008 spot runners in slavealloc

RESOLVED FIXED

Status

Infrastructure & Operations
RelOps: Puppet
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: grenade, Assigned: grenade)

Tracking

Details

(Whiteboard: [windows][aws])

Attachments

(2 attachments)

(Assignee)

Description

3 years ago
We need just 1 of each initially to validate spot allocation works as expected on Windows.
(Assignee)

Comment 1

3 years ago
does this look sane for the slavealloc dbimport rail, and arr did i get the c3.xlarge bit right?:

name,distro,bitlength,speed,datacenter,trustlevel,environment,purpose,pool,basedir
y-2008-spot-001,win2k8,64,c3.xlarge,use1,try,prod,build,try-use1,/builds/slave
b-2008-spot-001,win2k8,64,c3.xlarge,use1,core,prod,build,build-use1,/builds/slave
Flags: needinfo?(rail)
Flags: needinfo?(arich)
(Assignee)

Comment 2

3 years ago
arr: i think we need to revisit the naming for try. we have a 15 char limit. as you can see above, i dropped ec2 from the name because spot implies the same and reduced try to y in order to get it inside 15 characters.
do you object to this approach?
I thought the 15 char limit on windows was because of the domain (and since these aren't being attached to the domain, it wasn't a problem)?

Also, we want c3.2xlarge, not c3.xlarge.
Flags: needinfo?(arich)
The datacenter (datacentre?) field should match the region name used by AWS API, e.g. us-west-2 or us-east-1, see https://github.com/mozilla/build-cloud-tools/blob/master/cloudtools/tests/test_cloudtools_slavealloc.py#L87 as an example.
Flags: needinfo?(rail)
rail: so for "speed" there is no option for c3.2xlarge, apparently. Should we be creating a new speed entry type, or should be be using the misleading c3.xlarge?
Flags: needinfo?(rail)
"speed" is not used anywhere in cloud-tools, except for tst-linux64/tst-emulator64, see https://github.com/mozilla/build-cloud-tools/blob/master/cloudtools/slavealloc.py#L79-L88.

Creating a new entry for c3.2xlarge would be better, it'd be less misleading.
Flags: needinfo?(rail)
According to q, we are going to run into > 15 char naming concention for try, so we should stick with y-2008-spot-XXX (and rob and Q should chat about the name not sticking during a reboot). Changing to y over try- will also require some modifications to the cloud-tools configs, too (a grep for try-2008 should find everythign easily enough).
Whiteboard: [windows] → [windows][aws]
(Assignee)

Comment 9

3 years ago
insert into speeds (name) values ('c3.2xlarge')
done on both staging and prod
(Assignee)

Comment 10

3 years ago
slavealloc dbimport failed (db url and credentials removed for this comment only, creds were valid and work for normal queries):

[rthijssen@relengwebadm.private.scl3 ~]$ cat ./b-y-2008-trial.csv 
name,distro,bitlength,speed,datacenter,trustlevel,environment,purpose,pool,basedir
y-2008-spot-001,win2k8,64,c3.2xlarge,us-east-1,try,prod,build,try-use1,/builds/slave
b-2008-spot-001,win2k8,64,c3.2xlarge,us-east-1,core,prod,build,build-use1,/builds/slave

[rthijssen@relengwebadm.private.scl3 ~]$ /data/releng/www/slavealloc/slavealloc dbimport -D mysql://xxx...  --master-data ./b-y-2008-trial.csv
/data/releng/www/slavealloc/slavealloc
while executing subcommand
Traceback (most recent call last):
  File "/data/releng/www/slavealloc/deps/Twisted-12.1.0/twisted/internet/defer.py", line 290, in addCallbacks
    self._runCallbacks()
  File "/data/releng/www/slavealloc/deps/Twisted-12.1.0/twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/data/releng/www/slavealloc/deps/Twisted-12.1.0/twisted/internet/base.py", line 426, in _continueFiring
    callable(*args, **kwargs)
  File "/data/releng/www/slavealloc/deps/tools/lib/python/slavealloc/scripts/main.py", line 46, in do_command
    d = defer.maybeDeferred(func, args)
--- <exception caught here> ---
  File "/data/releng/www/slavealloc/deps/Twisted-12.1.0/twisted/internet/defer.py", line 134, in maybeDeferred
    result = f(*args, **kw)
  File "/data/releng/www/slavealloc/deps/tools/lib/python/slavealloc/scripts/dbimport.py", line 90, in main
    [r['distro'] for r in slaves])
  File "/data/releng/www/slavealloc/deps/tools/lib/python/slavealloc/scripts/dbimport.py", line 72, in normalize
    res = table.select().execute()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/expression.py", line 3098, in execute
    return e._execute_clauseelement(self, multiparams, params)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1628, in _execute_clauseelement
    connection = self.contextual_connect(close_with_result=True)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1669, in contextual_connect
    self.pool.connect(),
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 272, in connect
    return _ConnectionFairy(self).checkout()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 425, in __init__
    rec = self._connection_record = pool._do_get()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 777, in _do_get
    con = self._create_connection()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 225, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 318, in __init__
    self.connection = self.__connect()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 368, in __connect
    connection = self.__pool._creator()
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 279, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/lib64/python2.6/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 193, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (OperationalError) (1045, "Access denied for user 'xxxg'@'xxx' (using password: YES)") None None
[rthijssen@relengwebadm.private.scl3 ~]$ cat ./b
bitlengths.txt      b-y-2008-trial.csv
I added those 2 slaves
(Assignee)

Comment 12

3 years ago
Created attachment 8642987 [details] [diff] [review]
support y-2008 as a valid puppet node

I left try-2008 in place for the time being and just added y-2008 to see if there's anything else broken by the name change.
Attachment #8642987 - Flags: review?(arich)
(Assignee)

Updated

3 years ago
Attachment #8642987 - Attachment is patch: true
Attachment #8642987 - Flags: review?(arich) → review+
(Assignee)

Updated

3 years ago
See Also: → bug 1191717
(Assignee)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Comment 14

3 years ago
Created attachment 8650472 [details]
new 2008 spot instances
You need to log in before you can comment on or make changes to this bug.