847441 - Add remaining win8 machines to the production pool

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

This will require the following changes:
* add the slaves to production config [1]
* insert them into slavealloc [2]
** make sure to use the values for production rather than preproduction
* insert them into graphs [3]
** find the id of "WINNT 6.2 x64" rather than using LAST_INSERT_ID()
* reboot the machines into production [4]
** do not reboot them until the first 3 steps are accomplished

[1] http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/production_config.py#l7
[2] https://bugzilla.mozilla.org/attachment.cgi?id=720087
[3] https://bug844130.bugzilla.mozilla.org/attachment.cgi?id=719004
[4] for i in {11..97}; do ipmitool -U <username> -P <password> -H t-w864-ix-0${i}-mgmt.inband.releng.scl3.mozilla.com chassis power soft; done

Comment 1

[Carsten Book [:Tomcat]]

Attachment: add the slaves to production config

patch to add the slaves to production config

Comment 2

[Carsten Book [:Tomcat]]

Attachment: patch

updated patch

Comment 3

[Armen [:armenzg]]

Comment on attachment 721169 [details] [diff] [review]
patch

Review of attachment 721169 [details] [diff] [review]:
-----------------------------------------------------------------

Can you please post one more patch addressing the comments? I will review it right away.

::: mozilla-tests/production_config.py
@@ +4,5 @@
>      'xp': dict([("talos-r3-xp-%03i" % x, {}) for x in range(4,10) + range(11,101) \
>            if x not in [45, 58, 59]]), # bug 661377, bug 780515, bug 753357
>      'win7': dict([("talos-r3-w7-%03i" % x, {}) for x in range(4,10) + range(11,17) + range(18,105)]),
> +    'win8': dict([("t-w864-ix-%03i" % x, {}) for x in range(1,10) + range(11,98) \
> +        if x not in [98]]), # IT reserving t-w864-ix-098 to continue to work on the graphics automation

An interesting thing about range() in python is that the last number is excluded.
e.g. range(1,2) only contains 1.

Could you also make it:
range(1,98)
instead of:
range(1,10) + range(11,98)

We're not going to leave gaps in our ranges for staging slaves (We used to keep slaves 1 to 3 and 10 as staging slaves). We decided it yesterday in the Monday meeting.

If you want increase it to 99 and keep the if below OR keep it at 98 and remove the if statement.

Comment 4

[Carsten Book [:Tomcat]]

Attachment: updated patch

review comments adressed

Comment 5

[Armen [:armenzg]]

Comment on attachment 721222 [details] [diff] [review]
updated patch

Review of attachment 721222 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/production_config.py
@@ +4,4 @@
>      'xp': dict([("talos-r3-xp-%03i" % x, {}) for x in range(4,10) + range(11,101) \
>            if x not in [45, 58, 59]]), # bug 661377, bug 780515, bug 753357
>      'win7': dict([("talos-r3-w7-%03i" % x, {}) for x in range(4,10) + range(11,17) + range(18,105)]),
> +    'win8': dict([("t-w864-ix-%03i" % x, {}) for x in range(1,10) + range(11,99) \

Can you please do "range(1,99)" instead of "range(1,10) + range(11,99)"?

r+ with that change.

Comment 6

[Armen [:armenzg]]

(In reply to Armen Zambrano G. [:armenzg] from comment #5)
...
> 
> Can you please do "range(1,99)" instead of "range(1,10) + range(11,99)"?
> 
> r+ with that change.

Feel free to land once you make the change.

Comment 7

[Carsten Book [:Tomcat]]

Comment on attachment 721222 [details] [diff] [review]
updated patch

checked in https://hg.mozilla.org/build/buildbot-configs/rev/56ad3e4e38e6

Comment 8

[Carsten Book [:Tomcat]]

Attachment: csv file for slavealloc addition

part2:
* insert them into slavealloc [2]
** make sure to use the values for production rather than preproduction

Comment 9

[Armen [:armenzg]]

Comment on attachment 721649 [details]
csv file for slavealloc addition

There are two ways to insert data into slavealloc.

One of them is as a typical set of mysql insert statement like https://bug844130.bugzilla.mozilla.org/attachment.cgi?id=719005

The other one is through the script "slavealloc dbimport"
/tools/slavealloc/bin/slavealloc dbimport -D $db_url --slave-data mydata.csv [1][2]
"The CSV file should have the headers specified by 'slavealloc dbimport --help'". I've never tried it myself but here's what the headers should look like: 
name,distro,bitlength,speed,datacenter,trustlevel,environment,purpose,pool,basedir,enabled
talos-r3-xp-096,winxp,32,mini,scl1,try,prod,tests,tests-scl1-windows,C:\\talos-slave,1

You got it similar to the second method but it has to be comma separated and with the header. Otherwise slavealloc dbimport will complain.


[1] https://wiki.mozilla.org/ReleaseEngineering/How_To/Create_new_slaves_or_move_them_to_other_pools
[2] https://wiki.mozilla.org/ReleaseEngineering:Buildduty:Slave_Management#Adding_a_slave

Comment 10

[Carsten Book [:Tomcat]]

Attachment: new try, this time comma seperated

Comment 11

[Armen [:armenzg]]

Comment on attachment 721707 [details]
new try, this time comma seperated

It should be "prod" instead of "pro".
It should be "tests" instead of "pool".

These values can be determined by visiting http://slavealloc.build.mozilla.org/ui/#slaves and filtering by "t-w864".

Please remove t-w864-ix-042 (it already exists) and t-w864-ix-098 (we won't be using it) from that list.

r+ with those changes.

Please use "slavealloc dbimport" method when importing the data. Let me know if you would like to share a screen session on slavealloc so I can guide you with it.

Comment 12

[Carsten Book [:Tomcat]]

Hey Armen, thanks for the review, so the docs tell for db url

[root@slavealloc ~]# /tools/slavealloc/bin/slavealloc dbimport -D $db_url --slave-data win8slavealloc2.csv 
usage: slavealloc dbimport [-h] [-D DBURL] [--slave-data SLAVE_DATA]
                           [--master-data MASTER_DATA]
                           [--master-json MASTER_JSON]
                           [--password-data PASSWORD_DATA] [--enable]
slavealloc dbimport: error: argument -D/--db: expected one argument

here $db_url is most easily found in slavealloc's shell history, do -D $db is -d devtools-rw-vip.db.scl3.mozilla.com right ?

Comment 13

[Carsten Book [:Tomcat]]

(In reply to Carsten Book [:Tomcat] from comment #12)
> Hey Armen, thanks for the review, so the docs tell for db url
> 
> [root@slavealloc ~]# /tools/slavealloc/bin/slavealloc dbimport -D $db_url
> --slave-data win8slavealloc2.csv 
> usage: slavealloc dbimport [-h] [-D DBURL] [--slave-data SLAVE_DATA]
>                            [--master-data MASTER_DATA]
>                            [--master-json MASTER_JSON]
>                            [--password-data PASSWORD_DATA] [--enable]
> slavealloc dbimport: error: argument -D/--db: expected one argument
> 
> here $db_url is most easily found in slavealloc's shell history, do -D $db
> is -d devtools-rw-vip.db.scl3.mozilla.com right ?

and done!

Comment 14

[Carsten Book [:Tomcat]]

Attachment: graph server patch/csv file

Comment 15

[Carsten Book [:Tomcat]]

Attachment: graph server patch/csv file

Comment 16

[Armen [:armenzg]]

Comment on attachment 722235 [details]
graph server patch/csv file

Those have to be mysql insert statements.

Comment 17

[Carsten Book [:Tomcat]]

Attachment: graph server patch/csv file

Comment 18

[Armen [:armenzg]]

Once we have a reconfig we should be able to add the machines into production by running this:
[4] for i in {11..97}; do ipmitool -U <username> -P <password> -H t-w864-ix-0${i}-mgmt.inband.releng.scl3.mozilla.com chassis power soft; done

Comment 19

[Armen [:armenzg]]

Merged and reconfiguration completed.

Comment 20

[Carsten Book [:Tomcat]]

(In reply to Armen Zambrano G. [:armenzg] from comment #18)
> Once we have a reconfig we should be able to add the machines into
> production by running this:
> [4] for i in {11..97}; do ipmitool -U <username> -P <password> -H
> t-w864-ix-0${i}-mgmt.inband.releng.scl3.mozilla.com chassis power soft; done

and done! its just doing the power soft!

Comment 21

[Armen [:armenzg]]

It seems that we had to run "chassis power reset" instead.

I have taken care of this and all win8 machines are now in production.