734389 - Deploy Thunderbird keys to build machines

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P1)

Description

[John Hopkins (:jhopkins)]

We need to deploy Thunderbird SSH keys to build machines (in the Firefox pool) so we can upload completed builds.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

found during triage.

Comment 2

[John Hopkins (:jhopkins)]

We need to manually scp the Thunderbird key onto all the build slaves and update the ref images.

Comment 3

[John Hopkins (:jhopkins)]

Attachment: print list of production build machines

I copy/pasted the mozilla/production_config.py and mozilla-tests/production_config.py code to generate lists of build machines in the attached script.  I noticed there are only 72 matches on 'darwin' which seems low.  Can you have a look and see if I missed anything?

Comment 4

[Chris AtLee [:catlee]]

Comment on attachment 615708 [details]
print list of production build machines

looks ok to me...what slaves does it output?

when you're running this, can you verify the .ssh/ffxbld_dsa exists and that .ssh/trybld_dsa does NOT exist before copying the thunderbird keys over?

Comment 5

[John Hopkins (:jhopkins)]

Attachment: print list of production machines

updated: no tegras, no talos

Comment 6

[John Hopkins (:jhopkins)]

Attachment: list of build slave to copy thunderbird key to

Comment 7

[[e-mail] (:bhearsum)]

Comment on attachment 615733 [details]
list of build slave to copy thunderbird key to

I'm told that the bld-centos6* machines don't even have Firefox keys on them, as they only do b2g builds. No need to deploy tbirdbld keys to them.

moz2-darwin9-slave01 and w64-ix-slave22 are dev/pp machines according to slavealloc, please don't deploy the production keys to them.

moz2-darwin9-slave17 is dead, apparently (bug 714812)

w64-ix-slave02 isn't in slavealloc, I'm not sure if it still exists.

Do you have a staging tbirdbld key yet? We'll need one to deploy to the dev/pp build machines & dev-stage01. That list of machines is:
linux-ix-slave03, 04, 05
linux64-ix-slave01, 02
moz2-darwin10-slave02, 03, 04, 10
moz2-darwin9-slave01, 08, 68
moz2-linux-slave03, 04, 10, 17, 51
moz2-linux64-slave07, 10
mv-moz2-linux-ix-slave01
mw32-ix-slave01, 19, 21
w32-ix-slave01
w64-ix-slave04, 05, 22, 41

Comment 8

[[e-mail] (:bhearsum)]

From IRC, we need the tbirdbld staging key on all of the staging build machines, too. I think the complete list is:
linux-ix-slave03, 04, 05
linux64-ix-slave01, 02
moz2-darwin10-slave02, 03, 04, 10
moz2-darwin9-slave01, 08, 68
moz2-linux-slave03, 04, 10, 17, 51
moz2-linux64-slave07, 10
mv-moz2-linux-ix-slave01
mw32-ix-slave01, 19, 21
w32-ix-slave01
w64-ix-slave04, 05, 22, 41

Comment 9

[John Hopkins (:jhopkins)]

Attachment: print list of production machines to receive thunderbird keys

added exceptions

Comment 10

[John Hopkins (:jhopkins)]

Attachment: updated slave list to copy production thunderbird keys to

Comment 11

[John Hopkins (:jhopkins)]

Attachment: updated list of build slaves to copy production thunderbird keys to

Comment 12

[John Hopkins (:jhopkins)]

Attachment: list of staging build slaves to copy tbirdbld staging key to

Comment 13

[[e-mail] (:bhearsum)]

Comment on attachment 615768 [details]
updated list of build slaves to copy production thunderbird keys to

Almost forgot about ref images! No need to update this attachment, but the following need the keys too:
linux-ix-ref
linux64-ix-ref
moz2-darwin10-ref
win32-ix-ref

...and file a bug to get IT to take new images of them. Similar to bug 733865.

I don't know what the deal is with windows 64-bit ix machines.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Ben Hearsum [:bhearsum] from comment #13)
> Comment on attachment 615768 [details]
> Almost forgot about ref images! No need to update this attachment, but the
> following need the keys too:

Turns out we handle keys in different ways depending on the platform

> linux-ix-ref
> linux64-ix-ref
> moz2-darwin10-ref

These puppet-based boxes have staging keys on the ref images, presumably so they go to staging masters for proving before going into production.

> win32-ix-ref

This has production keys (except for the old cvs/cvs.pub which is staging). I don't know why win32 is treated differently. Ben ?

Comment 15

[[e-mail] (:bhearsum)]

(In reply to Nick Thomas [:nthomas] from comment #14)
> (In reply to Ben Hearsum [:bhearsum] from comment #13)
> > Comment on attachment 615768 [details]
> > Almost forgot about ref images! No need to update this attachment, but the
> > following need the keys too:
> 
> Turns out we handle keys in different ways depending on the platform
> 
> > linux-ix-ref
> > linux64-ix-ref
> > moz2-darwin10-ref
> 
> These puppet-based boxes have staging keys on the ref images, presumably so
> they go to staging masters for proving before going into production.
> 
> > win32-ix-ref
> 
> This has production keys (except for the old cvs/cvs.pub which is staging).
> I don't know why win32 is treated differently. Ben ?

Probably no good reason. We should follow this pattern for Thunderbird though, lest we end up even more confused.

Comment 16

[John Hopkins (:jhopkins)]

Key installation for staging, production, and win32-ix-ref is done.  briar-patch (https://github.com/mozilla/briar-patch/) and paramiko were used to automate this process.

Filed bug 747020 to track creation of a new win32-ix-ref image.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

I removed the staging copy of tbirdbld_dsa.pub from bld-lion-r5-041..80 (non-try prod), because it was causing errors connecting to stage. I also removed trybld_dsa*, which were also staging but not necessarily from this bug.