569675 - Setup buildbot slave(s) for BYOB repacks

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect, P2)

Description

[Chris Cooper [:coop] (he/him)]

Related to bug 563798.

We need to setup the BYOB pool of slaves to be identical. The easiest way to do this is to fully document the setup of a single slave, and then clone this slave as a ref image. 

Part of that setup should be the installation of puppet so we can deploy new tools easily.

Comment 1

[Chris Cooper [:coop] (he/him)]

The slaves already have puppet installed on them, but since they came from a talos image, I've had to scrub them a little since there are some incompatibilities with more recent tools installs.

I've created a new puppet class for byob-buildslave, and have made a new package for deploying the repackaging tools to both byob slaves and regular mac builders (for partner repacks). Some final testing is still required, but I should be ready for review soon.

Comment 2

[Chris Cooper [:coop] (he/him)]

Attachment: Create new class for BYOB buildslaves and new package for repackaging tools

This patch does the following:

* adds a new class byob-buildslave. This class is pretty much identical to the darwin10 builder at , but we *may* need to diverge the two at some point. Adding the dkpg tools for Mac is the standout example.

* adds a new tools package for the repackaging tools. This will be used by the BYOB buildslaves, but also by the production/staging slaves for repacking partner builds. There are versions of this package created for 10.5 and 10.6.

* adds the BYOB buildslaves to the list of production slaves

Using this patch on staging-puppet, I was able to successfully install these packages on moz2-darwin10-slave01, moz2-darwin9-slave03, and byob-repack01.

Comment 3

[Mike Taylor [:bear]]

Comment on attachment 450422 [details] [diff] [review]
Create new class for BYOB buildslaves and new package for repackaging tools

I have a question about including 7zip in the manifest for repackgingtools as it should already be present in the refimage.  If your just trying to make sure that it's present, shouldn't it be moved up to the more general osx.pp manifest?

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 450422 [details] [diff] [review]
Create new class for BYOB buildslaves and new package for repackaging tools

Since we're considering 'byob' slaves a subset of darwin10-i386-build can you create a new node for them, in base/nodes.pp? That way, you can inherit byob-buildslave in there rather than in all of the concrete nodes themselves.

Comment 5

[Chris Cooper [:coop] (he/him)]

(In reply to comment #3)
> (From update of attachment 450422 [details] [diff] [review])
> I have a question about including 7zip in the manifest for repackgingtools as
> it should already be present in the refimage.  If your just trying to make sure
> that it's present, shouldn't it be moved up to the more general osx.pp
> manifest?

7zip is only on the ref image for Windows, not for Mac which is where all our pacakging needs to happen due to the DMGs.

(In reply to comment #4)
> (From update of attachment 450422 [details] [diff] [review])
> Since we're considering 'byob' slaves a subset of darwin10-i386-build can you
> create a new node for them, in base/nodes.pp? That way, you can inherit
> byob-buildslave in there rather than in all of the concrete nodes themselves.

OK, I'll set this up and resubmit.

Comment 6

[Chris Cooper [:coop] (he/him)]

Attachment: Create new class for BYOB buildslaves and new package for repackaging tools, v2

Adds the new node as suggested by bhearsum, and renames the slave class to byob-repackslave to be more accurate.

Still works in staging.

Comment 7

[Mike Taylor [:bear]]

(In reply to comment #5)
> (In reply to comment #3)
> > (From update of attachment 450422 [details] [diff] [review] [details])
> > I have a question about including 7zip in the manifest for repackgingtools as
> > it should already be present in the refimage.  If your just trying to make sure
> > that it's present, shouldn't it be moved up to the more general osx.pp
> > manifest?
> 
> 7zip is only on the ref image for Windows, not for Mac which is where all our
> pacakging needs to happen due to the DMGs.

ah, I had added it to the 10.6 refimage when I created that.  Just assumed it had been done also for the 10.5.

> 
> (In reply to comment #4)
> > (From update of attachment 450422 [details] [diff] [review] [details])
> > Since we're considering 'byob' slaves a subset of darwin10-i386-build can you
> > create a new node for them, in base/nodes.pp? That way, you can inherit
> > byob-buildslave in there rather than in all of the concrete nodes themselves.
> 
> OK, I'll set this up and resubmit.

Comment 8

[Chris Cooper [:coop] (he/him)]

Comment on attachment 450681 [details] [diff] [review]
Create new class for BYOB buildslaves and new package for repackaging tools, v2

Landed:
http://hg.mozilla.org/build/puppet-manifests/rev/a10462b3e958

All 5 slaves are now connected to production-puppet and have buildbot 0.8.0 and the repack tools they need.