1007976 - Switch in house try builds from ceph to reverse-proxied S3

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Switch in house try builds from ceph to reverse-proxied S3

Comment 2

[Mike Hommey [:glandium]]

I'd need to give you the corresponding secrets to push in eyaml somehow.

Comment 3

[Mike Hommey [:glandium]]

Now, I'm wondering if this really ensures the secrets are going to be scraped when a slave is loaned.

Comment 4

[Mike Hommey [:glandium]]

Maybe this should be using slave_trustlevel instead of adding is_releng_try?

Comment 5

[Justin Wood (:Callek)]

Comment on attachment 8419776 [details] [diff] [review]
Switch in house try builds from ceph to reverse-proxied S3

Review of attachment 8419776 [details] [diff] [review]:
-----------------------------------------------------------------

::: manifests/moco-config.pp
@@ +47,5 @@
>      }
> +    $is_releng_try = $fqdn? {
> +        /.*\.try\.releng\..*\.mozilla\.com/ => true,
> +        default => false,
> +    }

This is not really a "config" thing imo, so I don't like it here.  (it also isn't completely accurate given 

Perhaps you can use "$slave_trustlevel"[1] instead?

[1]- http://mxr.mozilla.org/build/search?string=slave_trustlevel

Comment 6

[Mike Hommey [:glandium]]

Attachment: Switch in house try builds from ceph to reverse-proxied S3

Comment 7

[Justin Wood (:Callek)]

Comment on attachment 8421428 [details] [diff] [review]
Switch in house try builds from ceph to reverse-proxied S3

Review of attachment 8421428 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/slave_secrets/templates/try_dot_boto.erb
@@ +1,3 @@
>  [Credentials]
> +aws_access_key_id = <%=scope.function_secret(["releng_try_s3_access_key_id"])%>
> +aws_secret_access_key = <%=scope.function_secret(["releng_try_s3_secret_access_key"])%>

Per IRC convo we're changing the secret name to:

"sccache_s3_storage_access_key_<trustlevel>" /// "sccache_s3_storage_access_id_<trustlevel>" ----> so thatd be, for example: "sccache_s3_storage_access_key_try" // "sccache_s3_storage_access_id_try"

Comment 8

[Justin Wood (:Callek)]

Mike will be e-mailing me the secrets with gpg encryption, I plan to get them added tonight, so he can land this to `default` overnight, and can go live tomorrow with a production puppet push

Comment 9

[Mike Hommey [:glandium]]

Attachment: Switch in house try builds from ceph to reverse-proxied S3

<glandium> Callek: note, access_key/access_id is kind of confusing
<Callek> glandium: better idea?
<glandium> Callek: keeping the s3 terminology of access_key_id/secret_access_key ?
<Callek> glandium: that works for me, please just comment in bug re: changed agreement (since I already commented there)

Comment 10

[Justin Wood (:Callek)]

Comment on attachment 8421452 [details] [diff] [review]
Switch in house try builds from ceph to reverse-proxied S3

Review of attachment 8421452 [details] [diff] [review]:
-----------------------------------------------------------------

sounds good.

As of now I added:

sccache_s3_storage_access_key_id_try: >...
sccache_s3_storage_secret_access_key_try: >...

to heira secrets on the distinguished puppetmaster. feel free to land this on `default` puppet branch. I'm also filing a followup bug for me to remove the now defunct secret from the puppetmaster after a while (e.g. when this bug is done)

Comment 11

[Mike Hommey [:glandium]]

http://hg.mozilla.org/build/puppet/rev/4c708754c7cb

Comment 12

[Mike Hommey [:glandium]]

In production.