1299183 - Expose S3 bucket to Telemetry team [Stage environment]

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect)

Description

[Peter Bengtsson [:peterbe]]

We have an S3 bucket that needs to be exposed/granted to the Telemetry team (led by Mark Reid) so he can point his tools to suck down its content. 

The bucket is located in Oregon ("us-west-2"?) and it's called org-allizom-telemetry-crashes

JP, cc'ed, is the master of our IAM and best to direct technical questions to.

Comment 1

[JP Schneider [:jp]]

Mark, how can I transfer creds to you?  Lastpass?

Comment 2

[Wesley Dawson [:whd]]

:jp for this we want to have a bucket policy that allows access from the dev IAM, something like:

{
	"Version": "2008-10-17",
	"Statement": [
		{
			"Sid": "MozSvcDevAccountAccessList",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::927034868273:root"
			},
			"Action": [
				"S3:ListBucket",
				"S3:GetBucketLocation"
			],
			"Resource": "arn:aws:s3:::__S3_BUCKET__"
		},
		{
			"Sid": "MozSvcDevAccountAccessGet",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::927034868273:root"
			},
			"Action": "S3:Get*",
			"Resource": "arn:aws:s3:::__S3_BUCKET__/*"
		}
	]
}

927034868273 is the correct account number to use.

Once that's in place, :mreid we'll need to update the atmo instance IAM role to also have read access to the same bucket.

Comment 3

[JP Schneider [:jp]]

Fair, we can do it like that.  In socorro-land we've generally granted by IAM group rather than bucket policy, but since we're tying into an existing system, that is much nicer.

Comment 4

[Mark Reid [:mreid]]

Cool, please let me know when this is in effect on the socorro side so I can add the read policy on the telemetry side.

Comment 5

[JP Schneider [:jp]]

I've setup the bucket policy, so we should be set to update the IAM role.

Comment 6

[JP Schneider [:jp]]

Setting needinfo so it flags it for ya.  Thanks!

Comment 7

[GitHub Autolander Bot]

Attachment: [emr-bootstrap-spark] mreid-moz:read_crash_bucket > mozilla:master

Comment 8

[Mark Reid [:mreid]]

r=whd.

Access is working. Example notebook:
https://gist.github.com/mreid-moz/31ac995e3180c156db61e5f1c0ee745b