1266133 - [jenkins] how to add post-build step to publish test logs to S3 bucket

Status: VERIFIED FIXED

(Cloud Services :: QA: Test Automation, defect)

Description

[Richard Pappalardo [:rpapa][:rpappalardo]]

For treeherder integration, we need a means to publish logs to S3.  We already have S3 plugins installed on Jenkins via our puppet scripts.  It appears that this should be easily accomplished w/ that plugin, as long as permissions are correct on Amazon, and a remote bucket profile is properly configured in Manage Jenkins.


We'll be using something like:
services-qa-jenkins-artifacts/test-logs/<job-name>

The purpose of this bug is to outline steps needed to do this.

Comment 1

[Richard Pappalardo [:rpapa][:rpappalardo]]

HOW TO PUBLISH LOGS TO S3
[1]. Jenkins > Configuration > Amazon S3 Profiles
[2]. Add a profile name (can be anything you want)
[3]. click on checkbox: "Use IAM Role"
     NOTE: If ticked uses the instance IAM role to access S3 - so make sure to add S3 access to that role
[4]. Save
[5]. Create a new job 
[6]. Add a build step that creates some artifact (i.e. dummy.log)
[7]. Add a post-build step: "Publish Artifacts to S3 bucket"
[8]. Save and run job
[9]. logs will be output to bucket in this format:
<bucket-name>/jobs/<job-name>/<run-number>/<log-name>
https://s3-us-west-1.amazonaws.com/services-qa-jenkins-artifacts/jobs/demo-s3/2/dummy.txt

NOTE:
These are the steps needed to publish to S3, but won't be needed by folks writing tests.
These will eventually be baked into jobs by the Jenkins job generator, but can be added manually for now, if needed.
Just documenting here for reference.