1160380 - (beetmover) S3 bit mover

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Create a job which moves files from the TaskCluster artifact store to a candidates bucket in S3, applying pretty names and paths if necessary, and performing an antivirus scan. Run this after each job that produces bits, e.g. en-US builds, l10n repacks, partial update generation. Investigate security models which limit write access to only the bit mover, and only to the current 'release'. Implement bucket versioning and logging to provide history for audit.

Comment 1

[Rail Aliiev [:rail]]

We can also perform virus scanning as a part of this job.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

:rail, I think you've been working on this for funsize, is that work more generally too ?

Comment 3

[Rail Aliiev [:rail]]

The work is not generic, it's very limited to funsize (not flexible at all). The code is very small and can be replaced by something more generic in the future. I just didn't want to be blocked on something generic.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

From my talk with Rail about secrets yesterday, if we could find a way to transmit a signed S3 PUT URL to this task, securely, that would be cool.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Not working on release promotion at the moment, releasing the bug.

Comment 6

[Chris Cooper [:coop] (he/him)]

Whiteboards from Toronto sessions:

https://www.dropbox.com/s/ss8hag48ip0pll1/2015-11-17%2010.42.31.jpg?dl=0
https://www.dropbox.com/s/q040m3qpo0wshu7/2015-11-17%2010.56.36.jpg?dl=0
https://www.dropbox.com/s/iab35dnewx23e8k/2015-11-17%2011.27.44.jpg?dl=0

Comment 7

[Jordan Lund (:jlund)]

it's mine now!

Comment 8

[Jordan Lund (:jlund)]

I think we are all done here for now..