Open Bug 1663195 Opened 5 years ago Updated 1 year ago

beetmover should be smart about uploading same file to multiple destinations

Categories

(Release Engineering :: Release Automation, enhancement)

Product:
Release Engineering
Component:
Release Automation
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: mtabara, Unassigned)

Details

While digging in bug 1661845 I noticed one improvement that we could make to beetmover.

Within our release automation artifacts, some of these need to be transferred to multiple destinations, especially for the nightly releases, when we want artifacts in both dated dirs and also in the latest counterparts (e.g. here)

Right now, beetmover treats that as a separate upload (here). Instead, we could be using copy_object like we do for push-to-releases logic in here.

Uploading once and using S3 internal copy-ing mechanism could help us:

  • reduce the network bandwidth to upload to S3
  • reduce the time to upload, as S3 native copy-object is faster than an upload
Severity: -- → N/A
QA Contact: mtabara
Component: Release Automation: Uploading → Release Automation
You need to log in before you can comment on or make changes to this bug.