Upload build artifacts to s3 instead of ftp

RESOLVED FIXED

Status

RESOLVED FIXED
4 years ago
3 months ago

People

(Reporter: mshal, Assigned: mshal)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

4 years ago
Maybe somewhat similar to bug 707843, but we're looking at uploading build artifacts to s3 using the taskcluster interfaces rather than uploading to ftp.

When I spoke to jlal about this, he produced the following outline:

***
From mozharness the workflow would be like this:

    job starts
    create task
    claim task you just created
    [keep reclaiming task]
    attach artifacts to task
    mark task as completed success/fail
    job ends

Credentials can be embedded and a custom "buildbot" worker type and provisioner can be used.

The benefits to using the full task creation process are:

    we get more timing information
    more (coherent) pulse messages from buildbot using TC
    indexing support out of the box
    full load test of taskcluster web services
***

This bug can track the work related to it - it will probably involve build config changes, mozharness changes, deploying some of the taskcluster code to buildbot machines, and fixing whatever currently depends on things being in ftp.

Note that we also upload symbols via 'make uploadsymbols', but I'm not sure if that will be changed as part of this bug as well or will remain as it is.
Symbols is covered by bug 1071724, going to S3 too.
(Assignee)

Updated

4 years ago
Assignee: nobody → mshal
(Assignee)

Updated

4 years ago
Depends on: 1109136
(Assignee)

Updated

4 years ago
Depends on: 1109143
(Assignee)

Updated

4 years ago
Depends on: 1109147
(Assignee)

Updated

4 years ago
Depends on: 1108029
(Assignee)

Updated

4 years ago
Depends on: 1112252
(Assignee)

Updated

4 years ago
Depends on: 1112303

Updated

4 years ago
Blocks: 1117960
(Assignee)

Updated

4 years ago
Depends on: 1119500
(Assignee)

Updated

4 years ago
Depends on: 1132123
Depends on: 1132151
(Assignee)

Updated

4 years ago
Depends on: 1135250
Depends on: 1136822
(Assignee)

Updated

3 years ago
Depends on: 1145593
Depends on: 1147140
(Assignee)

Updated

3 years ago
Depends on: 1155349
Depends on: 1160410
Depends on: 1167768
(Assignee)

Updated

3 years ago
Depends on: 1174241

Comment 2

3 years ago
What is the status in here?
Are all deps still blocking this bug?

Thanks!
(Assignee)

Updated

3 years ago
Depends on: 1195865
(Assignee)

Updated

3 years ago
Depends on: 1195870
(Assignee)

Comment 3

3 years ago
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #2)
> What is the status in here?
> Are all deps still blocking this bug?

I believe so. I just filed two others for the pieces of android builds that are missing.
(Assignee)

Comment 4

a year ago
I think everything here has long since been fixed. The lone open bug is a treeherder issue that I think was solved separately by displaying all the artifacts for the task.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.