Closed Bug 1100624 Opened 6 years ago Closed 3 years ago

Upload build artifacts to s3 instead of ftp

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mshal, Assigned: mshal)

References

(Blocks 1 open bug)

Details

Maybe somewhat similar to bug 707843, but we're looking at uploading build artifacts to s3 using the taskcluster interfaces rather than uploading to ftp.

When I spoke to jlal about this, he produced the following outline:

***
From mozharness the workflow would be like this:

    job starts
    create task
    claim task you just created
    [keep reclaiming task]
    attach artifacts to task
    mark task as completed success/fail
    job ends

Credentials can be embedded and a custom "buildbot" worker type and provisioner can be used.

The benefits to using the full task creation process are:

    we get more timing information
    more (coherent) pulse messages from buildbot using TC
    indexing support out of the box
    full load test of taskcluster web services
***

This bug can track the work related to it - it will probably involve build config changes, mozharness changes, deploying some of the taskcluster code to buildbot machines, and fixing whatever currently depends on things being in ftp.

Note that we also upload symbols via 'make uploadsymbols', but I'm not sure if that will be changed as part of this bug as well or will remain as it is.
Symbols is covered by bug 1071724, going to S3 too.
Assignee: nobody → mshal
Depends on: 1109136
Depends on: 1109143
Depends on: 1109147
Depends on: 1108029
Depends on: 1112252
Depends on: 1112303
Blocks: 1117960
Depends on: 1119500
Depends on: 1132123
Depends on: 1132151
Depends on: 1135250
Depends on: 1136822
Depends on: 1145593
Depends on: 1155349
Depends on: 1160410
Depends on: 1167768
Depends on: 1174241
What is the status in here?
Are all deps still blocking this bug?

Thanks!
Depends on: 1195865
Depends on: 1195870
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #2)
> What is the status in here?
> Are all deps still blocking this bug?

I believe so. I just filed two others for the pieces of android builds that are missing.
I think everything here has long since been fixed. The lone open bug is a treeherder issue that I think was solved separately by displaying all the artifacts for the task.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.