Closed Bug 1524845 Opened 3 years ago Closed 3 years ago

Download label-to-taskId's in parallel

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(firefox67 fixed)

RESOLVED FIXED
mozilla67
Tracking Status
firefox67 --- fixed

People

(Reporter: dustin, Assigned: dustin)

Details

Attachments

(1 file)

https://tools.taskcluster.net/groups/exSDA0XCSgiBDbSGEn3T5w/tasks/Q5SzttalT-aS4Wnq-tnKkA/runs/0/logs/public%2Flogs%2Flive.log

...
[task 2019-02-03T12:45:42.059Z] fetching label-to-taskid.json for action task O3PiCoeUSzO1BIOJjk6ljw
[task 2019-02-03T12:45:42.487Z] fetching label-to-taskid.json for action task Q1Wxa9EkRCWezQcRB3nejA
[task 2019-02-03T12:45:42.855Z] fetching label-to-taskid.json for action task PODQp2v0TDu_FEZr_AM5Pg
[task 2019-02-03T12:46:01.365Z] fetching label-to-taskid.json for action task cFGuSW8HQuS5HQM7ntB0Gg
[task 2019-02-03T12:46:21.189Z] fetching label-to-taskid.json for action task NY00v69wSY6NBvxcVQ2mEQ
[task 2019-02-03T12:46:41.083Z] fetching label-to-taskid.json for action task EAIcB-1QQpS_t24RSF_WRg
[task 2019-02-03T12:47:00.956Z] fetching label-to-taskid.json for action task GnUAsx_WRc-p9KEYqPT2xQ
[task 2019-02-03T12:47:21.882Z] fetching label-to-taskid.json for action task c6UpHi3bQleoZqYyt8S71w
[task 2019-02-03T12:47:39.379Z] fetching label-to-taskid.json for action task NPN6bn5-SPWOsT1EL_s5wg
...

this task is trying to download a lot of label-to-taskid.json files, and as the first thing to download them is being forced to wait for a copy to the correct AWS region. We can get much better performance by running all of these downloads in parallel.

Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/45d561eacd00
download data about previous crons, actions in parallel r=aki

Joel, Sebastian -- this should help actions on pushes with lots of action tasks to succeed and run more quickly. Let me know if you see issues (and feel free to link this into the dependency tree of related bugs).

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
You need to log in before you can comment on or make changes to this bug.