Closed Bug 1399393 Opened 7 years ago Closed 7 years ago

Task submission from decision task is very slow for nightly graphs

Categories

(Taskcluster :: General, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla57

People

(Reporter: sfraser, Assigned: sfraser)

References

Details

Attachments

(1 file)

https://bugzilla.mozilla.org/show_bug.cgi?id=1342392 caused the decision task to exceed its maxRunTime of 30 minutes, and 29m12s of that runtime was task submission. 

Looking through the recent past, nightly decision tasks seem to take 18 to 23 minutes. The task graph generation is less than a minute, the rest of the time are create task calls. This doesn't affect non-nightly decision tasks, so we suspect this is due to the task ordering during submission, and that it blocks the entire submission when waiting for a task's dependencies to complete:

https://dxr.mozilla.org/mozilla-central/rev/f9a5e9ed62103c84e4cde915f4d08f1ce71be83e/taskcluster/taskgraph/create.py#81-82

We have a proposal for refactoring create_tasks, which takes the decision task runtime on 'date' down from ~18 minutes to ~3 minutes.
Assignee: nobody → sfraser
Blocks: 1342392
Comment on attachment 8907487 [details]
Bug 1399393 Refactor create_tasks to avoid bottlenecks

https://reviewboard.mozilla.org/r/179184/#review184522

<3
Attachment #8907487 - Flags: review?(dustin) → review+
Pushed by sfraser@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/45fe1c050645
Refactor create_tasks to avoid bottlenecks r=dustin
https://hg.mozilla.org/mozilla-central/rev/45fe1c050645
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in before you can comment on or make changes to this bug.