1518930 - (test-efficiencies) [meta] Test Task Efficiencies Project 2019

Status: RESOLVED INACTIVE

(Testing :: General, enhancement, P3)

Description

[Geoff Brown [:gbrown]]

This is a meta-bug to associate bugs related to our 2019 efforts to improve test task efficiency in continuous integration. There is an initial project outline at https://docs.google.com/document/d/1G-gGnIVQ43V8LG9041kMv7t-Rq4UnEDd3k-gdIyhhi4.

This project seeks to reduce inefficiencies in test tasks in continuous integration. These inefficiencies waste compute resources, directly increasing aws and other hosting costs. Indirectly, extra tasks and tests increase CI complexity, turnaround time, and sheriffing overhead.

Focus areas:

1. Reduce redundant or low-value tasks

• Unnecessary platform variants
• Redundant tasks

2. Reduce redundant or low-value tests

• Tests that always fail
• Tests for obsolete features
• Duplicate tests

3. Reduce task overhead

• Can some tasks be combined?
• Improve setup/teardown efficiency

4. Reduce task timeouts

• Ensure appropriate task max-run-time settings

5. Fail faster

• Investigate, resolve on-going hangs

Comment 1

[Geoff Brown [:gbrown]]

:catlee - I understand we have similar interests! I just want to make you aware of this bug. I don't have much time for this in Q1 but am trying to at least collect ideas here.

Comment 2

[Chris AtLee [:catlee]]

(In reply to Geoff Brown [:gbrown] (pto Jan 28-30) from comment #1)

:catlee - I understand we have similar interests! I just want to make you aware of this bug. I don't have much time for this in Q1 but am trying to at least collect ideas here.

Thanks for the headsup!

Comment 3

[Geoff Brown [:gbrown]]

This project has run its course. We made some good improvements here, but didn't have nearly enough time to follow-up on all of our ideas. Let's leave remaining dependencies open and hope to resolve them in the future.

Comment 4

[Edwin Takahashi (:egao | infrequent contributor)]

I'd like to suggest making a Task Efficiencies 2020 project and moving all the open dependencies onto that project. I believe it would be in line with our CI-A team goals for 2020 since Task Efficiencies is listed as one of the secondary focus item.