1621472 - SETA Low Value Job Priorities for large numbers of jobs lost during SETA update.

Status: RESOLVED FIXED

(Testing :: General, defect, P1)

Description

[Bob Clary [:bc] (inactive)]

Attachment: job-priorities-master-branch.diff

Bug 1608427 introduced a change to how SETA is handled in Treeherder in order to account for fuzzing builds as low value tasks. It appears to have resulted in a change in the production JobPriority table which has resulted in a large number of tasks no longer being considered as low value. In particular, all of the android-hw raptor tests have begun running on each push on autoland resulting in a very large backlog.

I downloaded the job-priorities json for priority 5 from my local treeherder instance using the earlier version of treeherder and compared it to the version after my pull request.

The difference includes the expected iris-, source-test-and web-render changes in addition to the desired fuzzing build change. See job-priorities-master-branch.diff attachment.

Comment 1

[Bob Clary [:bc] (inactive)]

Attachment: job-priorities-master-treeherder.diff

Comparing the version I obtained from my master earlier to the version I just downloaded from treeherder I see a number of differences. See job-priorities-master-treeherder.diff attachment.

Comment 2

[Bob Clary [:bc] (inactive)]

I just pulled the tip from treeherder and ran it locally and compared the job-priorities that it generated and it agrees with the values I obtained from production treeherder.

armengz, camd: Do you know why there is this difference? Did we lose historical data in the production JobPriorities database as a result of my pull request? Would reverting my pull request fix this?

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: job-priorities-treeherder-reverted.diff

I reverted my patch locally and generated the job priorities again. I compared it to the job priorities from treeherder in job-priorities-treeherder-reverted.diff. This restored the missing jobs.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I recommend reverting:
https://hg.mozilla.org/mozilla-central/rev/fa1dd2a86d113c462f84d1891ec50ae109f0df8d

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/6129

Comment 6

[Bob Clary [:bc] (inactive)]

Let's just revert Bug 1608427 and pick it up again when we are fresh.

Comment 7

[Bob Clary [:bc] (inactive)]

camd deployed my reversion of the original patch to treeherder. It does look like the raptor jobs stopped populated at https://treeherder.mozilla.org/#/jobs?repo=autoland&tier=1%2C2%2C3&searchStr=android-hw&revision=884162af76f5225bbf4efe486959d2fa9757bc56 at Tue, Mar 10, 17:09:14 .

I am closing this bug as fixed and will reopen Bug 1608427.