1328618 - SETA - Fix all production issues

Status: RESOLVED FIXED

(Tree Management Graveyard :: Treeherder: SETA, defect)

Description

[Armen [:armenzg]]

Attachment: Output from jobs with priority 1

After the following change:
https://github.com/mozilla/treeherder/commit/7145c4e2053992b7dbccc59ddb2a9b93189b0869

We only keep jobs considered as high value jobs.
Jobs that are not considered high value get their priority decreased.

In the attachment below we see "Windows 7 VM 32-bit mozilla-inbound debug test mochitest-browser-chrome-7" as the high value job while the talos jobs are added becaused of load_preseed.

Running analyze_failure after load_preseed does not change the priority of the jobs that were added by load_preseed since they have an expiration date on 2100.

Comment 1

[Armen [:armenzg]]

What I'm currently I'm aware that needs fixing:
* It seems that the TaskCluster task labels have recently been renamed
  * I need to fix string matching and probably some fixtures
* I fixed a typo introduced during the refactor/feedback flow

Comment 2

[GitHub Autolander Bot]

Attachment: [treeherder] armenzg:seta_follow_up > mozilla:master

Comment 3

[Armen [:armenzg]]

Comment on attachment 8824498 [details] [review]
[treeherder] armenzg:seta_follow_up > mozilla:master

Hi Joel,
This PR is ready for your feedback.

I will be asking for Ed's formal review once I have the test fixtures added.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8824498 [details] [review]
[treeherder] armenzg:seta_follow_up > mozilla:master

this is a nice refactor, most of this is just simple changes with a small bit of actual logic change.

Comment 5

[Armen [:armenzg]]

Attachment: [commit] TaskCluster jobs define their task labels as job type name

Hi camd, I believe you created the pulse consumer sample data and I would like to know if you're OK with these changes.

I just want to store some TaskCluster jobs that are a bit closer to reality. I have not updated all fields (e.g. moving from Linux job to Windows job) because that information is not necessary for the tests I'm writing.

Are you OK with whoever tries to write more accurate tests in the future can adjust the tasks?

I'm not going through the trouble of updating all fields manually because I don't know of an easy eay to take the metadata of a TaskCluster job in production to the sample data I'm touching. I've looked at the jobs api, however, the data structure is not the same.

Comment 6

[Armen [:armenzg]]

Comment on attachment 8824498 [details] [review]
[treeherder] armenzg:seta_follow_up > mozilla:master

Hi Ed, I'm mainly asking for review because I'm touching fixtures and load_initial_data.py

The rest of the changes are business logic.

Please let me know if you want me to adjust the commit messages.
Do we normally add "bug XXXXXX -" to every single commit we add to Treeherder? I just want to know the norm and keep using that.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 8824498 [details] [review]
[treeherder] armenzg:seta_follow_up > mozilla:master

If the PR doesn't have a bug at all, and is just for something trivial like a docs typo change then I'm fine with no bug number (just ignore the bot commenting on the PR; there's no way to suppress it with a "(no bug)" type string yet.

If there is a bug around though, then please put it in each micro-commit (helps with archaeology for git blame users later, plus ensures the bot posts the commit link and full commit message in the bug later).

Comment 8

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/3fb5bc6c2d9604eed7582058d4f751c8f708849c
Bug 1328618 - Make SETA adjust job priorities in both directions

Up until now, job priorities could only be increased to high value jobs.
This change allows resetting a job to a low value priority if it has not
been determined to be a high value job.

This change also fixes a race condition between load_preseed and
analyze_failures where we would attempt to update the job priority of a
job that is not yet in the database.

https://github.com/mozilla/treeherder/commit/656372b593d67aa274d039cffaae8862d9febc86
Bug 1328618 - Fix typo

https://github.com/mozilla/treeherder/commit/a71106395aa14b4684b74fc37718847457042998
Bug 1328618 - Use new tasklabels naming for test jobs

https://github.com/mozilla/treeherder/commit/1a6ef47acfa94af3840c3a43cd6e8266e995dca6
Bug 1328618 - We should populate runnable jobs as soon as Treeherder starts

This ensures that tools that rely on runnable jobs having data can work
as expected from the beginning.

https://github.com/mozilla/treeherder/commit/cb0975c47dfc0255fed833c58c2c4208ca002521
Bug 1328618 - Calling job priorities api without a priority returns all job priorities

https://github.com/mozilla/treeherder/commit/62734d5914adbe426d6c22e69df551fe1f9a20e5
Bug 1328618 - Fix return all job priorities for all build systems

Not specifying the build_system_type query parameter when calling the
job priorities API would only return TaskCluster jobs and exclude
Buildbot jobs.

This change fixes the issue.

https://github.com/mozilla/treeherder/commit/ea7d4fe4f15c04d931600af6d0d10005304c52e2
Bug 1328618 - Ignore jobs that SETA cannot parse

There are jobs in production that cannot be parsed by parse_testtype().
We will not add these jobs to the list of jobs fixed by commit but log
them for further investigation.
The initital suspition is that they are not test jobs but other kinds.

https://github.com/mozilla/treeherder/commit/edc3d7ad112c7c60e341fab7c1485c0e41408036
Bug 1328618 - TaskCluster jobs define their task labels as job type name

ref_data_name is a calculated value rather than a value defined by the
TaskCluster task. If ref_data_name cannot be calculated it fallsback to
a signature. Switching to job type name avois this issue.

Also add TaskCluster jobs to the list of jobs with notes

Comment 9

[Armen [:armenzg]]

I'm working on verifying if we're ready to switch Buildbot over.
TaskCluster needs more work on the in-tree side and rwood might be taking that over.
Any new issues will be filed as new bugs.

Comment 10

[Cameron Dawson [:camd]]

Armen--  Dang, sorry for my late (and now pretty irrelevant) reply.  But those changes are totally fine by me, fwiw.