1216080 - It should not needed to re-save a existing rule to get them applied

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder, defect)

Description

[Carsten Book [:Tomcat]]

We had last week 2 situations where tests that where hidden showed up in treeherder despite beiing hidden.

I was told:

RyanVMTomcat|sheriffduty: if it's the first time they reported to TH, they wouldn't until you re-save the rule 

14:57philorTomcat|sheriffduty: they weren't running before, so you have to resave the rule without making any changes, to have it apply now that they exist 

It should not needed to resave a rule and treeheerder should just apply rules that are there. The Problem on the re-save is that if you not remember that tests you might loose time and so resources to search while a test show up suddenly (might also be the cause for https://bugzilla.mozilla.org/show_bug.cgi?id=1214185#c1 )

Comment 1

[Cameron Dawson [:camd]]

Yeah, I can sure see how this is annoying.  Please add to the sheriff top-5 annoying if you'd like it to be looked at sooner rather than later.

This is, unfortunately, a bit of a chicken-and-the-egg thing where we can't hide a job till we know its signature.  But we won't know its signature until we've ingested it once.

We could create a process that runs in the background to re-generate the signature list every X minutes/hours.

Comment 2

[Phil Ringnalda (:philor)]

With the freedom of having absolutely no idea how things actually work, I would think that ingesting a job with a new signature should call for looking at whether or not that signature is something that should be hidden.

Comment 3

[Phil Ringnalda (:philor)]

I think our exclusion for gtest has a few things that don't actually exist (because the lists of platforms and jobs are absolutely insanely full of things that don't actually exist), but if it didn't, 18 branches, 17 trees, 5 flavors, that'd be 1,530 times resaving it after the jobs finally ran, including needing to do our own push to try to trigger the semi-hidden OS X 10.10 and Windows 10 variants (which may well still be visible right now). So, yeah, we ought to top-5 it.

Comment 4

[Ed Morley [:emorley]]

We're going to replace exclusion profiles with people always using the in-tree (mozilla-central) tier values instead (bug 1387640).