823016 - Make the dev.tree-management Talos email subjects easier to scan

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ed Morley [:emorley]]

I find the current subjects hard to scan quickly:

Talos Regression :( Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) increase 0.995% on Win7 Mozilla-Inbound

Talos Regression :( Trace Malloc Leaks increase 0.379% on MacOSX 10.7 Mozilla-Inbound

etc

We should:
* Put the tree name earlier in the string
* Use separators to more clearly break up the testname/platform/result

Comment 1

[Ed Morley [:emorley]]

Attachment: Patch v1

(In reply to Ed Morley [UTC+0; email:edmorley@moco] from comment #0)
> Talos Regression :( Tp5 No Network Row Major MozAfterPaint (Main Startup
> File IO Bytes) increase 0.995% on Win7 Mozilla-Inbound

This becomes...

Talos Regression: Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%

Also adjusted the email body copy to match.

Comment 2

[Chris AtLee [:catlee]]

Comment on attachment 693827 [details] [diff] [review]
Patch v1

Review of attachment 693827 [details] [diff] [review]:
-----------------------------------------------------------------

::: server/analysis/analyze_talos.py
@@ +579,2 @@
>          else:
> +            reason = "Regression"

tempted to r- just for this! :P

you don't like the emoticons?

@@ +581,5 @@
>  
>          if new_value > initial_value:
>              direction = "increase"
>          else:
>              direction = "decrease"

get rid of this block if you're not going to use 'direction'

Comment 3

[Jeff Hammel]

While we're at it, I wouldn't mind taking out the Talos from the subject line since not all of the emails are Talos regressions.  I've had to explain several times in the last year that "Number of constructors" is not actually a Talos test (and then they would logically ask: "Then why does the email say Talos?")

Comment 4

[Ed Morley [:emorley]]

Agree about dropping talos. Also means less of the subject is potentially wrapped/pushed off-screen.

Taking into account the comment in bug 823008, how about (imagine one were skim reading):

<Regression> Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%
(Improvement) Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%
(Improvement) Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%
<Regression> Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%
<Regression> Mozilla-Inbound - Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes) - Win - 0.995%

Re the emoticon, I like it, but it just didn't play nicely with the various separators I tried.

Comment 5

[Geoff Brown [:gbrown] (pto Apr 11-18)]

That works for me -- definite improvement over the existing format.

Comment 6

[Jeff Hammel]

lgtm; personally i would put the % earlier, but not a strong opinion.  Hopefully going forward we can get these test names tamed a bit too (but that's a different issue entirely)

Comment 7

[Ed Morley [:emorley]]

Attachment: Patch v2

(Patch as landed)

Thank you for the feedback :-) Regarding % earlier on in the string - with bug 822249 I hope that we'll have less irrelevant mail, so think it will be ok to favour grouping in a more logical manner vs seeing the % sooner. We can always adjust later on if it turns out to be annoying.

https://hg.mozilla.org/graphs/rev/7c9a79bc41d7

Comment 8

[Jeff Hammel]

To be fair, with datazilla in the pipeline we're going to have to change how this works across the board, so any improvements we do here are interim.

Comment 9

[Ed Morley [:emorley]]

(In reply to Jeff Hammel [:jhammel] from comment #8)
> To be fair, with datazilla in the pipeline we're going to have to change how
> this works across the board, so any improvements we do here are interim.

Agreed, but I imagine we'll be running in parallel for a bit, so the more tolerable the better :-)

catlee, how do we deploy the changes? :-)

Comment 10

[Chris AtLee [:catlee]]

(In reply to Ed Morley (Away 20th Dec-2nd Jan) [UTC+0; email:edmorley@moco] from comment #9)
> (In reply to Jeff Hammel [:jhammel] from comment #8)
> > To be fair, with datazilla in the pipeline we're going to have to change how
> > this works across the board, so any improvements we do here are interim.
> 
> Agreed, but I imagine we'll be running in parallel for a bit, so the more
> tolerable the better :-)
> 
> catlee, how do we deploy the changes? :-)

poke me tomorrow!

Comment 11

[Ed Morley [:emorley]]

In production

Comment 12

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/b0436d0f3001b7f50a46dd5c9bb24baaf428cd2b
Bug 823016 - Make the dev.tree-management Talos email subjects easier to scan; r=catlee