1496319 - Comments from Intermittent Failures Robot should also include build type (opt, debug, asan)

Status: VERIFIED FIXED

(Tree Management :: Treeherder, enhancement, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I find me a lot in the situation that the comments as added by the robot do not include all the necessary information to quickly get an overview about the failures as happened. Therefore I manually have to click the link to see the results in the Viewer. Mostly this is simply to check for the build type.

As such it would be kinda helpful if the comments as being made by the robot would also include this information beside the branch, and platform details.

Comment 1

[Sarah Clements [:sclements]]

So what you'd like is something like this?

Repository breakdown:
* mozilla-central: 33
* try: 11
* mozilla-inbound: 50
* autoland: 53

Platform breakdown:
* windows10-64: 52
* osx-10-10: 1
* windows7-32: 73
* windows10-64-msvc: 11
* windows7-32-msvc: 10

Build type breakdown:
* debug
* opt

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Yes, that is what I would have in mind. Joel, what do you think about it?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I think that is ok, possibly more:
Config breakdown:
* windows10-64: 52
** debug: 36
** opt: 1
** ccov: 10
** asan: 3
** nightly: 2
* osx-10-10: 1
** debug: 1
...

it might be harder to parse my proposal, but it wouldn't require loading up IFV to make a decision if there is a pattern or trend.  I would be happy with the original proposal as that would give much more information and we would know if it is wide spread or not, I just feel that for information to be useful we want to know what configs are the most problems occuring on and if there is a pattern we can investigate or disable.

Comment 4

[Sarah Clements [:sclements]]

I won't be able to get to this until early next year so if anyone wants to work on this in the meantime, here's where you'd need to make changes:

https://github.com/mozilla/treeherder/blob/master/treeherder/intermittents_commenter/commenter.py#L255
https://github.com/mozilla/treeherder/blob/master/treeherder/intermittents_commenter/commenter.py#L90
https://github.com/mozilla/treeherder/blob/master/treeherder/intermittents_commenter/comment.template

And the Job model (BugJobMap has a foreignkey on Job) for determining how to access the build type field.
https://github.com/mozilla/treeherder/blob/master/treeherder/model/models.py#L470

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sarah, it's been a while since the last update on this bug. With having confusion while observing those comments (see bug 1636885) it would be great to have this feature. Maybe you have an update for us?

Comment 6

[Sarah Clements [:sclements]]

I forgot about this. Sorry :) Someone else mentioned this same thing during a UX research project we're doing. I can probably work on it by the end of the quarter, unless something pressing comes up.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sarah, do you have an update for us? Thanks.

Comment 8

[Sarah Clements [:sclements]]

Yes, not until end of Q3 or early Q4. I know you've been waiting a while for this feature... it's in the queue.

Comment 9

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/6961

Comment 10

[Sarah Clements [:sclements]]

This has been merged and the changes should be reflected by end of the week: https://github.com/mozilla/treeherder/commit/a909ce1e409d1d56a81a76460bb0bf4c2c6aa368

The breakdown will look like what Joel suggested in comment #3. Sorry this took so long!

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sarah, thanks a lot for getting this landed. It's so helpful!

Comment 12

[Sarah Clements [:sclements]]

You're welcome! :D