1468943 - Mark confirmed alerts automatically

Status: RESOLVED WONTFIX

(Tree Management :: Perfherder, enhancement, P5)

Comment 1

[Ionuț Goldan [:igoldan]]

Assuming we retriggered/backfilled an alert and then marked it as "confirming", Perfherder should know when to change its state to "confirmed", once all the jobs finish.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973

Comment 3

[Ionuț Goldan [:igoldan]]

Comment on attachment 9005194 [details] [review]
Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973

I know this is WIP, but I could use your feedback on this, as I'm not really familiar with Celery's way of doing things or New Relic's agent.

Comment 4

[William Lachance (:wlach)]

Comment on attachment 9005194 [details] [review]
Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973

Left a bit of feedback in the review on some of the details, I'd like to know more about exactly the workflow we're going for / trying to automate. Should we consider automating the confirming process as well?

Comment 5

[Ionuț Goldan [:igoldan]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #4)
> Comment on attachment 9005194 [details] [review]
> Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973
> 
> Left a bit of feedback in the review on some of the details, I'd like to
> know more about exactly the workflow we're going for / trying to automate.

The workflow is as follows: when a new alert summary lands, we star a subset with the most relevant alerts. Then we retrigger & backfill them, while also marking them as 'confirming'. After this point, I only know which alerts have been retriggered/backfilled, but not if their jobs finished or not; I need to manually check that.
In this bug, I want to automate only this final check. This way, if all jobs have finished, then the corresponding alert is marked as 'confirmed'.

> Should we consider automating the confirming process as well?

This should be treated as a separate bug. We would need to talk with Joel about this. I recon he has some concerns about automating retriggers & backfills.

Comment 6

[Ionuț Goldan [:igoldan]]

Comment on attachment 9005194 [details] [review]
Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973

I addressed the review requests and tested only the check_confirming_perf_alerts_status() function locally (not the whole Celery & worker & queue setup) with success.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 9005194 [details] [review]
Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3973

Left a comment :-)

Comment 8

[Ionuț Goldan [:igoldan]]

Closed as WONTFIX because we want to get rid of this status. See bug 1582108.