1516630 - Delete mentionings of old frameworks that are not used

Status: RESOLVED FIXED

(Tree Management :: Perfherder, enhancement, P2)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

we have many old frameworks in perfherder:
* talos-aws (no alerts)
* hasal (no alerts)
* servo-perf (no alerts)

old frameworks that have alerts:
* autophone (old alerts)
* awfy (old alerts)

to resolve:
* platform_microbenchmarks (active alerts, but nobody uses it)

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:igoldan now that bug 1346567 is resolved, would we expect frameworks without results to be cleaned up after our retention period has elapsed?

Comment 2

[Ionuț Goldan [:igoldan]]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #1)

:igoldan now that bug 1346567 is resolved, would we expect frameworks without results to be cleaned up after our retention period has elapsed?

Not really. This is a registry list hardcoded in the database itself. Data cycling won't simply expire those frameworks.
These frameworks are registered & unregistered by hand, via a database migration script.

Comment 3

[Ionuț Goldan [:igoldan]]

(In reply to Joel Maher ( :jmaher ) (UTC-4) (on leave, back October 1st) from comment #0)

we have many old frameworks in perfherder:

• talos-aws (no alerts)
• hasal (no alerts)
• servo-perf (no alerts)

Indeed, these must be deleted entirely. There's no doubt.

old frameworks that have alerts:

• autophone (old alerts)
• awfy (old alerts)

We filed some Autophone bugs 1.5 years ago. We shouldn't have problems in deleting Autophone data.
Regarding awfy: Dave, can you confirm we can delete this? I want to make sure there aren't plans with re enabling this kind of perf data.

to resolve:

• platform_microbenchmarks (active alerts, but nobody uses it)

Perf data is still produced for this framework. But we need a decision. In the past, these were the most unstable tests we sheriffed & and we didn't gain almost anything from them. Maybe things have changed, but this needs some investigation. I'd postpone the deletion of this framework until we recheck its data quality.

Comment 4

[Ionuț Goldan [:igoldan]]

Implementation wise, I see 2 main parts for this ticket.

Delete the frameworks from the performance_framework table, using a migration script.

Then make sure you delete the fixtures that can add those undesired frameworks back.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriff from comment #3)

(In reply to Joel Maher ( :jmaher ) (UTC-4) (on leave, back October 1st) from comment #0)

we have many old frameworks in perfherder:

• talos-aws (no alerts)
• hasal (no alerts)
• servo-perf (no alerts)

Indeed, these must be deleted entirely. There's no doubt.

old frameworks that have alerts:

• autophone (old alerts)
• awfy (old alerts)

We filed some Autophone bugs 1.5 years ago. We shouldn't have problems in deleting Autophone data.
Regarding awfy: Dave, can you confirm we can delete this? I want to make sure there aren't plans with re enabling this kind of perf data.

When was the last awfy data injested? I'm not aware of anything currently running here as I believe this was all migrated to Raptor via bug 1445952. We need to review the remaining blockers for that bug, but they don't appear to be related to moving tests away from the awfy harness.

to resolve:

• platform_microbenchmarks (active alerts, but nobody uses it)

Perf data is still produced for this framework. But we need a decision. In the past, these were the most unstable tests we sheriffed & and we didn't gain almost anything from them. Maybe things have changed, but this needs some investigation. I'd postpone the deletion of this framework until we recheck its data quality.

Let's split this out into a separate bug to review the value of these tests.

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/5382