1637679 - Migrate arewefastyet.com from mozilla-central to autoland

Status: RESOLVED FIXED

(Testing :: Raptor, task, P2)

Description

[Greg Mierzwinski [:sparky]]

We recently had an idea for moving the dashboards from using mozilla-central data to autoland data so that we can disable tests that we run on mozilla-central which only exist for dashboards atm and aren't sheriffed. I think we should also be clear that this would only be done for tests that currently run on both branches and not the tests that are only running on mozilla-central (like our cron tests). We get the same data from both and perf regressions generally make it into mozilla-central unless they are backed out for some other reasons.

There are two questions we need to look into here:

1. Do the savings of disabling perftests on mozilla-central outweigh the cost to disabling them and switching the dashboards?
2. Given that autoland has more granularity, would the dashboards be able to handle the added data and still provide the same performance?

:jmaher, what do you think about question (1), do you think you would be able to provide some potential savings estimates? We think this could be a potential CI costs savings but maybe it won't be as impactful as we suspect.

Answering (2) will depend on the answers to (1).

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I think this is a great idea and we would automatically get alerts matched up.

As for #2, let me ask :ekyle.

Comment 2

[Kyle Lahnakoski [:ekyle]]

Given that autoland has more granularity, would the dashboards be able to handle the added data and still provide the same performance?

The health graphics dashboard may be fine. It can be tested easy by making a PR with all "mozilla-central" replaced with "autoland", and reviewing the preview. HERE IS THE PREVIEW

We can see there are a number of tests not run on autoland.

Comment 3

[Greg Mierzwinski [:sparky]]

:ekyle, that's expected since we don't run everything on both branches. This would only be for tests that currently run on both autoland and m-c.

For these tests, do you think we could start moving the dashboards to use autoland?

Comment 4

[Kyle Lahnakoski [:ekyle]]

:sparky yes, go ahead and turn off mozilla-central tests. Please add your changes as an issue to https://github.com/mozilla-frontend-infra/firefox-health-dashboard/issues

Comment 5

[Greg Mierzwinski [:sparky]]

Fantastic, thanks!

Comment 6

[Greg Mierzwinski [:sparky]]

Removing the discussion tag from this bug.

Comment 7

[Greg Mierzwinski [:sparky]]

https://github.com/mozilla-frontend-infra/firefox-health-dashboard/issues/691

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

what is the timeline for working on this bug?

Comment 9

[Greg Mierzwinski [:sparky]]

:jmaher, we're going to work on this in our june sprint. Here's the JIRA ticket: https://jira.mozilla.com/browse/FXP-1018

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Only arewefastyet.com has been migrated to use autoland where possible. We still need to consider the value of performing the same migration for health.graphics.