1434360 - Have a dedicated dashboard for talos tp6

Status: RESOLVED FIXED

(Testing :: Talos, enhancement, P1)

Description

[Robert Wood [:rwood]]

It would be really cool to have a dedicated dashboard for talos tp6, similar to what the devtools team has:

https://health.graphics/devtools

This would be a faster way of viewing the current results of the various tp6 pages (google, facebook, etc) and also once we support multiple measurements per pageload, all of the measurements could be displayed on the same dashboard.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

right now tp6 posts 4 different tests and we triage them the same way:
* google
* facebook
* amazon
* youtube

right now these measure a single value- our goal is for each to measure 4 values- so this would mean in our current model to support 16 different series to track (and then multiplied 10x for the different configs we run on).

As you can imagine that is a lot of data to track and possibly confuse developers when looking at a report.

We will figure out how to report the data values in talos so they exist fully in treeherder.  I imagine as subtests- either:
* tp6_google:
** metric1
** metric2
** metric3
** metric4

or a flip:
* metric1
** google
** amazon
** facebook
** youtube

likewise we might want to derive additional information, metric3-metric2 over time, etc.

Going with a custom dashboard for now would give us insight into the trends and what is most meaningful- then we could work on adjusting how we upload data or calculate/alert with perfherder to get the most value.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I assume this could be done now with js-perf that we use for afwy, :armenzg, can you comment on existing solutions or work that might support this?

Comment 3

[Armen [:armenzg]]

This can either be done in JS perf dashboard or health.graphics.

Probably the JS perf dashboard makes more sense.

We could have:
* arewefastyet.com/tp6
* health.graphics/tp6

I think JS perf would be a better place since it *only* focuses on Perfherder data.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

+1 for awfy.com/tp6

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

:armenzg, is this something :ekyle could pick up as a first project?

Comment 6

[Armen [:armenzg]]

I believe so.

FYI we won't be investing on the Firefox perf dashboard and instead have different views on Firefox health to reduce duplicating efforts.
The Outreachy work on the Firefox perf dashboard would roll back into the Firefox Health dashboard.

You can already see some Tp6 graphs in here:
https://health.graphics/quantum/64

The code to add any of these graphs is here:
https://github.com/mozilla-frontend-infra/firefox-health-dashboard/blob/master/src/quantum/index-64bit.js#L145-L165

We could also plot all individual tests OR have a way to drill down.
The drilling down feature needs to be worked on.

Comment 7

[Kyle Lahnakoski [:ekyle]]

jmhaer, 

I heard there is a talos tp6 and a raptor tp6.  May you give me a link to the talos tp6 so I can add it too.

https://github.com/mozilla-frontend-infra/firefox-health-dashboard/pull/233

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

actually we only care about raptor tp6 :)  there is a talos tp6 for stylo threads, but we only measure one data point there- so for the purposes of this, we only have raptor to consider

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'd say this is now covered by https://health.graphics/quantum/tp6 and https://health.graphics/android/tp6m