1698569 - Link to Raptor test documentation

Status: RESOLVED FIXED

(Tree Management :: Perfherder, enhancement, P2)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

With bug 1698567 we will be able to link directly to Raptor test documentation in the Firefox Source Tree Documentation. This means we will be able to reliably link to the documentation by knowing the framework (raptor/browsertime), suite name (amazon, allrecipes, etc) and platform (desktop/mobile).

We should provide a link to the test documentation wherever we show the test suite name. Once we have resolved bug 1698548 we will have a better idea of the places we want to add these links.

Comment 1

[Acasandrei Beatrice (needinfo me)]

I've reached out to Kimberly about mapping raptor tests in Perfherder to the documentation, as it is not clear to me how to properly generate the links in Perfherder.
She concluded that:
There doesn't seem to be a straightforward way to do this as with talos (https://firefox-source-docs.mozilla.org/testing/perfdocs/talos.html#{suite}). The suite name for raptor tests in perfherder are prefixed with raptor and suffixed with the platform and sometimes other information, for example -firefox-cumulative-power in raptor-youtube-playback-h264-full-1080p30-firefox-cumulative-power
the URLs for raptor tests are suffixed with a letter to indicate the type of test: (-b for benchmark, -m for mobile, -d for desktop, etc)
so raptor-youtube-playback-h264-full-1080p30-firefox-cumulative-power should direct to http://gecko-docs.mozilla.org-l1.s3-website.us-west-2.amazonaws.com/main/6393d11a-e31a-11eb-8190-0242ac110004/testing/perfdocs/raptor.html#youtube-playback-h264-full-1080p30-b (generated docs from the perfdoc patch I've been working on)

Let's continue the discussion here. I still don't know how to go about the generation of the URLs since I don't have a properly defined acceptance criteria.

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The suite name for raptor tests in perfherder are prefixed with raptor and suffixed with the platform and sometimes other information

It sounds like this is describing the raptor-webext tests. To keep this simple, let's only add links for the raptor-browsertime tests. This means tests with the framework "browsertime" rather than "raptor".

the URLs for raptor tests are suffixed with a letter to indicate the type of test: (-b for benchmark, -m for mobile, -d for desktop, etc)

We have different URLs for desktop/mobile page load tests, which is why we append -d and -m. I don't know why we have -b for benchmarks as these shouldn't clash with any of the page load test names, and I would suggest removing this suffix. For page load tests, we could look at the platform in Perfherder and add -m for any containing Android, otherwise add -d.

A couple of alternatives that come to mind:

• Ensure that all test names are unique and then we can even remove the -d and -m added by PerfDocs.
• Add anchors in PerfDocs for each platform class, so Perfherder can link directly to [test]-[linux|windows|macos|android]

Comment 3

[Kimberly Sereduck :kimberlythegeek]

I agree with adding new anchors for each platform class. It seems more straightforward than other options

Comment 4

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/7234

Comment 5

[Kimberly Sereduck :kimberlythegeek]

The github PR has been merged so marking this as resolved