Closed Bug 1777009 Opened 3 months ago Closed 2 months ago

Display new treeherder test failure info on test pages in the test info box

Categories

(Webtools :: Searchfox, enhancement)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: asuth, Assigned: asuth)

References

Details

Attachments

(1 file)

Treeherder and the test infrastructure has new failure reporting super-powers, and searchfox wants in on them! I'm filing a separate bug to get a treeherder artifact with aggregated test info that searchfox can display, but this would also then likely want to link to treeherder's UI(s) for additional details.

Depends on: 1777063

"failure_count" is now available. Examples from the current running config1 utc10 indexer:

Per-test:

      {
        "failure_count": 33,
        "skip-if": "\n!fission\n!crashreporter\nverify",
        "test": "dom/ipc/tests/browser_crash_oopiframe.js"
      },

At the root there's a description of the time horizon:

  "description": "Test summary report for tests, in all components, including historical run-time data for the last 30 days on trunk (autoland/m-c) as of 2022-07-21.",

That's a bit wordy and contextually weird to inline in the test info box without user interaction, but it seems like useful context to provide when explaining where we got the information from. Maybe this can be wrapped in a <details> tag that could reference the treeherder job that we got the information from as it is nice for people to understand where the information came from so they can validate it and potentially enhance it.

For hyperlinking purposes, we can use the test path and slap it onto the end of https://bugzilla.mozilla.org/buglist.cgi?quicksearch= and that will get the user to the bugs which can then get them to the treeherder intermittent-failures links. (The derive-per-file-info logic that runs as part of the AWS job does not and should not do any network traffic of its own to try and map the test failures to bugs; any additional information desired here should be handled in the existing taskcluster job whose output we're consuming here or in a new taskcluster job.)

Assignee: nobody → bugmail
Status: NEW → ASSIGNED

I went minimal on the changes here. I reused the old support pretty much as-is, just changing the JSON field we deserialize from, but left the rust struct as-is. I upped the 7 days to 30 days in our hard-coded string and did not tunnel the "description" because the <details> behavior would potentially interact non-intuitively with the hyperlink I did add for the "quicksearch this test" helper link.

We can iterate on this, of course!

Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.