1892260 - [meta] Ensure all performance tests produce a perfherder-data.json artifact

Status: RESOLVED DUPLICATE of bug 1986824

(Testing :: Performance, task, P2)

Description

[Greg Mierzwinski [:sparky]]

We'll need to start using the perfherder-data.json artifacts for parsing performance data from our Taskcluster tasks. To do this, all of our performance tests (mozperftest/awsy/raptor/talos/firefox-ios) will need to output a JSON file artifact called perfherder-data.json by default.

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

We'll have to double check every perftest to see what is missing it. From grooming we quickly checked on CI it seems many already have it, so we need to find the ones don't

Comment 2

[Alex Finder]

Based on what the current investigations, the following mozperftest clearly do not produce a perfherder-data.json artifact:

1. various mozperftest tasks containing "startup" in the name

• startup-fenix-cmff
• startup-fenix-cvns
• startup-geckoview-cmff
• startup-geckoview-cmff
• startup-chrome-m-cmff

2. source-test-python-mozperftest

• mpu

3. various fetch_content jobs

• concept-fetch

4. source-test-python-mozversioncontrol

• vcs

Comment 3

[Alex Finder]

We should probably create a separate ticket for adding perfherder-data.json file generation for all of the jobs mentioned.

Comment 4

[Greg Mierzwinski [:sparky]]

To add to the list that :afinder produced, there are also build tasks that need to generate JSON files. This searchfox search should provide all of the areas that need to be checked to ensure that the perfherder-data is being output to a JSON file into the artifact folder: https://searchfox.org/firefox-main/search?q=PERFHERDER_DATA&path=&case=true&regexp=false

Something to note is that some tasks will produce multiple perfherder-data entries. We will need to ensure that we append something to their name to ensure that they are unique. In terms of ingestion, on the treeherder side we could then parse the data from any JSON files that begin with or contain perfherder-data in the file name.

Comment 5

[Myeongjun Go]

Attachment: Bug 1892260 - [meta] Ensure all performance tests produce a perfherder-data.json artifact r=sparky

Comment 6

[Greg Mierzwinski [:sparky]]

Closing this one as a duplicate of bug 1986824.