Open
Bug 1945517
Opened 6 days ago
Update ML Perf Tests To Report All Data Points
Categories
(Core :: Machine Learning, task)
Core
Machine Learning
Tracking
()
NEW
People
(Reporter: nordzilla, Unassigned)
References
Details
Description
The machine-learning perftest metrics report only the median entry of their runs. This is great for combating false-positive perf alerts, but it unnecessarily smooths the data.
The ideal solution should be to report only the median to perf alerts, as we already are, but still record all of the data points for analysis.
Bug 1944155 modifies the way that mochiest perf data is parsed, allowing to report all of the data points along with the median. The ML code needs to be updated to take advantage of this.
Steps to Implement
Here is an example of how the Translations code was updated in Bug 1944065
/**
- * Logs the median value of each collected metric to the console.
+ * Logs the median value along with the individual values from all
+ * test runs for each collected metric to the console.
* The log is then picked up by the perftest infrastructure.
* The logged data must match the schema defined in the test file.
*/
reportMetrics() {
- const reportedMetrics = {};
+ const reportedMetrics = [];
for (const [name, values] of Object.entries(this.#metrics)) {
- reportedMetrics[name] = median(values);
+ reportedMetrics.push({
+ name,
+ values,
+ value: median(values),
+ });
}
info(`perfMetrics | ${JSON.stringify(reportedMetrics)}`);
}
};
You need to log in
before you can comment on or make changes to this bug.
Description
•