Identify results with mulitple data points per push
Categories
(Testing :: Performance, task, P3)
Tracking
(Not tracked)
People
(Reporter: davehunt, Unassigned)
Details
With bug 1672740 and bug 1672822 we've stumbled on results that are mixed in Perfherder when running in multiple variations. This should be something we can identify by looking for results that consistently report more than one data point per commit. Let's use this bug to track identifying and filing bugs for each occurrence, and to discuss ways to prevent or get early notification of future occurrences.
Reporter | ||
Comment 1•4 years ago
•
|
||
The following query helped to identify pushes with more than one datapoint. Some of these are due to retriggers, and a lot of them appear to be related to bug 1672740. Perhaps we should rerun this query once that bug is resolved.
SELECT
count(*),
push_timestamp,
datum.signature_id,
signature.framework_id,
signature.suite,
signature.test,
datum.push_id,
push.revision,
job_type.symbol
FROM performance_datum as datum
INNER JOIN performance_signature AS signature ON signature.id = datum.signature_id
INNER JOIN push ON push.id = datum.push_id
INNER JOIN job ON job.id = datum.job_id
INNER JOIN job_type ON job_type.id = job.job_type_id
WHERE datum.repository_id = 77
AND framework_id NOT IN (2, 14) -- exclude build_metrics and vcs
AND push_timestamp > DATE_SUB(CURDATE(), INTERVAL 1 DAY)
GROUP BY signature_id, push_id
HAVING count(*) > 1
ORDER BY count(*) DESC
Reporter | ||
Comment 2•4 years ago
|
||
This improved query shows all signatures that have more than one job symbol, and I believe confirms that the open bugs account for all issues:
SELECT
datum.signature_id,
signature.framework_id,
signature.suite,
signature.test,
push.revision,
COUNT(DISTINCT(CONCAT(job_group.symbol, '(', job_type.symbol, ')'))) AS symbol_count,
job_group.name,
job_group.symbol AS job_group_symbol,
GROUP_CONCAT(DISTINCT(job_group.symbol)) AS job_group_symbol_concat,
job_type.symbol AS job_type_symbol,
taskcluster_metadata.task_id AS task_id
FROM performance_datum AS datum
INNER JOIN performance_signature AS signature ON signature.id = datum.signature_id
INNER JOIN push ON push.id = datum.push_id
INNER JOIN job ON job.id = datum.job_id
INNER JOIN job_type ON job_type.id = job.job_type_id
INNER JOIN job_group ON job_group.id = job.job_group_id
INNER JOIN taskcluster_metadata ON taskcluster_metadata.job_id = job.id
WHERE datum.repository_id = 77
AND framework_id NOT IN (2, 14) -- exclude build_metrics and vcs
AND push_timestamp > DATE_SUB(CURDATE(), INTERVAL 1 DAY)
GROUP BY datum.signature_id
HAVING COUNT(DISTINCT(CONCAT(job_group.symbol, '(', job_type.symbol, ')'))) > 1
ORDER BY symbol_count DESC
Description
•