Closed Bug 1570702 Opened 5 years ago Closed 5 years ago

Raptor speedometer power is running on mozilla-central when backfills are performed

Categories

(Testing :: Raptor, defect, P1)

Version 3
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: sparky, Assigned: sparky)

References

Details

This bug is to investigate why raptor speedometer-power tests are running on mozilla-central when they shouldn't given that the run-on-projects flag is set to []1. There is currently a cron task which runs them on saturday's and it should not run at any other time 2.

Here's the view of when they've run: https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=mozilla-central,2008512,1,10&series=mozilla-central,1955233,1,10&series=mozilla-central,1955223,1,10&selected=mozilla-central,1955223,519610,875798773,10

The issue started on Friday, July 19 so we suspect that this may be caused by a regression from some code added/modified at around that point.

Looking at the mozilla-central runs, we find that the task is scheduled by a backfill trigger. In 3, we find the backfill action task in the decision task category. In 4, there is no backfill action task and no entry for the power task in the target-tasks.json file. But looking at the speedometer-power task 5, we also find that a backfill action task triggered it (here's the speedometer task run 6).

Ionut, would you have any ideas about what could be causing this or who we should speak to about it?

Flags: needinfo?(igoldan)
Priority: -- → P1
Flags: needinfo?(igoldan)
See Also: → 1563296

Hi Dustin, would you know of any recent changes that could be causing this issue?

Flags: needinfo?(dustin)

(In reply to Greg Mierzwinski [:sparky] from comment #0)

This bug is to investigate why raptor speedometer-power tests are running on mozilla-central when they shouldn't given that the run-on-projects flag is set to []1. There is currently a cron task which runs them on saturday's and it should not run at any other time 2.

Here's the view of when they've run: https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=mozilla-central,2008512,1,10&series=mozilla-central,1955233,1,10&series=mozilla-central,1955223,1,10&selected=mozilla-central,1955223,519610,875798773,10

The issue started on Friday, July 19 so we suspect that this may be caused by a regression from some code added/modified at around that point.

Looking at the mozilla-central runs, we find that the task is scheduled by a backfill trigger. In 3, we find the backfill action task in the decision task category. In 4, there is no backfill action task and no entry for the power task in the target-tasks.json file. But looking at the speedometer-power task 5, we also find that a backfill action task triggered it (here's the speedometer task run 6).

Ionut, would you have any ideas about what could be causing this or who we should speak to about it?

I think :ahal would be able to assist in properly troubleshooting this.

Flags: needinfo?(dustin)

Thanks! I've added a ni? for ahal.

Flags: needinfo?(ahal)

Looking at central, they haven't run since Jul 27th:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=sp-p

Oddly the first couple of pushes where it stops showing up don't seem to have anything relevant. Maybe the issue was with the cron task? Otherwise it's possible a transform somewhere was modifying run-on-projects after the fact.

But since it looks fixed now I didn't bother spending time looking into it.

Flags: needinfo?(ahal)

It appears that no power tests have run since July 27th, however they should have run on August 3rd...

(In reply to Greg Mierzwinski [:sparky] from comment #6)

The august 3rd android builds failed (the cron task ran though): https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=6954782553c76545a6ac10a2f5373e17b5e577c7

There are no results for August 10th shown on health.graphics. Could this be due to the change in subtest name?

Yes, that's definitely the issue here since they did run (although they didn't run at the right time which is pretty odd): https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=power&revision=3afb892abb74c6d281f3e66431408cbb2e16b8c4

This wrong push day issue isn't limited to the android-power-tests cron task, the tp6m-fennec64 (f64) task also ran on Friday instead of saturday: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=cron&revision=3afb892abb74c6d281f3e66431408cbb2e16b8c4

I've made an issue on the health graphics repo for this: https://github.com/mozilla-frontend-infra/firefox-health-dashboard/issues/532

Assignee: nobody → gmierz2
Status: NEW → ASSIGNED

Hey :sparky, what is the status of this issue (are you still looking at it, is it still P1)? Thanks!

Flags: needinfo?(gmierz2)

Let's close this now, I haven't seen this happen in the last little while: https://treeherder.mozilla.org/perf.html#/graphs?timerange=7776000&series=mozilla-central,2008508,1,10&highlightAlerts=1

I think it was fixed with the run-on-projects change in bug 1578723. I'll reopen if I find that this changes.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Flags: needinfo?(gmierz2)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.