Closed Bug 1602806 Opened 2 years ago Closed 2 years ago

subtest_lower_is_better fields don't seem to be respected

Categories

(Testing :: Raptor, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: away, Unassigned)

References

(Blocks 1 open bug)

Details

Here is a log from a Speedometer run: https://firefoxci.taskcluster-artifacts.net/aqoR-SCgSQKvnRTs3EfbUA/0/public/logs/live_backing.log

You can see the "input" value of subtest_lower_is_better is coming in correctly:
\"subtest_lower_is_better\": \"true\"

But when the test is producing perfherder data at the end, the value doesn't come through:
u'Angular2-TypeScript-TodoMVC': [112.17999999999938, 122.35999999999694, 108.34000000000015, 111.05999999999767, 118.61999999999534]}]]}, u'browser_cycle': u'1', u'subtest_lower_is_better': False,

And so the graphs on perfherder go the wrong way for speedometer subtests.

Component: Perfherder → Raptor
Product: Tree Management → Testing
Version: --- → unspecified
Priority: -- → P3

(In reply to Dave Hunt [:davehunt] [he/him] ⌚BST from comment #1)

I suspect this might be a regression from https://searchfox.org/mozilla-central/diff/2b036e45c518c1c6d9f217d6ad6001e4a40998b1/testing/raptor/webext/raptor/runner.js#130

As far as I can tell that was just a whitespace change.

Looks like this is working as expected now. I suspect it was fixed in bug 1585022.

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.