1602806 - subtest_lower_is_better fields don't seem to be respected

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P3)

Description

[(Away)]

Here is a log from a Speedometer run: https://firefoxci.taskcluster-artifacts.net/aqoR-SCgSQKvnRTs3EfbUA/0/public/logs/live_backing.log

You can see the "input" value of subtest_lower_is_better is coming in correctly:
\"subtest_lower_is_better\": \"true\"

But when the test is producing perfherder data at the end, the value doesn't come through:
u'Angular2-TypeScript-TodoMVC': [112.17999999999938, 122.35999999999694, 108.34000000000015, 111.05999999999767, 118.61999999999534]}]]}, u'browser_cycle': u'1', u'subtest_lower_is_better': False,

And so the graphs on perfherder go the wrong way for speedometer subtests.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I suspect this might be a regression from https://searchfox.org/mozilla-central/diff/2b036e45c518c1c6d9f217d6ad6001e4a40998b1/testing/raptor/webext/raptor/runner.js#130

Comment 2

[(Away)]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚BST from comment #1)

I suspect this might be a regression from https://searchfox.org/mozilla-central/diff/2b036e45c518c1c6d9f217d6ad6001e4a40998b1/testing/raptor/webext/raptor/runner.js#130

As far as I can tell that was just a whitespace change.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Looks like this is working as expected now. I suspect it was fixed in bug 1585022.