1309294 - Duplicate performance signatures being generated after heroku migration

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

We're seeing a ton of duplicated performance signatures being generated after the heroku migration. I believe this is due to heroku's python not being deterministic about what running json.dumps on a dict will generate, which perfherder's signature generation algorithm depends on. 

An easy solution would be to pass sort_keys to json.dumps (this is what should have been done originally), but since we're here and have to sort out so much messed up data anyway, let's go ahead and just remove the code that incorporates the child signatures into the calculation, which is what required json.dumps (and thus caused this bug).

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1309294 > mozilla:master

Comment 2

[Ed Morley [:emorley]]

It's worth noting that SCL3 was stuck on Python 2.7.10 (too much of a hassle to try and update it, since the rsynced venv used system Python ...) whereas Heroku is using Python 2.7.11 (I'm going to update that to 2.7.12 shortly), so presumably either this or different OS or bug 1292720 is responsible for the ordering changing. (iirc Python dicts are not expected to be deterministic, and thus json.dumps() output can vary unless sorted)

Comment 3

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/ee29f26efa47e2250204361969f923905600457a
Bug 1309294 - Don't use subtests in perf signature calculation (#1915)

Comment 4

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/0e0375c001df527acc5eb2117cf5e9a29b4aae8c
Bug 1309294 - Fix age parameter to cleanup script

Comment 5

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:1309294-2 > mozilla:master

Comment 6

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/b2098db6ad63ce31288c2d9c1a4f07c63dd41828
Bug 1309294 - Fixes to signature calculation for remove signatures script (#1924)

Comment 7

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/98f033bdb1bde739d7fbdff438758fc409abc6a5
Bug 1309294 - Another fix to the signature cleanup script

Comment 8

[William Lachance (:wlach)]

Ok, data on production should be ok again. Unfortunately signature hashes changed so old graph and compare subtest links are now invalid. :( Filed bug 1310386 to fix things up so this doesn't happen again.