1149341 - Talos series signatures have the same "repository" key in them

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect)

Description

[William Lachance (:wlach)]

The Talos series signatures all have the same useless "repository" keys in them, which unnecessarily tie them to the specific project. E.g. this signature for mozilla-central: 

{
    "build_architecture": "x86_64",
    "build_os_name": "win",
    "build_platform": "windows8-64",
    "build_system_type": "buildbot",
    "device_name": "unknown",
    "job_group_name": "Talos Performance",
    "job_group_symbol": "T",
    "job_type_name": "Talos g1",
    "job_type_symbol": "g1",
    "machine_architecture": "x86_64",
    "machine_os_name": "win",
    "machine_platform": "windows8-64",
    "option_collection_hash": "102210fe594ee9b33d82058545b1ed14f4c8206e",
    "repository": "mozilla-central", <---- this is a problem
    "suite": "tp5o_scroll",
    "test": "store.apple.com"
}

We should take these out, as it will make tracking series between projects a lot easier. This involves two steps:

(1) Not storing the repository key when creating signatures (easy)
(2) Rewriting the old performance data to use the new keys (it is important that we temporarily turn off ETL while this is happening so that we don't add more  performance data using the old signatures while this is happening).

I am not sure how easy it would be to do (2) at the moment. Need to think about it and/or talk to other treeherder devs.

Comment 1

[William Lachance (:wlach)]

Hey Mauro, how difficult would it be to turn off artifact ingestion (and new artifact processing) while we make a bunch of modifications to the database (see comment 0)? It turns out that this misfeature is kind of annoying, it would be good to fix it sooner than later.

Comment 2

[Mauro Doglio [:mdoglio]]

The easiest way I can think of is to put the api in read-only mode using a feature flipper django-waffle. I think this would be a good addition anyway. :wlach do you have a vague idea of the time required to do the db update?

Comment 3

[William Lachance (:wlach)]

It looks like this will be most likely be addressed as part of bug 1168214.

Comment 4

[William Lachance (:wlach)]

This was fixed with bug 1168214.