1216218 - autophone data on perfherder is tracked by platform, we should consider tracking by device type

Status: RESOLVED WONTFIX

(Testing Graveyard :: Autophone, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Right now in autophone, we submit a json blob with results upon completion to treeherder.  This includes meta data about the device and the build, then specifics about the test (including performance numbers).

The problem I have is when we access data from the perfherder interface it is sorted by branch/platform/test- there is no concept of device type (e.g. nexus-7).  As we switch from talos -> autophone we only plan to use a single device.  What happens 6 months later when we switch device types or add a second device type?

Right now the platform is taken from the overall metadata, so if we were to change this we could:
* report the overall platform as android-4-1-blah.<device>
* report a custom platform in the performance_data and modify treeherder to use that.

thinking about this more, we should find a way to break out device type in all treeherder views (pushlog format, and perfherder).  This would match up to the first bullet point above.  In doing that, it would work easily with perfherder, but confuse treeherder pushlog view.  

To adjust it we could just make the platform that we post something more complete:
https://github.com/mozilla/autophone/blob/master/autophonetreeherder.py#L236

What other options do we have?  What makes sense to do?

Comment 1

[William Lachance (:wlach)]

I think the ideal solution might be to expand the scope of MachinePlatform to incorporate a "hardware" field (maybe optionally null to cover the cases where we don't care, which is many/most of them)

https://github.com/mozilla/treeherder/blob/master/treeherder/model/models.py#L105

CC'ing various treeherder devs on this. I'll bring this up in the week's treeherder meeting as well.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

checking in on this bug, I would like to figure out what we want to do here in the short term so we can make adjustments before collecting large quantities of data.

Comment 3

[Ed Morley [:emorley]]

I guess this is another manifestation of bug 1060769?

Agree we should come up with a long term plan for this.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

yes, this sounds related.

Comment 5

[Bob Clary [:bc] (inactive)]

Autophone is going away. Resolving these to wontfix.