1178232 - Remove the ability to have multiple (numbered) Datasources of the same type

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

The datasource table has a 'dataset' field, to allow for multiple datasources of the same type (for partitioning; eg the "1" in `mozilla-central_jobs_1`). However we have never used it, and will likely not do so. 

Let's just remove it.

Comment 1

[Ed Morley [:emorley]]

Attachment: Remove support for multiple Datasources of the same type

Is currently a WIP, have left a couple of questions on the PR.

In addition to making the tests pass, I will further clean up some of the methods in derived/base.py and models.py.

Comment 2

[Ed Morley [:emorley]]

Comment on attachment 8627495 [details] [review]
Remove support for multiple Datasources of the same type

Tests are now passing. Up to you whether we take this change; would probably not be worth doing if it hadn't been started yet (given https://github.com/mozilla/treeherder/pull/694), but oh well.

Comment 3

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/d2010a621535259a10a9e32f73760b8df40e5cc0
Bug 1178232 - Remove support for multiple Datasources of the same type

The datasource table has a 'dataset' field, to allow for multiple
datasources of the same type (for partitioning; eg the "1" in
`mozilla-central_jobs_1`). However we have never used it, so let's just
remove it.

Comment 4

[Ed Morley [:emorley]]

mysql> SELECT table_schema, table_name, column_name FROM information_schema.columns WHERE column_name = 'dataset';
+------------------+------------+-------------+
| table_schema     | table_name | column_name |
+------------------+------------+-------------+
| treeherder_stage | datasource | dataset     |
+------------------+------------+-------------+
1 row in set (0.11 sec)

mysql> ALTER TABLE treeherder_stage.datasource DROP COLUMN dataset;
Query OK, 0 rows affected (0.07 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM information_schema.statistics WHERE column_name = 'dataset';
Empty set (0.04 sec)

Comment 5

[Ed Morley [:emorley]]

And prod...

[[e-mail]3 treeherder-service]$ ../venv/bin/python ./manage.py dbshell

mysql> ALTER TABLE treeherder.datasource DROP COLUMN dataset;
Query OK, 0 rows affected (0.12 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM information_schema.statistics WHERE column_name = 'dataset';
Empty set (0.44 sec)

Comment 6

[Ed Morley [:emorley]]

Dropped from heroku too.