1169934 - Get datasource tests running on its Travis & fix flake8 warnings

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

Datasource:
* is not being worked on by anyone else other than us
* is annoying to debug when things go wrong, since it requires digging into another repo
* could be pared down quite a bit (eg unnecessary DB-independent abstraction layer) even before we move away from it entirely
* is the last package in our requirements files that is not on pypi (and thus doesn't get installed by tools like landscape.io)

As such I think we should move it into the Treeherder repo.

Comment 1

[Ed Morley [:emorley]]

I've landed a few misc improvements to datasource in the datasource repo, to get things ready for any potential import (eg so flake8 doesn't fail):
https://github.com/jeads/datasource/commit/dfeb14eaec285c7aa16a2ac3b5caa0b9960f11c4
https://github.com/jeads/datasource/commit/58ca7f727e1add7ab5608ff229909a0127ed318c
https://github.com/jeads/datasource/commit/6e403553354f38b696f4ace4a25e583a4ac9bede

Comment 2

[Ed Morley [:emorley]]

Attachment: Get the datasource tests running on Travis from the datasource repo

The datasource tests aren't currently run on commit anywhere. Before we import datasource or make any more changes, it seems sensible to change that.

After this PR I think it makes sense to convert the tests to py.test so we can then more easily import datasource+tests into the Treeherder repo and run with all of the other tests.

Comment 3

[Ed Morley [:emorley]]

Attachment: Fix non-whitespace flake8 errors

flake8 actually found two real bugs \o/ There are still a couple of flake8 errors - but I need to look at them more closely, since I think they may be due to further bugs in datasource.

Comment 4

[Ed Morley [:emorley]]

Comment on attachment 8613261 [details] [review]
Fix non-whitespace flake8 errors

https://github.com/jeads/datasource/commit/c6c5508b727bf7554a3e72c3e572ec3a2a98a92b
https://github.com/jeads/datasource/commit/c876e33ef4a0a94fd97be0daf0937ef4f8950de5
https://github.com/jeads/datasource/commit/2267727f028de9a73026e2f2528f80870712a993
https://github.com/jeads/datasource/commit/882bd9f707e099480c04c79c0cf251ae293e366e
https://github.com/jeads/datasource/commit/12d75ccf9ad4e88abcde30ece24b87ae2e104114
https://github.com/jeads/datasource/commit/9be7d0b38e4fade2199a2e53753e805821c26c23
https://github.com/jeads/datasource/commit/f9b76e330e194d3764c9a876cf74a165a7f1664c

Comment 5

[Ed Morley [:emorley]]

Comment on attachment 8613260 [details] [review]
Get the datasource tests running on Travis from the datasource repo

https://github.com/jeads/datasource/commit/0f7bd0a628dc083b16b6bc64fa9abe532bd98e22
https://github.com/jeads/datasource/commit/7b4efb790c8418e54f5508bf6060fc06c4be8fc2
https://github.com/jeads/datasource/commit/fd36c723412e8255db37029cbd8c4d4084d00b48
https://github.com/jeads/datasource/commit/aab1f00d9533cb043c7e6074a0f01e02814fba2a

Comment 6

[Ed Morley [:emorley]]

Morphing this to be about what landed - I'll file a new bug for the remaining work (since it's less urgent).