1139517 - change log name of buildbot logs from 'builds-4h' to 'buildbot_text'

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect)

Description

[Cameron Dawson [:camd]]

The API has been (will by the time anybody gets to this) changed to look for the new name from the UI.  But in the DB, we still store it as the old name.  And there are lots of tests and test files to be changed.  

I didn't want to muddy my other PR with this.

Comment 1

[Jonathan French (:jfrench)]

I'm taking a flyer this might be doable as a 'good next bug' (maybe not a first bug), but I'm not really sure what it involved. If I'm wrong, free to remove whiteboard/mentor :)

Comment 2

[Krishnashish Gogoi [:doomkookie]]

Hey, I'm lookin to work on this bug. I'm not too sure on which files to start working on so some help would be necessary. 

I ran a search on the github repo for builds-4h: https://github.com/mozilla/treeherder-service/search?q=builds-4h&type=Code&utf8=%E2%9C%93

Are these the files that need changing? I suppose the lines in logslice.py will need to be removed as well (saw the reference to this bug from the todo above it). If there's some other stuff to change as well or tests to be run, please let me know. I just finished setting up my vagrant environment, so looking forward to this.

Thank you.

Comment 3

[Jonathan French (:jfrench)]

Hi Krishnashish, sure! Assigning this one to you. Cameron should be online in an hour or so.

Comment 4

[Cameron Dawson [:camd]]

Yep, those look like the right files to me.  :)  Thanks for taking this on!

Comment 5

[Krishnashish Gogoi [:doomkookie]]

Attachment: Changes buildbot logs from 'builds-4h' to 'buildbot_text'.

Comment 6

[Cameron Dawson [:camd]]

This is looking very good.  I want to do a bit more testing with it before I merge though.  Testing backward compatibility.  I am on PTO tomorrow (Friday) but will test if I'm able.  I may not get to merge till Monday, however.  Sorry for the delay.

Comment 7

[Cameron Dawson [:camd]]

Comment on attachment 8594240 [details] [review]
Changes buildbot logs from 'builds-4h' to 'buildbot_text'.

I've tested this with both the old and new log names and it works great.  Thanks!!

Comment 8

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/bc0b48b262a397f74b66e5d110d0fe8bea1b6104
Bug 1139517 - Changes log name of buildbot logs from 'builds-4h' to 'buildbot_text'

This commit changes all the references to 'builds-4h' to 'buildbot_text'. Following
are the changed files along with the no. of occurences that have been changed in
each.

1. tests/etl/test_buildapi.py: 3 occurences
2. tests/sample_data/job_data.txt: 304 occurences
3. treeherder/etl/buildapi.py: 1 occurence
4. treeherder/model/sample_data/job_data.json.sample: 2 occurences

In treeherder/webapp/api/logslice.py, a conditional was removed. The todo above
it instructed it to be removed once this bug was addressed.

To make sure all tests run properly, three files were renamed. Only the portion
of the filename that said 'builds-4h' was changed to say 'buildbot_text'.

https://github.com/mozilla/treeherder/commit/254f467757aadc5d27b4ccaa73cf6c0c6c4058fb
Bug 1139517 - Adds a check for old logs till they're expunged from the db.

Till all the old data is expunged from the db, this should act as a check
for them. New logs ingested use the new name. This should be removed
once all the old data is removed from the db in a few months, which should
be around 4 months.

https://github.com/mozilla/treeherder/commit/07ee8d0546e441aab5339cd6cddd40cfe3ae7aac
Merge pull request #471 from mister-raindrop/bug1139517

Bug 1139517 - Changes log name of buildbot logs from 'builds-4h' to 'buildbot_text'