1281820 - Stop using/storing bug suggestion artifacts, just generate them dynamically

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect)

Description

[William Lachance (:wlach)]

The next logical step after bug 1266062 is to migrate the error summary artifacts into the Django ORM. Lots of potential for cleanup and simplification here, as well as hopefully reducing the amount of redundant data.

Comment 1

[William Lachance (:wlach)]

After some experimentation, I believe we should be able to just generate bug suggestions dynamically, without bothering to store them. This should be beneficial from a lot of angles (ensuring they're relatively up to date, ease of modification, disk space usage...)

Comment 2

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1281820 > mozilla:master

Comment 3

[William Lachance (:wlach)]

Comment on attachment 8793921 [details] [review]
[treeherder] wlach:1281820 > mozilla:master

Hey James, most of the changes here were related to autoclassification in one way or another, so I'm passing this to you. I (tried to) clean up the tests as well as part of this patch, which have seperate commits, but I think this is best reviewed as a unit.

Comment 4

[Ed Morley [:emorley]]

The more I think about this, the more I think it's an awesome idea.

Traditionally Treeherder has had a very write-heavy DB workload (which hinders with many things - replication included), and this is one step to making it more read-heavy (well, with memcached, not much so, but same idea). It's much easier to scale out read-capacity, than write.

Comment 5

[William Lachance (:wlach)]

It probably doesn't make sense to land this when (1) we're in the middle of the heroku migration and (2) I'm about to go on PTO (until next Wednesday). I'll do that when I return next week.

Comment 6

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/69bdc2ed155226c8760b1ca23c70345d85e70ce8
Bug 1281820 - Clean up autoclassification unit tests

* Remove unused fixtures
* Add test_job, test_job_2 fixtures for testing job data, instead of
  ingesting 11 jobs
* Remove various uses of datasource

https://github.com/mozilla/treeherder/commit/9b9d37e6903589824eaa5456489defb56784ff57
Bug 1281820 - Move autoclassify methods into Django ORM class

https://github.com/mozilla/treeherder/commit/8fc9754f4e0665cd0fd18530ed4196868b3179e8
Bug 1281820 - Stop using bug suggestions artifacts

Instead, generate the data when required. We will store the return value
of this in memcache for a day to ensure things are responsive for the sheriffs
when classifying recent failures.

Comment 7

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I'm getting 404 errors when selecting jobs on ./web-server.js with a recent pull from master:

https://treeherder.mozilla.org/api/project/mozilla-inbound/jobs/37297801/bug_suggestions/
HTTP/1.1 404 NOT FOUND
Connection: keep-alive
Server: gunicorn/19.6.0
Date: Sat, 08 Oct 2016 05:26:16 GMT
Transfer-Encoding: chunked
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000
Content-Type: text/html; charset=utf-8
Access-Control-Allow-Origin: *
x-frame-options: DENY
X-Content-Type-Options: nosniff
Via: 1.1 vegur

Comment 8

[William Lachance (:wlach)]

(In reply to Wes Kocher (:KWierso) from comment #7)
> I'm getting 404 errors when selecting jobs on ./web-server.js with a recent
> pull from master:

Yeah, the server side changes are not yet deployed to production (I wanted to let this large change soak on stage for the weekend).

Meanwhile you can run `./web-server.js --stage` and the UI will work against stage, which does have these changes.

Comment 9

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1281820-2 > mozilla:master

Comment 10

[William Lachance (:wlach)]

Just posted a followup PR to actually remove the bug suggestion artifact storage code, now that we've validated things are working ok in production. Once that's applied, I think we can resolve this bug.

Comment 11

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1310691 > mozilla:master

Comment 12

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/b035647fe2543a1b657d66bb0b91a7d2e321fe83
Bug 1281820 - Remove all bug suggestion artifact storage code + tests (#1927)

Comment 13

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1281820-3 > mozilla:master

Comment 14

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/5abb9e86088c364a8d394b6c18665ec54e9446ca
Bug 1281820 - Revert db schema changes temporarily (#1933)

They need to be applied seperately

Comment 15

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/1b0d0f27f8a258ef825150790783c31e24cd5fef
Revert "Bug 1281820 - Revert db schema changes temporarily (#1933)"

This reverts commit 5abb9e86088c364a8d394b6c18665ec54e9446ca.

Comment 16

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/6ff8cef53273dfc95d85c4f8b21a25e88bb24f61
Bug 1281820 - Remove a few more references to artifacts from the ui

Comment 17

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/5cef814f0f7670c12bc847d04a361aea4e8a5f13
Bug 1281820 - Revert db schema changes temporarily (again)

These are not being applied (there are complaints about integrity
errors because of job guid conflicts), I think there are some issues with
dropping columns inside an actively updated table. Reverting until we
can figure them out.

Comment 18

[William Lachance (:wlach)]

I am having trouble getting the column deletes for the text log summary table applying, so have reverted them for now. We seem to be getting integrity errors during the migration process, probably due to some mysql quirk. Here's what I'm seeing in the papertrail logs:

Oct 18 16:30:46 treeherder-stage app/release.6181:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-2.68.0.50/newrelic/hooks/database_dbapi2.py", line 22, in execute 
Oct 18 16:30:46 treeherder-stage app/release.6181:     *args, **kwargs) 
Oct 18 16:30:46 treeherder-stage app/release.6181:   File "/app/.heroku/python/lib/python2.7/site-packages/MySQLdb/cursors.py", line 226, in execute 
Oct 18 16:30:46 treeherder-stage app/release.6181:     self.errorhandler(self, exc, value) 
Oct 18 16:30:46 treeherder-stage app/release.6181:   File "/app/.heroku/python/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler 
Oct 18 16:30:46 treeherder-stage app/release.6181:     raise errorvalue 
Oct 18 16:30:46 treeherder-stage app/release.6181: django.db.utils.IntegrityError: (1062, "Duplicate entry '38ccf415-93e8-4911-939f-41b0bb44d043/0-77' for key 'text_log_summary_job_guid_1e839aa49a62d0a1_uniq'") 

I could try to figure this out, but am tempted to just skip it. Does that seem reasonable James? I imagine you're thinking of just getting rid of this table entirely.

Comment 19

[James Graham [:jgraham]]

Yes, I plan to delete the entire table in time; feel free to ignore it for now.

Comment 20

[William Lachance (:wlach)]

Let's mark this as fixed.