Whilst it sounded all nice from a "we can have our ETL nodes running on a completely separate system" point of view, having the ETL tasks hit the external API rather than interacting with the DB directly adds additional load and means heavy-load issues with the API cause failure in our own ingestion, and not just external submitters. IMO we should make as many of our internal tasks interact with the DB directly and not have the additional round-trip of hitting our webheads to go via the API. We'll obviously need to make sure we still have enough tests for eg the python-client, since we'll not be using it internally, but IMO it will be worth it. Note also that the idea of "let's have another webhead just for internal requests" won't work on Heroku, where you cannot differentiate between webheads like that (and worker nodes cannot open web-facing ports).
Priority: P3 → P2
Tweaking summary so this is easier to find (just took ages).
Summary: Make ETL tasks submit directly to the DB instead of using our external API → Make ETL tasks submit directly to the DB instead of making requests to the API
Summary: Make ETL tasks submit directly to the DB instead of making requests to the API → Make ETL tasks interact directly to the DB instead of making requests to the API
Bug 1273231 made the log parser interact directly with the DB when updating the parse status of logs, which was a big win from a "requests Treeherder makes to itself per minute" point of view. Another massive contributor is the log parser querying the bugscache for bug suggestions, which is currently the #3 most frequent type of API request (at 120-200 requests per minute): https://rpm.newrelic.com/accounts/677903/applications/4180461/transactions#id=5b225765625472616e73616374696f6e2f46756e6374696f6e2f747265656865726465722e7765626170702e6170692e726566646174613a427567736361636865566965775365742e6c697374222c22225d&sort_by=throughput
Depends on: 1273231
Depends on: 1280761
I believe this is the only reason we have the (IMO rather silly) bugscache API, so we can probably go ahead and kill it after this is done. I would just convert the searching logic there into a simple function called inside `treeherder/model/error_summary.py`.
bc uses it I believe, though there's likely in underlying better way to solve the use case
Though looks unused now, perhaps after Cameron added the ability to submit your own text log summary json blob, but still have bug suggestions generated. https://github.com/mozilla/autophone/search?utf8=%E2%9C%93&q=bugscache_uri
No longer depends on: 1280761
Summary: Make ETL tasks interact directly to the DB instead of making requests to the API → Interact directly with the DB instead of making HTTP requests to our own API
(In reply to Ed Morley [:emorley] from comment #5) > Though looks unused now, perhaps after Cameron added the ability to submit > your own text log summary json blob, but still have bug suggestions > generated. > > https://github.com/mozilla/autophone/search?utf8=%E2%9C%93&q=bugscache_uri Filed bug 1281031 & opened a PR to remove the unused variable. Checking the prod weblogs shows no other consumers apart from Treeherder itself, so once bug 1280910 lands we can remove the bugscache API.
Priority: P2 → P3
Summary: Interact directly with the DB instead of making HTTP requests to our own API → [Meta] Interact directly with the DB instead of making HTTP requests to our own API
Depends on: 1281056
Depends on: 1281058
We're no longer hitting our own API \o/ :-)
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
Depends on: 1285925
Depends on: 1286205
You need to log in before you can comment on or make changes to this bug.