Push ingestion can currently occurs via any of: * Our REST API * Hg json-pushes polling (will be replace by Pulse in bug 1266559) * Pulse (for GitHub pushes via TC) As I understand it, we want to eventually move to a world where all push/job submissions happen via Pulse. Whilst job submissions is likely a further way out, push ingestion seems easier to do sooner, since most non-taskcluster submitters will be running jobs on repos that are already running taskcluster/buildbot jobs. For those that aren't, we can just add their Hg repo to Treeherder ourselves, or for GitHub ask TC to check for changes to their repo too. I think it would help with the ORM refactoring if we have fewer ingestion code-paths, so if we can remove this, I think we should. Looking at insights I see two people submitting via our API: https://insights.newrelic.com/accounts/677903/explorer/events?eventType=Transaction&timerange=week&filters=%255B%257B%2522key%2522%253A%2522request%252emethod%2522%252C%2522value%2522%253A%2522POST%2522%257D%252C%257B%2522key%2522%253A%2522appName%2522%252C%2522value%2522%253A%2522SCL3%2520treeherder-prod%2522%257D%252C%257B%2522key%2522%253A%2522response%252estatus%2522%252C%2522value%2522%253A%2522200%2522%257D%252C%257B%2522key%2522%253A%2522name%2522%252C%2522value%2522%253A%2522WebTransaction%252FFunction%252Ftreeherder%252ewebapp%252eapi%252eresultset%253AResultSetViewSet%252ecreate%2522%257D%255D&facet=name gaia-taskcluster and servo-perf (and only 21 submissions in the last 7 days between them). Now given B2G is EOL, we don't have to worry about breaking them, and for servo-perf, perhaps we can just get Taskcluster to send Pulse messages for their GitHub repo too? Will/Cameron, thoughts?
gaia-taskcluster is apparently being used for bugzilla resultset creation on prod, rather than just B2G related things. However they are planning on switching soon (we should give them a nudge in the meantime).
FYI, I don't think this is needed for conversion to the ORM (everything pretty much goes through the jobsmodel, which is all I'm modifying). I agree it would be nice to clean this up though.
There are no longer any resultset submissions via the REST API, so safe to remove it. https://insights.newrelic.com/accounts/677903/explorer/events?eventType=Transaction&filters=%255B%257B%2522key%2522%253A%2522name%2522%252C%2522value%2522%253A%2522resultset%2522%252C%2522like%2522%253Atrue%257D%252C%257B%2522key%2522%253A%2522request%252emethod%2522%252C%2522value%2522%253A%2522POST%2522%257D%255D&duration=2592000000&facet=name
Assignee: nobody → emorley
No longer blocks: 1306674
Priority: -- → P2
Created attachment 8860079 [details] [review] [treeherder] mozilla:rm-resultset-posts > mozilla:master
Commits pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/cc1fd9cc5713a21ad2bfa2c4f9e0b7c396062b8e Bug 1306844 - Remove unused attributes from push ETL data structures The ETL process creates intermediate push objects which are passed to `_store_push()` via `store_result_set_data()`. However many attributes were being passed in the dict but never used by `_store_push()`. https://github.com/mozilla/treeherder/commit/20ce1497bde6f577815f255feb30e1f8b70a4757 Bug 1306844 - Stop using TreeherderResultSetCollection for pushlog ETL Since `TreeherderResultSetCollection` is going to be removed shortly, and the use here is not necessary - but instead a leftover from the days of submitting pushlog data to our own REST API. The `.validate()` functionality is not required, since the Django model already performs validation. https://github.com/mozilla/treeherder/commit/f3a286709ddaf2cfde6205bb46e252c61fe6b659 Bug 1306844 - Simplify pushlog transformation during ingestion Previously `pushes` was iterated to create a transformed pushes list, which was iterated again to submit each push within. Now the list of pushes is only iterated once, with just one push being transformed and stored at a time. In addition, each push is stored by calling `store_push()` directly, to save having to pass a single element list to `store_result_set_data()`. https://github.com/mozilla/treeherder/commit/55650d582dd0b25d212a09c0d2712b711c1562bd Bug 1306844 - Remove support for submitting pushes via the REST API This endpoint is deprecated in favour of Pulse submissions, and is no longer being used.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.