Remove support for submitting pushes (resultsets) via the REST API



Tree Management
Treeherder: API
2 years ago
a year ago


(Reporter: emorley, Assigned: emorley)


(Blocks: 1 bug)



(1 attachment)



2 years ago
Push ingestion can currently occurs via any of:
* Hg json-pushes polling (will be replace by Pulse in bug 1266559)
* Pulse (for GitHub pushes via TC)

As I understand it, we want to eventually move to a world where all push/job submissions happen via Pulse.

Whilst job submissions is likely a further way out, push ingestion seems easier to do sooner, since most non-taskcluster submitters will be running jobs on repos that are already running taskcluster/buildbot jobs. For those that aren't, we can just add their Hg repo to Treeherder ourselves, or for GitHub ask TC to check for changes to their repo too.

I think it would help with the ORM refactoring if we have fewer ingestion code-paths, so if we can remove this, I think we should.

Looking at insights I see two people submitting via our API:

gaia-taskcluster and servo-perf (and only 21 submissions in the last 7 days between them).

Now given B2G is EOL, we don't have to worry about breaking them, and for servo-perf, perhaps we can just get Taskcluster to send Pulse messages for their GitHub repo too?

Will/Cameron, thoughts?

Comment 1

2 years ago
gaia-taskcluster is apparently being used for bugzilla resultset creation on prod, rather than just B2G related things.

However they are planning on switching soon (we should give them a nudge in the meantime).
FYI, I don't think this is needed for conversion to the ORM (everything pretty much goes through the jobsmodel, which is all I'm modifying). I agree it would be nice to clean this up though.


a year ago
Blocks: 1349182
Created attachment 8860079 [details] [review]
[treeherder] mozilla:rm-resultset-posts > mozilla:master


a year ago
Attachment #8860079 - Flags: review?(cdawson)
Attachment #8860079 - Flags: review?(cdawson) → review+

Comment 5

a year ago
Commits pushed to master at
Bug 1306844 - Remove unused attributes from push ETL data structures

The ETL process creates intermediate push objects which are passed to
`_store_push()` via `store_result_set_data()`. However many attributes
were being passed in the dict but never used by `_store_push()`.
Bug 1306844 - Stop using TreeherderResultSetCollection for pushlog ETL

Since `TreeherderResultSetCollection` is going to be removed shortly,
and the use here is not necessary - but instead a leftover from the days
of submitting pushlog data to our own REST API.

The `.validate()` functionality is not required, since the Django model
already performs validation.
Bug 1306844 - Simplify pushlog transformation during ingestion

Previously `pushes` was iterated to create a transformed pushes list,
which was iterated again to submit each push within. Now the list of
pushes is only iterated once, with just one push being transformed and
stored at a time.

In addition, each push is stored by calling `store_push()` directly, to
save having to pass a single element list to `store_result_set_data()`.
Bug 1306844 - Remove support for submitting pushes via the REST API

This endpoint is deprecated in favour of Pulse submissions, and is no
longer being used.


a year ago
Last Resolved: a year ago
Resolution: --- → FIXED


a year ago
Blocks: 1387711
You need to log in before you can comment on or make changes to this bug.