Closed Bug 1150287 Opened 5 years ago Closed 3 years ago

Publish github events to pulse using org-level hooks (proposal)

Categories

(Taskcluster :: Services, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jonasfj, Unassigned)

References

Details

We should make a bridge that can handle github organization-level hooks, and
publish events about PRs, pushes, revisions, etc. to pulse.

1) Find github org-level APIs and figure out what we can get and how
2) Design routing key pattern and JSON payload pattern for pulse
3) Implement this
4) Deploy

I suspect it's okay to hardcode the organizations to monitor in to the
configuration of this component. We won't update it very often anyways.
So no need to make APIs for adding a new githug-org and developing a UI.

I know jlal has looked at this before.

@jlal do we have to have all TC integration in one repo, or is
it perhaps beneficial to split out the pulse bridge so other things
can listen for messages too.

Thoughts, anyone?
> @jlal do we have to have all TC integration in one repo, or is
> it perhaps beneficial to split out the pulse bridge so other things
> can listen for messages too.

I think it probably makes sense to roll this into mozilla-taskcluster... We have support there for retries/exponential back off of jobs/etc... (which we would probably want?)

I like the direction we are heading through with bridge / hooks we can implement more and more of this logic in lower level "decision" tasks (maybe these are "boostrapping" tasks :shrug:)
Hmm... I didn't know we were doing github things in mozilla-taskcluster, I thought that was mainly
a pushlog -> taskcluster -> treeherder thing.

I guess adding a: github -> taskcluster -> github/treeherder, flow... makes sense, so all our
treeherder logic lives in the same place.

Ideally, I would have preferred these to be separate components. Ie. treeherder integration just
picked up messages based on routing key, etc... I think we failed this model because we needed
to generate the treeherder revision_hash identifier thingy.

Anyways, I don't care where all these integration points live. Keeping the separate would make them
more reusable, but I'm not sure it's easy to do.
Summary: github-pulse: Publish github events to pulse using org-level hooks (proposal) → mozilla-taskcluster: Publish github events to pulse using org-level hooks (proposal)
One of the things I need to do in Q2 is move that logic into mozilla-taskcluster (from the super old gaia-taskcluster stuff). It's nice to have all this together so it can be tested together/etc... Setting up good end to end testing for treeherder/github/etc... is PITA I don't want to do it in more then one spot if we can avoid it (and yes I don't want to ship something without these end-to-end tests).
Component: TaskCluster → General
Product: Testing → Taskcluster
Component: General → Integration
Component: Integration → Github
QA Contact: jwatkins
Summary: mozilla-taskcluster: Publish github events to pulse using org-level hooks (proposal) → Publish github events to pulse using org-level hooks (proposal)
I think taskcluster-github has been doing this for a while:
https://docs.taskcluster.net/reference/core/github/references/events
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Component: Github → Services
You need to log in before you can comment on or make changes to this bug.