Open Bug 1930722 Opened 22 days ago Updated 15 days ago

Test migrating GitHub Webhook to Splunk to new API gateway, and add other mozilla orgs to webhooks

Categories

(mozilla.org :: Github: Administration, task)

Product:
mozilla.org
Component:
Github: Administration
Type:
task

Tracking

(Not tracked)

People

(Reporter: mlarsonweber, Unassigned)

Details

User Story

When Gene departed Mozilla he asked us to look into taking over the Github Webhooks to Splunk stream which was hosted in the Devservices AWS account. Based on available documentation, I've recreated the setup in our Infosec Prod AWS account, but need to test the API gateway trigger to the lambda function to ensure it's been set up correctly. Once we have it working we can decommission the old setup. We'd also like to look into adding other Mozilla orgs into this webhook stream, as it provides more logging than our Github Enterprise integration, but we currently only get webhook events from the Mozilla org.
No description provided.

So - Is this request to setup a second, parallel webook to the one in play in the mozilla org so you can test and compare? Or something else?

Also, as to the future - there's MANY orgs (60+), so I have concerns about the long term maintainability of an org-by-org solution.

Let me know how you want to proceed with your testing.

Slacking with Mackenzie - setting up a new hook for testing.

I'll need -
the payload URL, the content type, the secret (I assume there’s a secret) and that should be it.

The current hook appears to be set to get ALL events.

Here's the endpoint https://5l6tmsu5ye.execute-api.us-west-2.amazonaws.com/prod/lambda-receive, there shouldn't be a secret required (though I'd be open to discussing adding one once we get this working); content type application/json

Alright - created the webhook.

It says that it sent a ping event to test things out - I'm guessing it won't fully activate until that's handled somehow?

https://docs.github.com/webhooks/#ping-event

Flags: needinfo?(mlarsonweber)

Hey Chris, I spent a bit of time trying to look into why we might be getting 500s, and tried some testing on my own and was getting different errors. Do you think we might be able to try sending the webhook data directly to Splunk? I'd give you a token and you can use that to send events as long as they can be formatted like this: https://docs.splunk.com/Documentation/Splunk/latest/Data/FormateventsforHTTPEventCollector does that sound like it might work? If so I can get you the token and endpoint to test

Flags: needinfo?(mlarsonweber)

So, all I have is the elements that GitHub gives me to work with, so I don't think that's going to be possible - but I DO have the raw data that github is giving the hook - I have been giving you the response data - as I was hoping that would be enough - but I can extract several of the events and hand that data to you to dissect and figure out what you're concerned with.

hmm you think you could set up continuous delivery of the raw events?

I think we're talking past each other - I'm talking about giving you the specific data that GitHub has handed your hook - (pretty sure it's JSON) - so the hook IS just delivering that to your hook.

I'm not very available today - but we could setup some time to zoom so I could more accurately show you what I'm talking about.

(I'll ping if time becomes available today - else feel free to throw something at my calendar.)

Sounds good, I'll find a time this week for us to sync!

You need to log in before you can comment on or make changes to this bug.