1536853 - Migrating pulselistener from Heroku to CloudOps

Status: NEW

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Bastien Abadie [:bastien]]

PulseListener is a tool built by the Release Management Team, using release-services, currently running on Heroku using a worker dyno.

One recently added feature needs a local Mozilla Central clone, and this is hard to do efficiently on Heroku (memory limitations, container restart every day, no disk cache).

So i would like to run Pulselistener on GCP, like Releng scriptworkers.

As for releng, i would need 3 environments:

• testing, with docker tag mozilla/release-services:pulselistener-testing
• staging, with docker tag mozilla/release-services:pulselistener-staging
• production, with docker tag mozilla/release-services:pulselistener-production

These environments do not need to expose any port, but need access to the following services:

• Papertrail on *.papertrailapp.com
• Sentry on sentry.prod.mozaws.net
• Pulse server on pulse.mozilla.org
• Taskcluster APIs on *.taskcluster.net
• Phabricator on https://phabricator.services.mozilla.com & https://phabricator-dev.allizom.org/
• Mercurial server on https://hg.mozilla.org

The docker container needs Taskcluster credentials & secret to run like in the example below:

docker run \
	--env TASKCLUSTER_SECRET=project/relman/pulselistener-demo \
	--env TASKCLUSTER_CLIENT_ID=XXXX \
	--env TASKCLUSTER_ACCESS_TOKEN=YYYY \
	-v /path/to/pulselistener-cache:/cache \
	mozilla/release-services:pulselistener-testing \
	pulselistener --cache-root=/cache

Comment 1

[Sylvestre Ledru [:Sylvestre]]

Habib, Adrian, is that something you could help with? Thanks

Comment 2

[Habibullah Pagarkar (use Slack)]

Hi Sylvestre, we would love to take this on. However, the team is stretched to capacity at the moment and we have a few existing commitments we need to meet. What's the urgency on this front?

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Not crazy urgent, we just need to add this to your pipe :)
We can wait a couple weeks (but I don't know how long is your backlog) for this!
Thanks

Comment 4

[Habibullah Pagarkar (use Slack)]

Thanks for the clarification. I'm working with my management to see if we can get additional capacity to pick up work like this, but at the moment it looks like the earliest we can work on this would be after Q2.

Comment 5

[Marco Castelluccio [:marco]]

Would this be easier to do now? Is the process more streamlined?
We've been thinking of moving a few things from Heroku to cloudops to have better control over the performance characteristics, I'm trying to gauge the level of effort that would be required from us and whether it is something that you could help with or if your plates are still full as they were 2 years ago :)

Comment 6

[Marco Castelluccio [:marco]]

This is planned now for H1 2023.