1484192 - Allow Pulse ingestion to work in development without Pulse credentials

Status: NEW

(Tree Management :: Treeherder, enhancement, P3)

Description

[George Hickman]

Broken out from: https://bugzilla.mozilla.org/show_bug.cgi?id=1281821#c15

Guest access to Pulse would allow a dev setup to work out of the box and simplify how we build Pulse connections.

Comment 1

[Ed Morley [:emorley]]

I'm able to see Pulse events as an unauthenticated user when visiting https://tools.taskcluster.net/pulse-inspector - which made me hopeful that there perhaps was already guest support for pulse.mozilla.org. However it is using an API on https://events.taskcluster.net, whose source is here:
https://github.com/taskcluster/taskcluster-events

And which itself is using credentials:
https://github.com/taskcluster/taskcluster-events/blob/94538808933c76f907d81ce87bc97e51907b7641/config.yml#L19-L20

So it seems like our options for making pulse ingestion work "out of the box" for people developing Treeherder locally, are:
1) Add support for guest accounts (with non-durable queues and abuse protection) on pulse.mozilla.org
2) Try and consume events from events.taskcluster.net (but will require some kind of conversion given our ingestion expects to be interacting with a rabbitmq server)
3) Come up with some way of making creating pulse credentials and setting them locally easier (eg some kind of automation/script we can trigger from the development environment)

It does seem like this should be a solved problem though - there are lots of other teams interacting with pulse who presumably need to have it working locally too.

Dustin, have the Taskcluster projects explored any ideas in this area to improve development UX for pulse consumers?

Comment 2

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #1)
> Dustin, have the Taskcluster projects explored any ideas in this area to
> improve development UX for pulse consumers?

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Yes, a bit.  We have two services you could use:

 * https://github.com/taskcluster/taskcluster-events - public read access to pulse messages over websockets
 * https://github.com/taskcluster/taskcluster-pulse - trade taskcluster credentials for pulse credentials

The first is similar to (2) in your list.  You're welcome to use it, but you're right that it's not AMQP.  The second is similar to your (1) but requires taskcluster credentials, which are a little harder to come by than pulse credentials.

For us, we just recommend users get themselves a pulse account (at pulseguardian.mozilla.org) and use that.  That hasn't caused any difficulties.  Maybe the error message can suggest that?

Comment 4

[Ed Morley [:emorley]]

Waiting on possible support being added to pulse guardian (see bug 1508741).

Comment 5

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #4)

Waiting on possible support being added to pulse guardian (see bug 1508741).

Unfortunately the previous maintainer of Pulse no longer works at Mozilla as of this month, so that bug (which was a pre-requisite for the preferred approach in this bug) is on hold for now.

Comment 6

[Armen [:armenzg]]

I believe setting up the Pulse credentials + the current documentation is easy enough, thus, I believe we could close this one.