1031597 - Tree Status should send pulse notifications that the tree is opening or closing

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Seth Fowler [:seth] [:s2h]]

It would be *very* useful if bots like pulsebot could notify us if the tree is opening or closing. This requires that Tree Status send pulse notifications for these events.

Comment 1

[Allan [:asilva]]

Hello Dustin,
Is this bug still needed?
I can try develop this feature, can you provide-me more instructions?

Thanks

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Sorry for the delay -- I was away.

Yes, I think this would still be useful!

I'm not so sure I'm a good mentor for the project, though.  Rok is now the maintainer of treestatus, so perhaps he can provide a bit of guidance for getting started.

Comment 3

[Gregory Szorc [:gps]]

If using Python, the code for sending Pulse events is pretty simple. Just install "kombu" and steal the code from https://hg.mozilla.org/hgcustom/version-control-tools/file/tip/pylib/vcsreplicator/vcsreplicator/pulsenotifier.py or https://hg.mozilla.org/hgcustom/version-control-tools/file/tip/github-webhooks/lambda_pulse.py. The latter is an EC2 Lambda handler and is pretty simple to deploy and trigger.

Comment 4

[Allan [:asilva]]

Thanks, Dustin, Gregory.

I will take a look in the mercurial documentation then I going back to this bug. So far I used only the github haha.

Some questions:
-Where can I check the status of the tree?
-The pulsebot is the another python script, right?

Comment 5

[Gregory Szorc [:gps]]

I'm not sure where Mercurial comes into play here: I thought the tree status API lived in Git as part of Treeherder?

I also don't know where the Pulse code would live. It could either be inline in the Tree Status API server component. Or, the Tree Status API server could trigger code living elsewhere. I know little about the Tree Status API and what solution would be preferred. Hint: writing generic Python code that can be imported as a Python module could be used in pretty much any scenario.

Comment 6

[Allan [:asilva]]

(In reply to Gregory Szorc [:gps] from comment #5)
> I'm not sure where Mercurial comes into play here: I thought the tree status
> API lived in Git as part of Treeherder?
> 
> I also don't know where the Pulse code would live. It could either be inline
> in the Tree Status API server component. Or, the Tree Status API server
> could trigger code living elsewhere. I know little about the Tree Status API
> and what solution would be preferred. Hint: writing generic Python code that
> can be imported as a Python module could be used in pretty much any scenario.

Ah, I saw a link from hg.mozilla and I thought that the repository was the mercurial.
I'm a relatively new contributor, I will research better.

Comment 7

[Rok Garbas [:garbas]]

:ex-dev: Hi, tomorrow - Tue, 31.01.17 - I'm pushing new TreeStatus live. That means redirecting/proxying from current url[1] to new url[2]. The code has also moved to a new location[3] and if I understand this ticket correctly then you are interested in this line[4].

Instructions how to get started with any project in github.com/mozilla-releng/services is described in documentation[5]. I am aware that current documentation should/must be improved and would gladly write pieces of the documentation that you find missing or unclear.

Feel free to also ping me on IRC (nickname: garbas) if you get stuck.


[1] http://api.pub.build.mozilla.org/treestatus
[2] http://mozilla-releng.net/treestatus
[3] https://github.com/mozilla-releng/services/tree/master/src/releng_treestatus
[4] https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L39
[5] https://docs.mozilla-releng.net/

Comment 8

[Allan [:asilva]]

(In reply to Rok Garbas [:garbas] from comment #7)
> :ex-dev: Hi, tomorrow - Tue, 31.01.17 - I'm pushing new TreeStatus live.
> That means redirecting/proxying from current url[1] to new url[2]. The code
> has also moved to a new location[3] and if I understand this ticket
> correctly then you are interested in this line[4].
> 
> Instructions how to get started with any project in
> github.com/mozilla-releng/services is described in documentation[5]. I am
> aware that current documentation should/must be improved and would gladly
> write pieces of the documentation that you find missing or unclear.
> 
> Feel free to also ping me on IRC (nickname: garbas) if you get stuck.
> 
> 
> [1] http://api.pub.build.mozilla.org/treestatus
> [2] http://mozilla-releng.net/treestatus
> [3]
> https://github.com/mozilla-releng/services/tree/master/src/releng_treestatus
> [4]
> https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/
> releng_treestatus/api.py#L39
> [5] https://docs.mozilla-releng.net/

Many thanks, Rok.
I will take a look in all links and contact you.

Comment 9

[Rok Garbas [:garbas]]

:ex-dev:

I released new TreeStatus this morning[1].

To make things easier for you I created a branch[2] and added "kombu" package (as suggested by gps) to releng_treestatus.

I'm currently struggling with writing documentation, which wouldn't be too long and too short. For this reason here are the steps that are needed to get up a development environment.

```
0. Install Nix

% sudo mkdir -m 0755 /nix
% sudo chown $USER /nix
% curl https://nixos.org/nix/install | sh
% mkdir -p /etc/nix
% echo 'binary-caches = https://s3.amazonaws.com/releng-cache/
https://cache.nixos.org/' > /etc/nix/nix.conf


1. checkout the correct branch

% git clone [e-mail]:mozilla-releng/services
% cd services
% git checkout -b bug-1031597 origin/master

2. enter development shell / environment (like virtualenv)

% make develop APP=releng_treestatus
(nix-shell) % ...

3. run releng_treestatus in development mode

% make develop-run APP=releng_treestatus
...

Then point your browser to https://localhost:8003.

You might need to accept a certificate since it was locally generated,
but then you will see an API documentation.

4. Running tests (eg. before creating PR)

% make build APP=releng_treestatus
```

I hope that help you and I would love to hear what you find hard to understand or which part you found most confusing.


[1] https://mozilla-releng.net/treestatus
[2] https://github.com/mozilla-releng/services/tree/bug-1031597

Comment 10

[Allan [:asilva]]

Thanks, :garbas

I'm working in a balrog bug now and probably next weekend I can start this.

I saw a little of the project structure, and thought to start in the following way:

-Create a pulse notifier in https://github.com/ex-Dev/services/tree/master/lib/releng_common/releng_common (a commom location like said :gps)

-Publish a message from here, after session.commit: https://github.com/ex-Dev/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L86

What you think?

Then I will need informations like: Exchange, routing_key(or a repo) - and the json structure too.

Comment 11

[Rok Garbas [:garbas]]

:ex-dev sounds good, I'm just making sure you have everything to start the work.

I never had the chance to work with Pulse (yet) but here is my understanding of it:

1. Exchange: Get it from flask config

    from flask import current_app

    def api_call(..):
        ...
        session.commit()
        exchange = current_app.config.get('PULSE_EXCHANGE')

2. Routing key: /<tree>/status_change

3. Json structure: I think you will need to check if status actually changed and only publish a message when status changed. I suppose it would be useful to know from/to status and probably it wont hurt to include other data (reason, motd, ...)

Comment 12

[Allan [:asilva]]

Hello,
:gps, :garbas, I started write a pulse notifier module. But I have a question, for now:

Does the pulse message must follow the format bellow?

        data = {
            'payload': payload,
            '_meta': {
                'exchange': exchange,
                'routing_key': routing_key,
                'serializer': 'json',
                'sent': datetime.datetime.utcnow().isoformat(),
            }
        }

Or are this format specific to " pylib/vcsreplicator/vcsreplicator/pulsenotifier.py" example?

Comment 13

[Allan [:asilva]]

(In reply to Allan [:ex-dev] from comment #12)
> Hello,
> :gps, :garbas, I started write a pulse notifier module. But I have a
> question, for now:
> 
> Does the pulse message must follow the format bellow?
> 
>         data = {
>             'payload': payload,
>             '_meta': {
>                 'exchange': exchange,
>                 'routing_key': routing_key,
>                 'serializer': 'json',
>                 'sent': datetime.datetime.utcnow().isoformat(),
>             }
>         }
> 
> Or are this format specific to "
> pylib/vcsreplicator/vcsreplicator/pulsenotifier.py" example?

I have another question: -Who is reponsible to create the exchange, queues, binding to queues, etc ?

_
If you want follow the current progress before PR, see:
https://github.com/ex-Dev/services/blob/bug-1031597/lib/releng_common/releng_common/pulse.py

https://github.com/ex-Dev/services/blob/bug-1031597/src/releng_treestatus/releng_treestatus/api.py#L150

https://github.com/ex-Dev/services/blob/bug-1031597/src/releng_treestatus/releng_treestatus/api.py#L41

Comment 14

[Rok Garbas [:garbas]]

:ex-dev: I never worked with Pulse as well, but I will find somebody today that can answer this question.

Comment 15

[Rok Garbas [:garbas]]

:ex-dev: I looked more closely at instructions of Pulse[1] and it looks your message follow the bullet points. Also I looked at a few implementation and looks like a ``_meta`` became an unwritten rule which some people follow.

Could you submit a PR with current work, so that I can comment on specific lines?

Once we have a PR ready we can ask somebody with more experience (like :gps:) to give final approval.


[1] https://wiki.mozilla.org/Auto-tools/Projects/Pulse#Appendix_A:_Everything_in_Bullet_Points

Comment 16

[Allan [:asilva]]

Ok, I'll put pulse configurations in config file.
I think that I understood the real use case, I'll try a integrated test with pulse guardian, Then I open the PR.

Comment 17

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Note that gps is out until March 20, so you'll probably need to find someone else.

Comment 18

[Rok Garbas [:garbas]]

:ex-dev: :+100:, sounds great!
:KWierso: oh thank you! Will find somebody else to do final review

Comment 19

[Allan [:asilva]]

Attachment: PR for review

Comment 20

[Allan [:asilva]]

Attachment: pulse config.zip

Hello, Rok!
I opened a PR and attached here my pulse configurations that I use on my tests.

Comment 21

[Rok Garbas [:garbas]]

:ex-dev

This week was a busy one for me and this is the reason it took longer then usual to get changes released. But yesterday new release of mozilla-releng/services was pushed to production (release notes[1]).

I made quick video explaining the feature here: https://vimeo.com/206449114

Thank you!!!!!


[1] https://github.com/mozilla-releng/services/releases/tag/v9

Comment 22

[Allan [:asilva]]

Great!
Thanks Rok!

Comment 23

[Allan [:asilva]]

Hey Rok,
I reviewed the code for this bug, and just now I noticed that maybe the following import is not necessary:

https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L19

We can just use the already imported current_app in these lines:

https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L44
https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L46
https://github.com/mozilla-releng/services/blob/master/src/releng_treestatus/releng_treestatus/api.py#L64

What you think?

Comment 24

[Rok Garbas [:garbas]]

:ex-dev oh yes, +100, that probably skipped my radar when I was reviewing the code.