1078522 - Update BuildBot Pulse publisher to use mozillapulse 1.0

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Mark Côté [:mcote]]

As part of the Pulse security update, the BuildBot Pulse publisher should be updated to use mozillapulse 1.0.  BuildBot already uses its own Pulse account, but it should be updated to start publishing to the new exchange format (exchange/build/ instead of org.mozilla.exchange.build).  The timing isn't critical, since we have already bound the old exchange to the new one, but it should be no earlier than Tuesday, Oct 14, since we need to give consumers a chance to upgrade.

BuildBot is still using a very old version of mozillapulse (.4).  There have been a few significant changes since then, particularly switching to kombu and enabling ssl.  To avoid breaking too many things at once, we should probably upgrade in steps.

Kombu was introduced in mozillapulse 0.80, though 0.90 just has a small fix so we could upgrade to that first.  Latest kombu is 3.0.23 which in turn requires amqp (not, apparently, amqplib), the latest of which is 1.4.5.

SSL support by default was introduced in mozillapulse 0.93.  After applying that version and restarting, if you don't add any extra config parameters to your publisher code, you'll connect automatically via ssl on the ssl port (5671 for Pulse).

Finally, 1.0 was released today, with the main new feature being the updated exchange names.

Upgrading to 0.90 and 0.93 can be done at any time.

Comment 1

[Chris AtLee [:catlee]]

I think I'd rather set 1.0 as the goal than doing this in steps. We're going to have to get all the python packages mirrored locally first, and jumping to our desired end state means less churn with package mirroring.

Comment 2

[Mark Côté [:mcote]]

Sure, completely up to you. :)

Comment 3

[Mark Côté [:mcote]]

A pip freeze of a fresh install of MozillaPulse 1.2.1 gives me this:

amqp==1.4.6
anyjson==0.3.3
kombu==3.0.26
MozillaPulse==1.2.1
pytz==2015.6
wheel==0.24.0

Although wheel isn't required; it's just installed by default in virtualenvs now.

Comment 4

[Mark Côté [:mcote]]

It would be very good to do this before we move to Pulse in the cloud next week, if there's any way to make that happen.

Comment 5

[Chris AtLee [:catlee]]

Ok, need to make sure all those packages are in our internal pypi first.

Comment 6

[Chris AtLee [:catlee]]

From what I can tell, the only changes to our existing virtualenvs would be:
- MozillaPulse 0.80 -> 1.2.1
- anyjson 0.3 -> 0.3.3
- argparse 1.1 -> 1.2.1
- pytz 2011d -> 2015.6

carrot, and amqplib are no longer required.

the only change there that could have possible impact to other code is the argparse update. I'll dig into that a bit more today.

Comment 7

[Chris AtLee [:catlee]]

published MozillaPulse 1.2.1, argparse 1.2.1 and pytz 2015.6 on our pypi.

Comment 8

[Chris AtLee [:catlee]]

Ran a test of pulse_publisher with the updated libraries, and everything worked fine.

Comment 9

[Chris AtLee [:catlee]]

Attachment: Update MozillaPulse for buildbot queues

Comment 10

[Chris AtLee [:catlee]]

Attachment: Update MozillaPulse for buildbot masters

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 8669951 [details] [diff] [review]
Update MozillaPulse for buildbot masters

SSL is the new default now https://hg.mozilla.org/automation/mozillapulse/file/e6d10b16c666/mozillapulse/config.py#l17

The flows look ok:
[[e-mail] ~]$ nc -vz pulse.mozilla.org 5671
Connection to pulse.mozilla.org 5671 port [tcp/amqps] succeeded!

Comment 12

[Chris AtLee [:catlee]]

found a small typo in the puppet patch: anyjson==1.3.3 should be 0.3.3.

also, our puppet manifests don't restart the queue services when updating the virtualenv.

Comment 13

[Chris AtLee [:catlee]]

Attachment: Update MozillaPulse for buildbot queues

fixes typo, and ensures queue processors are restarted after the virtualenv changes

Comment 14

[Chris AtLee [:catlee]]

all pulse publishers have been upgraded

Comment 15

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/8a94eb6d73f7