785856 - Upgrade to Celery 3.0.23

Status: RESOLVED FIXED

(support.mozilla.org :: Code Quality, task, P2)

Description

[James Socol [:jsocol, :james]]

The latest celery version is 3.0.7. I'm not sure what the corresponding Kombu version is. This may address issues we've had with zombie celery processes (see bug 746854).

I'm not sure what the 2.5.x -> 3.0.x upgrade is like, so I won't estimate it here, but I'm happy to do it if time is hard to come by.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm not sure this is relevant anymore for SUMO. Between when bug 746854 was created and now, we upgraded celery to 2.5.3 (bug 757444). A lot of our celery problems went away after that. I don't remember having problems with zombie processes.

Ricky: ^^^ do you?

I know I looked into this recently and I vaguely remember there being API changes between celery 2.5 and 3.

I think the latest Kombu is 2.4.3: http://pypi.python.org/pypi/kombu

Also, I have a related playdoh issue: https://github.com/mozilla/playdoh/issues/125

Comment 2

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Will Kahn-Greene [:willkg] from comment #1)
> I don't remember having problems with zombie processes.
> 
> Ricky: ^^^ do you?

Nope, it has been working great since that last upgrade.

Comment 3

[James Socol [:jsocol, :james]]

Fair enough--still worth doing sooner than later but no rush. The 2.x series isn't being developed anymore, afaict.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

They're up to 3.0.23 now. Updating the summary.

We really should update to 3.something since 2.something isn't supported anymore and hasn't been in a while.

Putting this in the backlog. It'd be good to do, but I bet it's going to be a project.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

They're up to 3.1.8 now.

I did some work on this for playdoh and then backpedaled and did a minor upgrade to 2.5.something because it was crazypants. Lot of API changes.

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

We are on an old and unsupported celery. Bugs such as bug 966548 are starting to creep up on us.

As :willkg said, this will be painful. But we have to do it eventually.

Filing as a 3pter for now, but it may be more or require spinoff bugs.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pretty sure the new versions of celery require an additional binary requirement. Whoever works on this should check to see if that's true and if it is, write up a bug for webops to install that on the environments.

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Landed on master:
https://github.com/mozilla/kitsune/commit/f1f14c69846d22d81d4dcae2d4214eef05dee303

The pull request was here:
https://github.com/mozilla/kitsune/pull/1817

Comment 9

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Will Kahn-Greene [:willkg] from comment #7)
> Pretty sure the new versions of celery require an additional binary
> requirement. Whoever works on this should check to see if that's true and if
> it is, write up a bug for webops to install that on the environments.

To follow up on this... librabbitmq is a optional binary requirement that we should probably switch to after we are done upgrading celery to the latest greatest. billiard also has optional binary parts.

Comment 10

[Ricky Rosario [:rrosario, :r1cky]]

This was deployed to prod now. I'll file bugs to upgrade to 3.1.x next :)