844366 - sync: deploy hekad-0.1.1-2 and metlog-py-0.10.0-1 to syncstorage prod

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[Rob Miller [:rmiller]]

Attachment: hekad-0.1.1-2.x86_64.rpm

hekad and metlog-py have stood up well under functional testing and load testing in the sync staging environment, we're ready to have them in production. RPMs for both hekad and metlog-py are attached, as well as syncstorage.json config file (to be stored in /etc/mozilla-services/heka/syncstorage.json).

Comment 1

[Rob Miller [:rmiller]]

Attachment: metlog-py-0.10.0-1.noarch.rpm

Comment 2

[Rob Miller [:rmiller]]

Attachment: /etc/mozilla-services/heka/syncstorage.json

Comment 3

[Bob Micheletto [:bobm]]

Attachment: Heka puppet manifest diffs for production sync heka deploy

Proposed puppet manifest changes for production heka deploy

Comment 4

[Wesley Dawson [:whd]]

Comment on attachment 719195 [details]
Heka puppet manifest diffs for production sync heka deploy

You know what you did.

Comment 5

[Bob Micheletto [:bobm]]

Attachment: Heka puppet manifest diffs for production sync heka deploy: Now with operational fixes!

New and improved puppet manifest changes!

Comment 6

[Wesley Dawson [:whd]]

Comment on attachment 719248 [details]
Heka puppet manifest diffs for production sync heka deploy: Now with operational fixes!

It makes me angry, but go ahead and do it.

Comment 7

[Bob Micheletto [:bobm]]

Production change scheduled here: https://intranet.mozilla.org/Services/Ops/ChangeWindow_20130305

Comment 8

[James Bonacci [:jbonacci]]

I want to help out :bobm on this if there is time for some QA input/testing.

Comment 9

[James Bonacci [:jbonacci]]

Sorry for the delayed response!
Services QA signs off on this release in Stage.
Let's get this into Prod!

Comment 10

[James Bonacci [:jbonacci]]

Also, I want to add that the Stage testing was performed with the default configuration: single core.
We can open a new ticket and schedule more testing in Stage if we want to look at multi-core support.

Comment 11

[Bob Micheletto [:bobm]]

Change scheduled here: https://intranet.mozilla.org/Services/Ops/ChangeWindow_20130312

Comment 12

[Bob Micheletto [:bobm]]

Unable to deploy to production due to RPM conflict problem involving metlog that did not manifest during the stage deploy.  The Sync staging environment is presently in use, but will investigate this problem there when available.

Comment 13

[Rob Miller [:rmiller]]

Attachment: python26-metlog-py-0.10.0-1.noarch.rpm

Comment 14

[Rob Miller [:rmiller]]

Comment on attachment 724464 [details]
python26-metlog-py-0.10.0-1.noarch.rpm

correct spelling for the rpm file name

Comment 15

[Bob Micheletto [:bobm]]

python26-metlog-py-0.10.0-1.noarch.rpm has been pushed back to stage, and Sync has been restarted without incident.  Running a stage load test.

Comment 16

[James Bonacci [:jbonacci]]

:bobm how are those load tests in stage?

Comment 17

[James Bonacci [:jbonacci]]

I am going to assume this is long since done.
:rmiller or :bobm can you mark this ticket?

Comment 18

[Bob Micheletto [:bobm]]

:jbonacci Actually, it hasn't.  I am planning on rolling it out with this latest version of Sync 1.15-7 from Bug 854073 provided it passes QA.

Comment 19

[Bob Micheletto [:bobm]]

This has been deployed.

Comment 20

[James Bonacci [:jbonacci]]

Verified across PHX1 and SCL2.