1299223 - please deploy balrog 2.7 to stage/prod

Status: VERIFIED FIXED

(Cloud Services :: Operations: Deployment Requests - DEPRECATED, task)

Description

[[e-mail] (:bhearsum)]

Balrog version 2.7 is ready to be pushed to stage. Please deploy the new Docker image (master-2016-08-30-16-36).

We'd like the production push for this to happen sometime between 11am and 1pm pacific on Wednesday, August 31st.

This push requires a schema change that needs to be done _prior_ to the new code going out. Thas can be performed by running the Docker image with the "upgrade-db" command, with DBURI set.

This push will pick up https://bugzilla.mozilla.org/show_bug.cgi?id=1289178, which should improve monitoring for the admin app by updating a dummy table as part of the __heartbeat__ check.

Comment 1

[[e-mail] (:bhearsum)]

This was pushed to stage yesterday. Johan and I verified it this morning, and everything we could test looks good.

One thing we couldn't fully check was that the watchdog table was getting updated as part of __heartbeat__. Benson, can you run "SELECT * from dockerflow" to check that? It should be at least in the double digits by now, maybe the triple. If that looks right, we're ready to go to production here.

Comment 2

[Benson Wong [:mostlygeek]]

The stage dockerflow table has the value: "19". 
I think it would be ok to have __heartbeat__ simply return the value it wrote to the database. We really only monitor on the HTTP response status code. Anything other than 200 results in an alert. 

If it is something other than 200, the body should contain a reason (text/plain preferred) on the failure.

Comment 3

[Benson Wong [:mostlygeek]]

Production database updated with the dockerflow table.

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Benson Wong [:mostlygeek] from comment #2)
> The stage dockerflow table has the value: "19". 
> I think it would be ok to have __heartbeat__ simply return the value it
> wrote to the database. We really only monitor on the HTTP response status
> code. Anything other than 200 results in an alert. 

Good idea.

> If it is something other than 200, the body should contain a reason
> (text/plain preferred) on the failure.

Yeah, this should be the case already. Any exceptions that happen for __heartbeat__ will bubble back up as a 500.

Comment 5

[[e-mail] (:bhearsum)]

Thanks!