1230255 - Fix http2.mozqa.com

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

Per IRC discussion just now, :atoll to "fix" http2.mozqa.com (cross-reference: bug 1222147):

10:51 AM <stephend> atoll: gave jgmize et al the heads-up that HTTP/2 is here for CloudFlare, and they can enable it, at will, via their admin panel: https://blog.cloudflare.com/introducing-http2/
10:51 AM <@atoll> can you ask them on my behalf to enable it on the stage site?
10:52 AM <@atoll> (to maintain parity with the zeus deployment of bedrock-stage)
10:52 AM <stephend> sure can
10:52 AM — @atoll bows
10:54 AM <@atoll> stephend: file a bug for me to fix http2.mozqa.com? it might not be broken yet, but i probably should now
<snip>
10:55 AM <@atoll> i'll explain in bug

Comment 1

[:Atoll]

The previous VIP, http2-mozqa-zlb.vips.phx1.mozilla.com, was (knowingly) destroyed as part of the PHX1 migration. Rather than introducing new content into PHX1 External, I'm deploying it to SCL3 External instead.

Replaced http2-mozqa-zlb.vips.phx1.mozilla.com / 63.245.217.16 with http2-mozqa-zlb.vips.scl3.mozilla.com / 63.245.215.55 and altered the http2.mozqa.com CNAME in DNS.

Created virtual server http2.mozqa.com using TrafficScript to respond with 200 OK, header "X-Backend-Server: TS", content-type application/json, body {"http_protocol":"2.0"}, configured to permit http2 negotiation using the Mozilla SHA2 CA-signed SSL certificate we previously issued for this purpose.

Note that the value of http_protocol changes depending on the HTTP/x.y version used against the server. Typical values *should* include "HTTP/1.1", "HTTP/2", and so on. I honestly have no idea what you'll get for HTTP/0.9, but that's not really the point of this VIP, anyways.

Setting needinfo on :stephend to bring the client QA tests into compliance, and meanwhile resolving since we're all set up and documented here. I left notes on most everything pointing back to this bug as well.

http.sendResponse("200 OK", "application/json", "{'protocol':'" . http.getClientVersion() . "'}\n", "X-Backend-Server: TS");

Comment 2

[:Atoll]

Oh, and, to test this on OS X, I used:

$ brew install curl --with-nghttp2  # safe - does NOT install into $PATH anywhere

$ /usr/local/Cellar/curl/*/bin/curl --cacert ca_sha2.pem -v --http2 https://http2.mozqa.com/
* ALPN, server accepted to use h2
< HTTP/2.0 200
< x-backend-server:TS
< content-type:application/json
{'protocol':'HTTP/2'}

$ /usr/local/Cellar/curl/*/bin/curl --cacert ca_sha2.pem -v --http1.1 https://http2.mozqa.com/
* ALPN, server accepted to use http/1.1
< HTTP/1.1 200 OK
< X-Backend-Server: TS
< Content-Type: application/json
{'protocol':'HTTP/1.1'}

$ /usr/local/Cellar/curl/*/bin/curl --cacert ca_sha2.pem -v --http1.0 https://http2.mozqa.com/
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.0
< HTTP/1.1 200 OK
{'protocol':'HTTP/1.0'}

$ openssl s_client -connect http2.mozqa.com:443
GET /
{'protocol':'HTTP/0.9'}

Comment 3

[Stephen Donner [:stephend] Not actively reading bugmail]

Thanks, Richard!  Glad we're moving forward again with HTTP/2 support.  As for our tests, I don't anticipate any specific changes or impact (negative, at least) which enabling HTTP/2 will incur.

(We do use the Python requests library, though, so I'll check with the team to see the potential impact there.)

In terms of next steps, what can we do to put this into a near-term TCW?  I'd be more than happy to help test this during/post-deploy, then, as well.

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Screen Shot 2016-02-17 at 8.57.38 PM.png

Comment 5

[:Atoll]

The above steps are a changelog. You can begin using it immediately at https://http2.mozqa.com/.

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Richard Soderberg [:atoll] from comment #5)
> The above steps are a changelog. You can begin using it immediately at
> https://http2.mozqa.com/.

Right; sorry, more specifically, I meant: when can we enable HTTP/2 support on our ZLB cluster(s), by default, for our Web properties?

Comment 7

[:Atoll]

"our Web properties" is approximately three thousand sites. You may wish to prioritize your requests for sites you care about the most initially, since they have to QA their site over HTTP/2 and make sure it still functions correctly (it's not a perfectly-transparent upgrade, there are some slight variations in how cookies are delivered to the app).