725253 - Affiliates Facebook sharing message is wrong because we send a 206 HTTP header instead of 301

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Staś Małolepszy :stas]

One of the users reported that the default message of the facebook post that we show when you click 'share on facebook' reads the following:

> 301 Moved Permanently
> http://affiliates.mozilla.org/

Comment 1

[Anthony Ricaud (:rik)]

First, we should change our share button to include "https".

I found http://stackoverflow.com/questions/8923268/share-301-status-code-urls-on-facebook to help debug this.

https://developers.facebook.com/tools/debug/og/object?q=https%3A%2F%2Faffiliates.mozilla.org%2F
This says that we send a 206 response code. That's not good.

Comment 2

[Anthony Ricaud (:rik)]

Facebook sends a Ranger header.

To test, add "Range: bytes=0-1000"

I only get this 206 header on prod. Dev and stage return a 301. So it's one of the server in front of our dev setup that sends this header.

Jason: Can you take a look?

Comment 3

[Chris More [:cmore]]

Yeah, since Affiliates HTTP redirects to HTTPS by default, it doesn't surprise me that http://affiliates.mozilla.org shows a 301 redirector. We added all of the Facebook open graph meta data before and it tested out fine.

Bug 696397, is where we added all of the Facebook OG tags and tested it out. I even tried sharing "https://affiliates.mozilla.org/en-US/" and it didn't show a 301, but it also didn't pull in our OG tags.

Comment 4

[Anthony Ricaud (:rik)]

Fix for the https link with https://github.com/mozilla/affiliates/commit/69e5e84587a082ac8132ec033cd70c18276018ba

Comment 5

[Anthony Ricaud (:rik)]

Moving to server ops. See comment #3.

Comment 6

[Jake Maul [:jakem]]

I'm confused. Do you need a code push for affiliates.mozilla.org?

Comment 7

[Anthony Ricaud (:rik)]

No, we need help investigating the 206 http response.

Oh and I meant "See comment #2" not 3.

Comment 8

[Jake Maul [:jakem]]

Okay, after reading into this some more, I'm just even more confused. :)

Comment 1 indicates that a 206 is bad, but comment 2 indicates that we have a problem on dev, which throws a 301... guessing that's a typo on comment 2, and was supposed to say "in front of our *prod* setup".

HTTP 206 is "Partial Content Returned", which is what happens when the request includes a Range header (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html). Comment 2 says Facebook is sending one, so a 206 response sounds like expected behavior. What's the problem? How else would we want to respond?

We 301 to https:// when the request is http://, and respond normally when the request is https:// to begin with. I'm not sure how the 301 behavior is relevant though, except that comment 0 indicates we were linking to the wrong version initially (fixed in comment 4?) and that this was causing some display problems.

Comment 9

[Anthony Ricaud (:rik)]

(In reply to Jake Maul [:jakem] from comment #8)

> Comment 1 indicates that a 206 is bad, but comment 2 indicates that we have
> a problem on dev, which throws a 301... guessing that's a typo on comment 2,
> and was supposed to say "in front of our *prod* setup".
You're right, that was a typo. Sorry for the confusion. I must have been tired when writing this.

> HTTP 206 is "Partial Content Returned", which is what happens when the
> request includes a Range header
> (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html). Comment 2 says
> Facebook is sending one, so a 206 response sounds like expected behavior.
> What's the problem? How else would we want to respond?
I think we should respond a 301, even if it's a Range request. Without that, the Facebook scraper can't see a real page. See http://developers.facebook.com/tools/debug/og/object?q=https%3A%2F%2Faffiliates.mozilla.org%2F

> We 301 to https:// when the request is http://, and respond normally when
> the request is https:// to begin with. I'm not sure how the 301 behavior is
> relevant though, except that comment 0 indicates we were linking to the
> wrong version initially (fixed in comment 4?) and that this was causing some
> display problems.
The http vs https is another issue, not related to this one.

Comment 10

[Jake Maul [:jakem]]

Okay... what should https://affiliates.mozilla.org/ 301 to?

Comment 11

[Anthony Ricaud (:rik)]

It should redirect to https://affiliates.mozilla.org/{locale}/. This is what the Django app sends. I think it is Zeus (or whatever HTTP proxy we use) that overrides the 301 with a 206. Or maybe it is directly Apache.

Comment 12

[Jake Maul [:jakem]]

Okay, this makes more sense to me now.

Unfortunately I don't know how to fix it.

[jakemaul@JakeBook ~]$ curl -k -v -H 'Range: bytes=0-1000' -H 'Host: affiliates.mozilla.org' http://node27.seamicro.phx1.mozilla.com:81/
* About to connect() to node27.seamicro.phx1.mozilla.com port 81 (#0)
*   Trying 10.8.32.33... connected
* Connected to node27.seamicro.phx1.mozilla.com (10.8.32.33) port 81 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Accept: */*
> Range: bytes=0-1000
> Host: affiliates.mozilla.org
> 
< HTTP/1.1 301 MOVED PERMANENTLY
< Date: Thu, 16 Feb 2012 23:45:30 GMT
< Server: Apache
< Strict-Transport-Security: max-age=2592000
< Vary: Accept-Language,X-Mobile,User-Agent
< x-frame-options: DENY
< X-Content-Security-Policy: allow 'self'; img-src 'self' https://affiliates-cdn.mozilla.org https://statse.webtrendslive.com; script-src 'self' https://browserid.org https://statse.webtrendslive.com; frame-src 'self' https://browserid.org; font-src 'self' https://www.mozilla.org
< X-Backend-Server: node27
< Location: http://affiliates.mozilla.org/en-US/
< Content-Length: 0
< Content-Type: text/html; charset=utf-8
< 
* Connection #0 to host node27.seamicro.phx1.mozilla.com left intact
* Closing connection #0

So from a particular node directly, it responds as expected... 301, no delays.


[jakemaul@JakeBook dnsconfig]$ curl -v -H 'Range: bytes=0-1000' https://affiliates.mozilla.org/
* About to connect() to affiliates.mozilla.org port 443 (#0)
*   Trying 63.245.217.71... connected
* Connected to affiliates.mozilla.org (63.245.217.71) port 443 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Host: affiliates.mozilla.org
> Accept: */*
> Range: bytes=0-1000
> 
< HTTP/1.1 206 Partial Content
< Content-Length: 1001
< Content-Range: bytes 0-1000/0
< Content-Type: text/html; charset=utf-8
< Server: Apache
< Vary: Accept-Language,X-Mobile,User-Agent
< X-Backend-Server: node27
< X-Content-Security-Policy: allow 'self'; img-src 'self' https://affiliates-cdn.mozilla.org https://statse.webtrendslive.com; script-src 'self' https://browserid.org https://statse.webtrendslive.com; frame-src 'self' https://browserid.org; font-src 'self' https://www.mozilla.org
< Strict-Transport-Security: max-age=2592000
< Date: Thu, 16 Feb 2012 23:38:23 GMT
< Location: https://affiliates.mozilla.org/en-US/
< x-frame-options: DENY
< Connection: Keep-Alive
< X-Cache-Info: cached
< 
* transfer closed with 1001 bytes remaining to read
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
curl: (18) transfer closed with 1001 bytes remaining to read


But through Zeus, the status changes, more headers are added, and it doesn't actually close the connection right away... like it's waiting on more info or something.


I've looked over the rules applied to this virtual server and don't see anything that's obviously relevant. There's a rule that returns a 403 Forbidden if there are too many range headers (vulnerability prevention), but clearly we're not hitting that.

Adding cshields and oremj for comment.

Comment 13

[Jake Maul [:jakem]]

Need to open a bug with Zeus on this, I think... don't have any better ideas.

Comment 14

[Corey Shields [:cshields]]

Jake is out on pto this week, assigning to Brandon.  I'll make sure he has access to open a case with Riverbed.

Comment 15

[Brandon Burton [:solarce]]

Opened case 214627 with Riverbed about this. Waiting on an update from them.

Comment 16

[Brandon Burton [:solarce]]

Just wanted to update you. I am still working with Riverbed on this.

Thanks

Comment 17

[Brandon Burton [:solarce]]

Riverbed supplied us with a rule so we don't cache 301 when a range request is sent in.

The curl tests :jakem did above are now consistently returning 301 requests.

Comment 18

[Anthony Ricaud (:rik)]

Awesome, it actually fixes this.

Comment 19

[Fred Wenzel [:wenzel]]

Nice job debugging and fixing this!