758000 - blog.mozilla.org not handling X-Forwarded-For properly

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Reed Loden [:reed]]

blog.mozilla.org is showing internal IPs/hosts for commenters on posts rather than their real IP. Guessing the list of valid proxy servers is not configured correctly in WPMU so X-Forwarded-For is getting ignored (or whatever header is supposed to be used).

Comment 1

[Reed Loden [:reed]]

Specifically, I'm seeing:

Author : offshore bank account (IP: 10.8.81.216 , pp-zlb09.vlan81.phx.mozilla.com)

Comment 2

[Jake Maul [:jakem]]

I judging by some comments I see, I think this stopped working properly when blog.mozilla.org moved from SJC1 to PHX1. Either Zeus or the Netscaler in SJC1 was tweaking X-Forwarded-For, and Wordpress was configured to pick up on this and use it instead.

However, Zeus in PHX1 was not configured to touch X-Forwarded-For. By default, it instead adds an X-Cluster-Client-IP header. I have added a chunk to the wp-config.php to use this header if it exists... we can configure Zeus to also mess with X-Forwarded-For, but it seems more future-proof to make Wordpress deal with *both* scenarios, so that's what I've attempted. This will work on any Zeus cluster right out of the box with no special rules, or on anything that munges X-Forwarded-For.

I waited around a few minutes after deploying, and eventually a spam comment came in for the main blog. It shows an external IP, so this appears to be fixed!

Thanks for catching this. :)