611269 - fill in X-Forwarded-For in bugzilla requests

Status: RESOLVED FIXED

(Webtools Graveyard :: BzAPI, enhancement, P1)

Description

[timeless]

bzapi is a proxy, it should fill in X-Forwarded-For

ideally it should also have a conf field where you can specify trusted hosts from which it should transclude X-Forwarded-For values.

On the bugzilla server side, someone would of course have to know about specific bzapi instances and choose to trust their X-Forwarded-For values, but that's unrelated.

Comment 1

[Gervase Markham [:gerv]]

So basically, you want me to set X-Forwarded-For to:

join(",", $req->headers->{'X-Forwarded-For'}, $ENV{'REMOTE_ADDR'})

on all requests sent to Bugzilla?

Gerv

Comment 2

[Gervase Markham [:gerv]]

timeless or reed: ping?

Gerv

Comment 3

[Gervase Markham [:gerv]]

timeless or reed: reping?

Gerv

Comment 4

[Gervase Markham [:gerv]]

Resolving INCOMPLETE due to lack of input from reporter...

Gerv

Comment 6

[Reed Loden [:reed]]

re #c1, yes. If BzAPI has any type of proxies in front of it, things get complicated, but that's a simple fix, yes.

Comment 7

[Reed Loden [:reed]]

Take a look at the code in https://github.com/jsocol/commonware/blob/master/commonware/request/middleware.py

Implementing something similar would work great and be the best thing you can do.

Comment 8

[Gervase Markham [:gerv]]

Reed: I'm really not sure how that code snippet applies to BzAPI. That code seems to be working out the correct REMOTE_ADDR by using X-Forwarded-For. Surely that's what an end application should do, not something working as a proxy?

I've implemented comment 1. It's now live on the "/latest" install, and a lot of traffic flows through that. Can someone confirm it's working, by looking at requests to Bugzilla?

Gerv

Comment 9

[Gervase Markham [:gerv]]

dkl/glob: Can either of you confirm this X-Forwarded-For is working, or tell me who can?

Gerv

Comment 10

[:glob ✱]

bmo only sees the header which zeus presents to it, so IT will have to check the zeus is configured to pass bzapi's x-forwarded-for header through to bmo (if possible).

ashish should be able to help.

Comment 11

[Dave Miller [:justdave]]

If you're sending one, zeus will tack your IP on the end and pass it through.

Easiest way to tell is to authfail repeatedly via the bzapi so you get locked out.  The lockout email will tell what IP it thought you were on.  We've gotten a few (more than a week or two ago last time we got one) that had bzapi's IP address as the IP listed on the notice (which would indicate that it wasn't working).  Based on the timing of comments you probably hadn't implemented it yet then.

Comment 12

[:glob ✱]

(In reply to Dave Miller [:justdave] from comment #11)
> Easiest way to tell is to authfail repeatedly via the bzapi so you get
> locked out.  The lockout email will tell what IP it thought you were on. 

i've just triggered another, and we're still getting the api-dev ip:

> The IP address 63.245.210.151 failed too many login attempts (5) for
> the account [e-mail]. 

63.245.210.151 is api-dev.bugzilla.mozilla.org.

note: this ip is already included in bmo's inbound_proxies parameter.

Comment 13

[Gervase Markham [:gerv]]

It's been in place for several weeks now, but on the /latest endpoint only. So if people were using e.g. the /1.0 endpoint, they wouldn't have seen it.

glob: which endpoint did you use?

Gerv

Comment 14

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #13)
> glob: which endpoint did you use?

i used /latest

Comment 15

[Gervase Markham [:gerv]]

I've done a local test using an MITM proxy and it does seem like BzAPI is setting X-Forwarded-For correctly in my test system. Someone will need to look on b.m.o. at a request coming in from BzAPI and see whether it's coming through in the production system. If it's not, we need to figure out why. If it is, we need to figure out why Bugzilla isn't respecting it.

Gerv

Comment 16

[Gervase Markham [:gerv]]

We figured this out - FastCGI doesn't populate $ENV{'REMOTE_ADDR'} so we had to get the info from elsewhere ($c->request->address) for the production system.

Gerv