32359 - Wrong referrer given to images after redirect

Status: VERIFIED WORKSFORME

(Core :: Networking, defect, P2)

Description

[Jesse Ruderman]

http://komodo.mozilla.org/buster/random/random.html will occasionally load a 
page using a certain "free counter" image.  The image shows up as something 
similar to "Host random.yahoo.com not in site list".  That can't be right.. and 
it means we're giving the WRONG information (probably referrer) to the counter 
site.

Most counters aren't affected by this problem - just that free one that relies 
on the referrer.  Unfortunately, I managed to lose my screenshot, and due to 
bug 32353, I lost the page that had been loaded as well.  This isn't just a 
fluke, though, since I've seen it several times while using the "browser 
buster" (mozilla's debug menu) interface to random.yahoo.com.

Severity=major because I'm too tired to convince myself that this isn't a 
security problem.

Comment 1

[Jesse Ruderman]

OK, I found a convoluted way to reproduce this locally.
- I use the OmniHTTPd webserver from http://www.omnicron.ab.ca/httpd/index.html
- Omni redirects from http://jesser.dhs.org/math to http://24.13.163.233/math/
- The "math" directory had no index file, so the contents got listed
- Omni shows icons for different types of files on automatic directory listings.

Highly edited web server logs show the two browsers giving different referrers.

Mozilla   "Mozilla/5.0 (Windows; N; Win98; en-US; m14)"

"GET /icons/default.gif HTTP/1.0" 200 132 "http://jesser.dhs.org/math"
"GET /icons/image.gif HTTP/1.0" 200 231 "http://jesser.dhs.org/math"
"GET /math/ HTTP/1.0" 200 548 ""
"GET /math HTTP/1.0" 302 274 ""

Internet explorer  "Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)"

"GET /icons/image.gif HTTP/1.1" 200 231 "http://24.13.163.233/math/" 
"GET /icons/default.gif HTTP/1.1" 200 132 "http://24.13.163.233/math/" 
"GET /math/ HTTP/1.1" 200 548 ""
"GET /math HTTP/1.1" 302 274 ""

Note: mozilla does get the referrer right when I click on one of the items in 
the directory -- this seems to only apply to images loading in the page that 
gets redirected to.

This bug might be related to bug 24143, history and location bar don't reflect 
server-side redirects.

Comment 2

[Asa Dotzler [:asa]]

--> networking

Comment 3

[Gervase Markham [:gerv]]

Confirming to get attention; but [e-mail] - does your local reproduce 
still show the wrong thing?

Gerv

Comment 4

[Jesse Ruderman]

Marking dependent on bug 36901, because currently images aren't getting 
referrers (referers?) at all.

Comment 5

[Doron Rosenberg (IBM)]

jesse - are you still seeing this on new builds? If yes, please confirm thisbug.

Comment 6

[Jesse Ruderman]

Confirming (2000 060408 win98).  Bug 36901 no longer seems to be in the way of 
duplicating this bug, so removing dependency.

Comment 7

[Jason Summers]

Created a test page for this:
http://home.mieweb.com/jason/testbed/referer/

Comment 8

[Dan Rosen]

I'm seeing a similar symptom (on linux), and believe my symptom to be caused by
this bug as well: the ad banners on http://slashdot.org redirect you to the
correct site, but neither set the correct URL redirected to in the location bar,
nor does the site load images correctly (since they are URLs relative to the
mangled redirection cgi URL rather than the actual base URL).

This is a serious problem since much commerce on the web (and I'm trying not to
laugh) relies on ad banners. Nominating nsbeta3. This is also likely an HTTP
bug, so indicating correctness. Also, can QA or somebody with mad skillz (Jason
Summers?) verify what's up here?

Comment 9

[timeless]

This might be related 

Additional Comments From Gagan 2000-08-20 13:53 in bug 48200:
Finally figured out whats going on here. hyatt's recent checkins to change 
GetURI to GetOriginalURI (in nsDocument::Reset()) broke the mechanism. I have a 
fix but this is in hyatt's code so I have to get it reviewed from him. Will 
attach the diffs here.
--
CC'ing hyatt; Warning: I could be totally offbase ;-)

Resetting Milestone, this is nsbeta3 nominated lets leave it non futured until 
pdt kills it [PDT don't consider this advocation for not plussing, if you 
consider commerce at all then you probably can't not plus it]

Comment 10

[Dan Rosen]

An even better reason to fix this bug: the ad banners on My Netscape for
AOL-subsidiary branded services (if you could only see the look on my face...)

Comment 11

[Jesse Ruderman]

most images will work correctly despite this bug (the slashdot.org ad banners 
leading to pages with broken images was bug 48200).  only some images, such as 
some counters and certain dynamic images, 

dan, you did bring up something (maybe unintentionally) that i hadn't thought 
of before.  This bug could cause lost advertising revenue: site A redirects to 
site B, which shows ads from site C, but site C sees site A's url in the 
referer and ignores the ad request.  

(Not that lost advertising revenue is always bad -- i'm not against allowing 
users to block doubleclick.net images, for example -- but in this case the user 
probably sees the ad, so site B should get credit for the ad display :)

Comment 12

[Dan Rosen]

I can't say I like online ads much either... But a company like DoubleClick
might just be "righteous" enough to file suit against Netscape or, far worse,
Mozilla.org for intentionally not fixing this bug. Anyway, enough said. Can we
get a nsbeta3+ on this?

Comment 13

[Gagan]

plus and p2

Comment 14

[selmer (gone)]

pdt agrees p2.

Comment 15

[Dan Rosen]

nominating rtm, if not high enough priority for beta3...

Comment 16

[Phil Peterson]

Not holding PR3 for this. Marking nsbeta3-

Comment 17

[Gagan]

approving for rtm (need info). 

Comment 18

[Gervase Markham [:gerv]]

So, do we want to relnote this in some way, or just push it under the carpet and 
hope no-one sues? :-)

The blurb:
It seems unclear to me whether this bug requires either of a "developer" or 
"user" release note for Netscape 6 RTM. If anyone feels it does, can they please 
draft one and then nominate with the relnote-user or relnote-devel strings in 
the Status Whiteboard.

Thanks :-)

Gerv

Comment 19

[timeless]

relnote-user: mozilla might do something silly when you attempt to load an 
object from a web server on a port other than :80.  We are sorry and hope to 
fix it by ns601.  Most likely this will result in content not loading. It is 
possible [although we can't describe a process, and it might take 6-10 reloads 
and restarting your web browser] to retrieve this content.

There is no relnote-developer, if there were one it would say that would don't 
support servers on alternative ports and that they should set up their servers 
on :80 instead, that's not a valid relnote.

Comment 20

[Gagan]

Um... this seems like a wrong rel-note... or rather a misplaced one. There is a
separate bug about non-default ports where this relnote belongs.

Comment 21

[Jason Eager]

Are we going to fix this or not? If not, please take it off the rtm radar.

Comment 22

[Darin Fisher]

I am unable to confirm this bug on today's build.  I tried out the test page
setup at http://home.mieweb/com/jason/testbed/referer, and both NS4.75 and
and NS6.0 cause the CGI script to complain that the referer was not set!
I took a look at the HTTP transaction (using ngrep on port 80) and it appears
that we do set the referer to http://home.mieweb.com/jason/testbed/referer/
on each and every GET.  I am attaching a copy of that HTTP transaction log.

Moreover, I spent some time clicking on ad banners, and I did not encounter
a single one that resulted in images not being displayed, or the web site
being somehow incorrect b/c of not getting the correct referer header.  I also
hit a couple websites with counters (the open source one... Count.cgi) via
redirect and didn't notice anything out of the ordinary.

The source code (nsHTTPChannel.cpp) seems to be doing exactly the correct thing.
It forwards the referer to the new channel that it creates on redirect.  The
cvsblame CGI script on lxr.mozilla.org seems to be down right now, or I could
take a look and see when this section of nsHTTPChannel.cpp was added.

At any rate, I am convinced that we don't have to worry about this bug, but
all the same, can someone verify this bug again?

Comment 23

[Darin Fisher]

Attachment: HTTP transaction log

Comment 24

[timeless]

darin this bug should not be verified until its branch status is settled. Would 
you consider repeating w/ a branch build?

Comment 25

[Jesse Ruderman]

wfm trunk and branch.

Comment 26

[Jason Summers]

There was a problem with the part of my test case that displayed the Referer of 
the HTML page (sorry -- it's now fixed), but that's irrelevant to this Bug.

I agree that the Mozilla bug appears to be fixed.

Comment 27

[Darin Fisher]

WFM on the branch as well [linux build 2000110209]
-> marking resolved: WORKSFORME

Comment 28

[Tom Everingham]

verified WFM

Comment 29

[benc]

-relnote, relnoteRTM