61426 - HTTP_REFERER not sent on reloads, frames, and 'open in new window'

Status: RESOLVED FIXED

(Core :: DOM: Navigation, defect, P3)

Description

[Frank Johnson]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux 2.4.0-test11 i686; en-US; m18) Gecko/20001128
BuildID:    2000112821

The HTTP_REFERER header is not sent on page reloads (while it is sent on the
request that got you to that page), frame target requests, and middle-click/Open
Link in New Window.

Reproducible: Always
Steps to Reproduce:
Follow the example link http://web.they.org/mozerr/

Comment 1

[Jacob Steenhagen]

*** Bug 61483 has been marked as a duplicate of this bug. ***

Comment 2

[Jacob Steenhagen]

OS -> All
Will search for possible dupe.  If one isn't found, I'll confirm it.

Comment 3

[Jacob Steenhagen]

Status -> NEW

Comment 4

[Gagan]

http bugs to "Networking::HTTP"

Comment 5

[Darin Fisher]

This seems like a uriloader/docshell bug.  To HTTP, the referrer header is
an attribute set by whomever created the channel.  HTTP has no way of knowing
which page a particular request corresponds to.

Comment 6

[Boris Zbarsky [:bzbarsky]]

*** Bug 64258 has been marked as a duplicate of this bug. ***

Comment 7

[Jerry Baker]

CC: self

Comment 8

[Adam Lock]

Can you verify this is still a problem with the latest nightly?

The reload and frame test behaviour both appear to be functioning correctly for 
me. The only place it doesn't seem to work is when loaded into a new window.

Comment 9

[Adam Lock]

After further testing I see that opening in a new window works too.

Everything works for me.

Comment 10

[Adam Lock]

Marking this WORKSFORME to clear off the 0.8 plate. Reopen and retarget if it 
still occurs.

Comment 11

[Ben Bucksch (:BenB)]

Referrer on reload?? What is it sat to?

Comment 12

[Adam Lock]

The reload referer points to the previous page no matter how many times you do 
it.

Comment 13

[Frank Johnson]

Referrer still not properly set on "Open link in new window"

Comment 14

[Judson Valeski]

-> mozilla0.9

Comment 15

[Sean Harding]

It also doesn't send REFERER when you click 'back' (in all of the cases in which
I've seen this happen, it seems like the previous page should have been cached
anyway, so I see no reason that it should have re-GETed the page. But it did,
and if it's going to it should send HTTP_REFERER again). 

Comment 16

[karl_stas]

Referrer is not sent when you open a popup window using Javascript. See this
page: http://civiseuropaeus.virtualave.net/gallery/gallery1.htm . Click any of
the thumbs.
OS: Win 98
Build: latest nightly build

Comment 17

[Adam Lock]

Attachment: Popup window prints out referrer info (see next attachment)

Comment 18

[Adam Lock]

Attachment: Test HTML for opening the referrer info window.

Comment 19

[Adam Lock]

The second attachment opens the first attachment in two different ways, via 
Javascript and via a link with a target. The first attachment then prints 
information about its referrer.

This appears to work for me with a recent build. Inspection of the HTTP traffic 
also shows that the referrer is being sent.

Comment 20

[Adam Lock]

Attachment: Updated test HTML handles javascript: urls as used in the civiseuropaeus site

Comment 21

[Adam Lock]

CC'ing Vidur (see the civiseuropaeus comment below)

Results of the test are:

1. Passed
2. Passed
3. Passed

But... If you right mouse click "Open in new window" on option 1 the new window 
has an empty referrer. Reload appears to work fine, but hitting the Back button 
clears the referrer.

It also appears that http://civiseuropaeus.virtualave.net/gallery/gallery1.htm 
is failing because the code is opening an empty window i.e. window.open(""), so 
there is no associated http channel to obtain the document referrer from. Vidur, 
should we modify the window opening code so that a new window's referrer is 
initially set taken from the caller's window?

So the outstanding problems appear to be:

1. Session history loses referrer on back/forward
2. Empty new windows have no http channel and therefore no referrer
3. Targets opened in new window from the context menu have no referrer

For reference IE 5.5 has empty referrers with tests 2 & 3 but links work fine 
for 1 and when opened in a new window.

Comment 22

[Adam Lock]

Opening bug 75884 to deal with the window.open("") issue.

Comment 23

[Adam Lock]

Attachment: Patch adds referrer URI property to session history

Comment 24

[Adam Lock]

The patch adds a referrerURI property to session history entries. This means 
that when an entry is created the referrer is saved and used when you go back or 
forward to that page.

Radha & Rick. Can I have reviews & super reviews please?

Comment 25

[rpotts (gone)]

sr=rpotts.  Looks good...

The only question I have is whether it would be better to store the referer as a 
string rather than caching the entire URI...  This would cut down on the bloat 
of each SHEntry...

However, the same argument could be made about the URI attribute :-)

Comment 26

[Radha on family leave (not reading bugmail)]

Here's something that comes to my mind:

If  Page A --> leads to Page B, a SH Entry is created for Page B with PageA as 
the referrer and stored in SH. After several link clicks, let's presume the user 
is in Page F. Now if the user clicks on the Go menu and visits Page B, the 
referrer that will be passed  to the channel will be Page A, the original 
referrer for B instead of F. Is this the behavior we want? If so, the attached 
patch does that. 
 

Comment 27

[Adam Lock]

I think it is correct to leave the referrer for B be A whether you use Go from 
page F to get to it or by clicking the Back/Forward menus or buttons.

Otherwise you could see some very strange behaviour. For example, some pages 
check the referrer and they would work fine for Back/Forward but would start 
complaining if you got to them from the Go menu.

Comment 28

[Frank Johnson]

Um, isn't Go only ever used for fresh (no referrer) requests?

Comment 29

[Radha on family leave (not reading bugmail)]

I understand that the behavior should be the same and we should pass a referrer 
whether the user gets to a page thro' Back and forward or Go menu. My question 
is, in  all SH loads, the referrer be, where it is going from right now(page F) 
or  what the referrer was originally for the page (Page A). I don't know what we 
did  in 4.7 or what IE does. But as long as we keep the pages :-) r=radha 

Comment 30

[timeless]

a few questions and concerns.

i navigate to:
http://whitehouse.gov
http://AllYourHeadersBelong.To/Us
http://localhost
https://internal.topsecret:4421/myname/mypassword/myproject/
I select back[http://AllYourHeadersBelong.To/Us]
Why should AllYourHeadersBelong.To/Us be told about where i went in the future?

webpages can in some cases use <something>.go(-1), are the allowed to do that 
reguardless of domain crossings?

Comment 31

[Adam Lock]

I believe the referrer for a session history item should be the same as when 
the page was last viewed, otherwise you might end up with unexpected behaviour. 
From a user perspective, it doesn't make any sense to do it any other way.

Both IE and 4.x appear to keep the referrer the same no matter how you go back 
to the page, either by the back button or the Go/GoTo menus.

Comment 32

[Matthias Versen [:Matti]]

*** Bug 76096 has been marked as a duplicate of this bug. ***

Comment 33

[Adam Lock]

With the sr I am inclined to check this in unless anyone has objections strong 
enough to delay for 0.9.1. 

Comment 34

[Adam Lock]

Fix for the session history problems is checked in. Bug 75884 and bug 76322 have
been opened to cover the window.open("") and the "Open Link in new window" issues.

Comment 35

[Jerry Baker]

Mass removing self from CC list.

Comment 36

[Jerry Baker]

Now I feel sumb because I have to add back. Sorry for the spam.