Mozilla intentionally (I think) hides the referer when going from one HTTPS
page to another in a different domain.
This is a *very* bad design decision, as it prevents using at least one major
online credit card processing service which looks for referer information
(along with other info) for security checks.
Currently we have been telling our customers that they have to use IE because
mozilla/netscape simply does not work. We do not want to do this!
I understand that this is done to improve security, since some sites place
private information in plain text in URLs. Consider, though, that this is a
problem with the site and cannot be fixed by the browser. For example, if
someone walks by a person using the insecure site there private information
will be right there on the screen ready to read (in the URL).
The decision to make fundamentally insecure sites more secure is causing the
browser to be *much* less usefull overall since it can often not be used to
place orders online, forcing users to switch back to IE.
By default, referer information should be included when jumping from one HTTPS
page to another in a different domain. If you want to go beyond that and add
the option (not selected by default!) of not providing the referer in this
I hope this descision will be reconsidered!
Reporter (Alan): what site or sites does this bug affect?
According to bug 128213, there's a hidden pref an advanced user can set to allow
referer information to be sent from one https site to another. That's just
wrong in so many ways:
1. Yet another (hidden) pref. Hurts startup time, complicates code, doesn't do
2. Why does it matter whether the target is http or https?
3. Giving sites a way to prevent the referer from being sent is, IMO, a security
hole. A minor one, but enough of a hole that many porn sites refuse image
requests when the referrer is not sent instead of assuming that the request is
from a bookmark.
I think the best solution is to send the referer but strip off the pathname,
query string, and hash. For example, a link from
https://answers.google.com/answers/main?cmd=threadview&id=8149 would have its
referrer set to https://answers.google.com/.
Yeah, that last solution you propose was what badami, darin, mstoltz all seemed
to agree to last time I mailed them about it... (send referrer from https to
https, but strip everything but the hostname out of the url).
Alan, thanks for your report and that you care about Mozilla-based products.
Which site are we talking about? Which information does it exactly what to see
in the referrer? E.g. is *any* referrer (even a wrong one) sufficient? Is the
hostname of the referring site sufficient? DOes it need the full, valid referrer?
Any site which relies on the Referrer header to be present is broken and
violates the HTTP specification. The spec is ultra-clear that the referrer is
something optional that can be disabled on the user's discretion. Please notify
the site about that. Reference: RFC 2616, sections 14.36, 15.1.2 and 15.1.3.
The latter also says:
Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
This is not exact the case discussed here, but might be worth consideration.
Sending the hostname of the referring host as Referrer between 2 https hosts
might strictly be a violation of the HTTP spec. The hostname is not "the address
(URI) of the resource from which the Request-URI was obtained".
Apart from that, I currently don't know a reason not to send the hostname in
The credit card processor in this case is authorize.net. Perhaps there are others as well.
Thousands of sites (including my site, FreeFind.com) use this service to handle credit card
transactions, but it does require a referer to be set.
To guarantee complete compatibility
with what's out there now, you'd need to send the complete URL.
I would guess that stripping
just the query string would probably work for most cases. Only sites which have them in their order
processing sequence would need to update their credit card processing security handshake
If you set the referer to just the domain name it would force essentially all sites using
authorize.net to reconfigure their security handshake info before mozilla would work. Ideally
they would (and I certainly would), but I think that practically speaking most would not. In a way,
this does sound safest (if your willing to have your users take the compatibility/usability
hit), but in some cases it might be less safe. For instance, the referer for two different sites:
would be the
same... (and again, yes I know that the referer shouldn't be part of a security mechanism, but what
is out there being used is what the browser needs to work with..)
Again, where I've noticed
this is going from a HTTPS page on one site (my site) to an HTTPS page on a different site (the credit
card processor's site). And this may be more widespread than the thousands of sites using
authorize.net. It's easy to envision lots of services having a referer check as part of their
I have notified authorize.net of the problem. We've been experiencing
this for months now, but they seem to be asleep at the wheel on this. For me, this underlines the
importance of having mozilla work with the way things are currently set up on the net and not force
people (both developers and users) to reconfig/recode/conform in order to support mozilla. I
think that's very important to success.
Given the above, I would recommend (and I hate to say
this..gag) referer behavior that is 100% compatible with IE, at least by default. If including
query strings is truly out of the question then at least keep everything else (the
Think "always compatible" and thanks again,
will address this for moz 1.0.1
Before sending the full path, I can't we try to advocate the broken site not to
require a referrer? Move to evang?
Even if you could get authorize.net to update their system to support mozilla (and I
think there are real issues with getting them to abandon referers, which are already intergrated
into their system), what about the thousands of other sites that you don't know about that rely on
the referer for some check/feature/etc? How do you track those sites down and tell them to re-code
too? Even if you could track them down, would they? or would you just be systematically offending
the internet community? (While your at it, you could tell them to get all the syntax errors out of
their HTML pages, too ;)
This is a basic compatibility issue. How compatible you want your
browser to be? Do you think that most people will keep on using it if they have a small but steady
stream of problems with it?
mass futuring of untargeted bugs
Alan, there are not thousands of sites. This is a corner case any this site is
the only problem we know of.
I just called customer support of Authorzie.net, which declined to let me speak
with a developer, claimed that this is working fine with Netscape 6.2 and
refused to change their site. Well, their problem, I'd say.
And it seemed to work fine with their test account for WebLink with Beonex
Communicator with the fake 3rd party referrers option enabled (bug 55477).
*** Bug 151640 has been marked as a duplicate of this bug. ***
Ben, I just tried again with the latest Mozilla 1.1 alpha (build 2002061104)
and the very latest netscape (6.2.3) with the same (failing) results.
It is my understanding that authorizenet has thousands of customers, each of
whom is a webmaster with their own site(s), so this problem is probably greater
than a single site.
It is very difficult to get complete/accurate feedback from users (of all
types), so I would hesitate to conclude anything about the scope of the problem
based on lack of feedback. Typically only a minute percentage of people provide
feedback even when it would be trivial to do so. When there are additional
obstacles to providing feedback (registering for bugzilla and figuring it out,
for example) you can expect even less than a minute percentage. Furthermore in
the case of browser software, first a small percentage of the end users will be
complaining to the webmaster with the problem ("your website doesn't work"),
and then a small percentage of those webmasters will be then be downloading
mozilla, testing, and then bugzillaing the feedback on to you.
To make the challenge of writing a compatible browser even more difficult, it
is difficult if not impossible to comprehensively "test the web" to make sure
your browser is compatibile, especially when the test may involve buying things
Because it is impossible to truly test the web and because getting a
statistically significant quantity of feedback from users is problematic, it
seems that a good strategy to ensure compatibility for web users would be to
compare mozilla's precise behavior with ..I truly hate to say this.. IE, since
that is the current standard.
Insisting that companies re-write their code to be techinically correct is
clearly a non-starter. One phone call to authorizenet (or more, as I have
made!) demostrates this conclusively.
Technically unconforming sites might be "their problem" but in fact it is "your
problem" because if mozilla doesn't work everywhere they go the users will not
keep using it regardless of "whose problem" it actually is.
So my recommendation for the success of Mozilla is to be maximally compatible -
work like (...) IE, even if IE is wrong.
..hmmm...there's a option for you!
(x) IE compatibility mode
( ) mozilla RFC validator mode
As for my specific case, all *I* personally need is for mozilla to by default
send something (anything!) in the HTTP referer field in this specific case.
Even "this referer was not set due to security concerns" would work. Then I
could adjust my authorizenet account to accept it. :)
My company's icon technology makes specific use of the browser's referrer
field. We are yet another example of a company that uses the browser's referrer
field for added security and to provide a service. Our model has proven to work
in all browsers until this more recent change to the Netscape source to prevent
referrers from being passed from one HTTPS domain to another. The omission of
the referrer results in our icon not being displayed to the end user(the
browser) making the browser much less useful. I too would not like to suggest
to all of our customer's customers to not use any Netscape browser products.
Even if the browser only sent over the referring domain I could work out
solution. GeoTrust's icons get millions of hits per week, and provide valuable
information to the end user.
I do not see anywhere in RFC2616 where it states the referrer should be omitted
in HTTPS if another server is also HTTPS. If the user feels he has privacy
concerns and wishes to disable referrer sending, and therefore disable services
like GeoTrust's True Site, that is ok. But by default the browser should be
sending this vital information along.
This is not the only case where referrer sending has been left to the
programmer's interpretation. Another example of where the referrer is not sent
would be in SCRIPT SRC tags. Again Netscape browser's do not send a referrer,
while other browsers do. Making the Netscape browsers a little less useful
> Another example of where the referrer is not sent would be in SCRIPT SRC tags
Irrelevant to this bug, and fixed in bug 64248 a while back... 7.0beta1 should
be sending "Referer:" in that situation, modulo this bug.
*** Bug 154062 has been marked as a duplicate of this bug. ***
*** Bug 157054 has been marked as a duplicate of this bug. ***
we're also suffering from this problem. our website uses third party for a drug
info. they provide us with a drug description and info http://cp.gsm.com/
we link to them from regular pages and from secured pages. the latter links
don't work :(
they use HTTP_REFERER as some sort of authorization. yesterday i filed a bug
157054 that is a dup of this one.
It seems from the comments that it is a authorize.net problem. The referer field
is optional in the HTTP specs; So I agree 100% I agree with Comment #3 From Ben
It seems to me also, from the doc of authorize.net (see:
that this strange behaviour should occur only in WEBLINK mode (and perhaps ADC
RELAY RESPONSE). It should not occur in ADC DIRECT RESPONSE, because the
commnunication process relies only on the server (not on the browser).
Am I right?
The issue isn’t about whether or not the Referrer is optional or not. The issue
is that many applications use this field as an added security check. If someone
wishes to disable Referrer passing, and therefore not have access to the
application that checks it, so be it. But until recently all browsers passed
Referrer regardless if they were under HTTP or HTTPS. I agree with the
suggestion to not pass GET data in the Referrer, but a Referrer should be
passed by default in my opinion.
As a site administrator I want to share my own content between domains. But I
don’t want unauthorized sites to use my content, and my bandwidth. So I put a
simple check in my configuration to state which domains (Referrers) are allowed
to get at the content. With the recent change I can’t make this check, and
therefore I can’t get at my own content.
This is merely one of many examples. It is not necessarily a fault of the
application. The application acts as expected: The security check fails
therefore the action is not performed. But as an end user, or the client of the
application, this is not the desired behavior, and there is no way to fix it
other than using a “working” browser (IE, Netscape <4, Opera, etc).
John, this is *clearly* a fault of the application. The HTTP standard says that
the Referer is optional, meaning that the site *must not* rely on it. If it
does, it does not comply to HTTP anymore.
Even worse are browsers passing Referers from HTTPS to HTTP. I don't know, if
any one does that, but it would be a clear violation of HTTP, and a potential
All we are talking about is *maybe* adding a workaround for a certain egde case.
But the fact remains that the site is at fault.
> a fault of the application.
> Even worse are browsers passing Referers from HTTPS to HTTP.
what about going from HTTPS to HTTPS on another host? i think it should pass
the referer in that case. what does the HTTP spec sez about it?
I am not fighting to change the standard at the moment. I understand that the
Referrer is optional. But many WWW sites provide functionality based upon the
Referrer. The example case I listed above is used by many web administrators,
mainly to restrict content down to a single domain, but sometimes to a list of
domains. Search around and you will find many examples on how to enable
this "security" feature for your content.
If I were to follow your point to completion, why send a referrer ever? It is
optional, so why not just delete it completely? The answer is because it
provides the WWW server being called valuable information it can use to make
simple security checks, fraud checks, etc.
I am largely responsible for the icon technology we use here at GeoTrust. I
don't consider my application broken. It does just what it should, if I am not
sent a referrer I have no idea which icon I should render, therefore I render
nothing. I followed the specification, I don’t crash because I don’t get a
referrer, but I also can’t provide the client with the functionality they want.
I have not seen an example where an HTTPS page sends a referrer to an HTTP
page. This would be a clear violation of the specification. But HTTP to HTTP,
or HTTPS to HTTPS, is not a violation, and is what browsers have been doing
since the beginning of the WWW.
I obviously do not believe this is a fault of the application. The application
looks for information it gets from “working” browsers, and acts upon that
information. If the information is missing, it does nothing. That is following
the specification to the letter. By not sending referrer the browser is merely
proving to be less useful to clients and servers. Therefore consumers will
simply use a browser that provides them the content/features/actions they
We now have about a million hits a day on our icon. And currently 90% of the
browsers hits are from IE. Personally I’d like to see an alternative, but this
issue is an example of why there isn’t right now.
For what it's worth, Bank of America's eStores Settle Up Credit Card
processing system also uses HTTP_REFERRER to authorize credit card
"When a customer places an order, the address of the site generating the
order is compared to this list of sites authorized by the merchant before
the transaction is submitted for payment processing. If the addresses do
not match, the order is rejected and no credit card authorization is
obtained. When the addresses match, the order is submitted for credit
card authorization and finalized accordingly."
BofA's system returns a "site not authorized to post" error. Sure enough,
in using their debugging system, HTTP_REFERRER is not passed.
i think we should settle on a solution for this bug by mozilla 1.2 ... our
current solution (of a hidden pref) is just not satisfactory.
i'm treating this as an enhancement bug right now. our current security policy
prevents sending a referrer in this case. i'm not sure if we can reasonably do
anything... sending a only the prepath as a referrer is breaking the spec (lying
to the website) and sending the complete referrer is really putting the user at
risk. i'm also not sure how many sites break because of our restrictive
security policy. all this makes it very difficult to determine the best
solution for this bug.
I for one would prefer to see half of the truth as apposed to nothing. Does the
specification state that the entire GET string must be sent as part of the
Referrer? I didn't infer that, the spec states:
Referer = "Referer" ":" ( absoluteURI | relativeURI )
Example: Referer: http://www.w3.org/hypertext/DataSources/Overview.html
The definition of absoluteURI shows Query String as optional. So isn't sending
the complete referrer without the query string acceptable? I don't see this as
being a security risk. If you want to provide a paranoid option for turning off
referrer that's fine. But by default the solutions already suggested seem
Every day I come across another cgi/web application that makes use of referrer.
Luckily for you most applications are running under HTTP so this bug doesn't
show its ugly head often. But more and more people are realizing they do need
SSL for these applications.
We had hoped this bug would have been resolved many months ago. Certainly
before the Mozilla 1.1, and Netscape 7 were released. Webmasters and end users
are getting frustrated, and our only work around is to suggest not using
Mozilla based browsers. You are losing many loyal fans because of issues like
Our icon is now getting a couple of million hits a day. We are still receiving
complaints about all of the new Mozilla based browsers. More alarming to you
should be the statistics we are noticing, before this bug was introduced we had
about 86% IE hits, now we are seeing closer to 93% IE hits.
I've always thought that https->https should send referrer. The spec says:
Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
so HTTPs is not a "(non-secure HTTP request)", so the refererer should be sent.
I think that the main purpose of this is so that a GET login to an https page
doesn't then get sent out in the clear subsequently. This does leave open the
case of a link on the original server to an https page which grabs the referer
in some malicious way. GET login requests are evil anyway.
ISTR getting outvoted last time, htough...
Any bad guy can run an https site...
Right, but you'd ahve to get that 'bad guy' to get a link to their page onto the
other site's https page. Not impossible, but its not something 'anyone' can do
What about XSS (Cross Site Scripting)? It would be hard, but not impossible to create a link that would appear legit.
isn't cross site scripting a same origin violation? so wouldn't the user have
to explicitly allow it (assuming signed js were involved)?
given that the attacker would have to add their link to the legitimate site's
pages, i'm inclined to agree with bradley. what is the real risk of enabling
cross site HTTPS referrers?
websites can anyways link to other HTTPS sites indirectly via a HTTP redirect if
they really don't like this behavior, which i'm sure they're well acquainted
with since IE sends cross site HTTPS referrers.
are we really helping by preventing cross site HTTPS referrers?
fixed summary to state the problem instead of just one of the proposed solutions.
I assume we send no referrer when the user types something in the url bar,
right? If so, I'm also leaning toward allowing this....
right, no referrer should be sent in that case.
> isn't cross site scripting a same origin violation?
It's not scripting, but just linking. What about e.g. webmail? Not all webmail
sites are sane...
I still think that any site depending on referrers is broken per the HTTP spec,
especially in this case. IMO, this is an Evang issue, not a browser issue. Why
can't we first try to solve this via Evang, like we do with many other cases?
Don't forget that this can't happen automatically, because if js was enabled,
then the malicious page just has to grab the current rul via js.
If js is disabled, then you have to entice someone to click on an https page to
an https page on another domain, where the current URL has sensitive information
encoded as part of a GET url.
> If js is disabled, then you have to entice someone to click on an https page to
> an https page on another domain
No clicks needed. Think of images in HTML mail sent to webmail accounts.
Darin states that he's not sure how many sites are affected. Every single
online store that uses authorize.net's Automated Direct Connect Relay Response
mechanism will not work. I don't know how many customers authorize.net has, but
my guess would be that it is in the thousands.
I'm tired of having MSIE as a "backup browser". I get so frustrated when I have
to revert to it to conduct business on the Web. We can sit back and blame the
site designers or we can accept the reality of today's imperfect Web and
accomodate some of the major mistakes.
If I understand correctly, the reason you don't send the referrer is to prevent
sensitive info in the URL from being transmitted to an untrusted site. Encoding
sensitive information in a URL is pretty brain-dead. Isn't this also a failure
of the site designer? So we're working around one class of mistake but refusing
to accomodate another, and the end result is a browser that is less useful to
the end user on today's Web.
I know that mass popularity is not the end-all goal of the Mozilla team. But if
you don't achieve some sort of critical mass, MS will continue to drive the de
facto standards of how sites are constructed, and Mozilla will always play catch-up.
I love Mozilla passionately and evangelize at every opportunity. But the
reality is that even I have to keep MSIE on standby. Most of the time, I need
it because some Frontpage idiot can't design with an eye toward standards. In
those cases, I either skip over the site entirely or begrudgingly open IE. But
in this case, I find it more difficult to blame the site owners when the
referrer behavior of major browsers has been consistent for the past 5-6 years.
Suggestion: I know Mozilla needs to be secure out of the box. But if you
implement a "fix" for this "bug", I feel that a checkbox is not enough; Mozilla
needs to send the referrer on HTTPS _out of the box_. Otherwise, it will never
gain mainstream acceptance. Believe me, if you saw the authorize.net error
message, you'd never think that there was a browser setting you could adjust to
correct the problem. Joe User would just say "Mozilla sucks" and go back to IE.
How about a baseline security policy that is "mostly secure"; at install time,
the power user could check something that would adjust the overall policy toward
"manic paranoid", locking down anything and everything (including referrer on
I have to agree with Jason Priebe above. There are countless sites that are
using Referrer checks as part of a normal security/sanity/fraud check.
GeoTrust's customer list is in the many thousands right now.
The discussion about Webmail is an interesting one, but sure sounds like you
are trying to invent a problem for the current "solution". Ask yourself two
things; 1) How many webmail sites use HTTPS? AND 2) How many of those webmail
sites encode the password in a GET query string. Are there any? Certainly none
of the ones I am familiar with do(EXs: Yahoo, Hotmail, Outlook Web Access).
Since almost all webmail systems use HTTP, if they simply encoded your
username/password in the GET query string this bug doesn't add any security
benb: I forgot about <img>. hmm....
Any site using the referer as a security check is broken, since its trivially
forged. Using it as a sanity check is only slightly less broken, because people
may choose not to send it for privacy/etc reasons. (Using it as a sanity check
to reject incorrect, as opposed to non-existant, referers may be more useful,
although that can again be set by the user)
Using it in combination with other things (ie username/password) doesn't make it
less broken, since you then have an extra layer of authentication making the
referer stuff almost useless.
_However_, there are sites which care about this, and since the spec doens't
forbit us to do this, and a large % of users have browsers (including ns4) which
do so, and there are sites which rely on it, no matter how broken they may be....
Can we just strip off the query paramaters? That would get rid of most
potnetially sensitive info. People who code sensitive stuff into the path
(making it work via server side support of some sort) are likely to be rare.
I dunno - thoughts?
A site checking the referrer as a security check is not "broken". How else are
webmasters supposed to prevent the mass stealing of their content and bandwidth
by other sites? The answer for the last 10 years has been they simply check the
referrer to make sure it is one of thier own servers making the request.
It isn't trivial to hack every end user's client browser to pass a specific
referrer when calling a particular url. In fact I think that would be very
difficult. If you could hack the client's browser, you could do a lot more harm
than sending a bogus referrer.
My situation is an extension of the above. We host thousands of images using
one image tag. We use the referrer to figure out which image to send. If we
don't get a referrer, we send an empty image.
Well, its 'broken' in that people can and do disable referer strings.
Personally, I don't, but there are those that do, so....
Note that what you're doing is not 'security' in the sense that I was using the
word. I meant that relying on the referer to make sure that someone only visits
your page from the login page, or has completed part 1 of the form before part
2, or something like that, is silly, and broken. You're trying to protect
yourself against a malicious thrid-party page, but an uncaring user, rather than
a malicious user, which is a different situation.
Anyway, I'm still leaning towards opening this up, both because the spec doesn't
forbid it, and for 4xp/iecompat reasons. Darin pointed out to me that this is
actually a pref ATM, so we'd just be swapping the default value.
(BTW, Bug 89995 was the other bug I was refering to earlier, where this check
was *way* too strict.)
Created attachment 99018 [details] [diff] [review]
ok, here's the patch to enable cross-site HTTPS referrers. i also thru in a
minor perf tweak suggested by bbaetz.
Comment on attachment 99018 [details] [diff] [review]
>+// This pref controls whether or not we send HTTPS referrers
>+// cross domain. By default, this is enabled for compatibility
>+// with other browsers and many websites (see bug 141641).
I'd prefer the comment be more explicit that both hosts are HTTPS
so that "cross domain" isn't misinterpreted as https->http
// Controls whether we send HTTPS referres to other HTTPS sites.
// By default this is enabled for compatibility (see bug 141641)
dveditz: yeah, that's much clearer.. thx!
Comment on attachment 99018 [details] [diff] [review]
This treats all HTTPS servers as equally trusted WRT referrees, which seems
absurd, but I recognize the compatibility issues here and I think the actual
risk is small.
Ditto on adding a more explicit comment. Other than that, r=mstoltz.
Comment on attachment 99018 [details] [diff] [review]
email@example.com for 1.0 branch. Change mozilla1.0.2+ to fixed1.0.2 when
hello, wrt comments #41 and #42:
stripping of the query string can easily be worked around using path info.
when I had my https-/referrer problem, I solved it this way. leaves as much
information as before to be shipped in the referrer...
wrt mass stealing of content: in theory, not accounting for server load
and application complexity, using time stamped urls for your images could
prevent most bogus image requests since the urls time out after a while
(just like time stamped email addresses which bounce after their time out
This is needed by a major bank customer in Japan. CC'ing
jaimejr and buckland. Please make sure that this will be
in the next NS release.
tever: pls verify this as fixed on the trunk. thanks!
This is what I am seeing:
Referers passed from HTTPS to HTTPS (original problem now working)
" " " HTTP to HTTP
" " " HTTP to HTTPS
NOT passing Referers from HTTPS to HTTP
which from my understanding is how it is supposed to work.
internal tests at http://bubblegum/tever/refererSSLb.html and
verified trunk - 09/18/02 builds - winNT4, linux rh6, mac osX
Adding a dependency to bug 150046, as it is needed by a major embedding customer
in the JA market.
In and email I asked Kat if fixing this bug was both necessary AND suffient to
get Mitsui-Sumitomo to support us. I interpret the answer below to mean that if
we fix this bug all known blocks to their support are cleared.
We have word from Japan that this is the blocking issue and they have no other
issue currently. We want this for getting Mitsui-Sumitomo to support us. This is
a vital step in our overall plan to get top 24 banks to support us.
If the committee needs to talk to me about this, I will be happy to bring any
info you need.
adt1.0.2+, already has drivers approval -- please land on the 1.0 branch.
tever: can you pls verify this as fixed in tomorrow's 1.0 branch builds? thanks!
verified1.0.2 - 09/24/02 builds - winNT4, Linux rh6, mac osX
the extension conflicts with a plugin of this page.
so error message open continuously.
even if it is off, error causes.