Closed Bug 83038 Opened 23 years ago Closed 23 years ago

mailnews reveals local mail path in referer

Categories

(MailNews Core :: Networking, defect, P1)

Product:
MailNews Core
Component:
Networking
Platform:
x86
Windows 2000
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
mozilla0.9.2

People

(Reporter: bugzilla, Assigned: mscott)

References

Details

(Keywords: regression, Whiteboard: [PDT+])

Attachments

(2 files)

a patch demonstrating a possible solution
1.26 KB, patch
Details | Diff | Splinter Review
This is the almost the same os the previous patch, except it is a white list of http and https.
1.16 KB, patch
Details | Diff | Splinter Review 
when viewing a html message with references to objects on a web server my
firewall says

Blocked Referer: mailbox:///[local path omitted] sent to
http://hotwired.lycos.com/news/ibd/wired-news/other/images/[info omitted]

This behavior reveals the random name of the mail subdir and the name of my
mailserver.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Anyone know the correct RFC behavior off-hand?
For referrer?

rfc2616 says:

14.36
"
The Referer field MUST NOT be
   sent if the Request-URI was obtained from a source that does not have
   its own URI, such as input from the user keyboard.
"

Security concerns are mentioned in section 15.1.3, which really only says:

"
 Clients SHOULD NOT include a Referer header field in a (non-secure)
   HTTP request if the referring page was transferred with a secure
   protocol.

"

(Do we follow that?)

However, we just shouldn't be sending local paths ever, even with file urls (I
don't know if we do for file:///). I'd argue that nothing should be sent, (think
1x1 tracking gifs in mail), but we could possibly just send: mail://, or
news://<msgid>. I don't know if we should send the msgid though.
Reassigning to Networking. Let's look carefully at when we send referrers. If 
the referrer is a mail message we should never send the referrer header, and 
there are probably other circumstances as well. I tend to agree with Bradley 
that maybe we should never send local file paths as referrers, as that reveals 
information about the local file system.
Assignee: mstoltz → mscott
Component: Security: General → Networking - General
QA Contact: junruh → huang
Target Milestone: --- → mozilla0.9.2
Ccing David
Whiteboard: [PDT+]
Priority: -- → P1
I'm afraid I'm a bit lost on what exactly the bug is here that I need to fix.
Are we somehow running an HTTP url from the mail window and we are explcilty
setting the referrer header from mailnews? I'm pretty sure we don't have any
mailnews specific code that does that. Or is someone else setting the referrer
on this http url before we run it based on the document url of the message pane
(which would be a mailnews url). I'm guessing the later which means someone who
owns the setting of http referrers should get this bug. 
I'm guessing the latter. Should this go to Necko?
darin, does http control the setting of the referrer header? i.e. should we send
this bug to your group?
HTTP does not know how to set the referrer header on its own.  Someone has to
explicitly tell HTTP to set a referrer.  See nsIHttpChannel::SetReferrer.
Thanks Darin. Mitch, it looks like the main caller of HTTP channel's SetReferrer
is nsDocShell on line 5183. Just so I understand the problem fully, you are
saying we should not make this call if the referrer url has one of the following
schemes:

file
mailbox
imap

any others we should black list as well? I'll try to do them in one fell swoop.
actually I see image lib also sets the referrer. So we need to check all these
urls in 2 places. I wonder if it makes more sense to modify
nsHTTPChannel::SetReferrer to block certain url schemes instead of trying to
maitain this list in a couple of places. 
Mitch/Darin, here's a patch which shows what I mean by modifying
nsHTTPChannel::SetReferrer to add protection against passing urls containing
sensitive data in as referrer urls. I felt a bit hokey making up the black list
since I picked url schemes arbitrarily. I'm hoping there's an official list we
can come up with. I just listed: imap, imaps, news, snews, mailbox and file urls
as urls we never want to give out as referrers. 

While it feels a bit weird having this list of protocol schemes in the http
channel, I think it makes the most sense to have this kind of blocking happening
there instead of in the several places were the referrer is set on an http channel.

Questions:
1) Objections or different solutions to this kind of approach?
2) What other url schemes do we need to block?


On my todo list is a general set of HTTP test for things like basic auth, proxy
auth and referer, so when mailnews qa needs to test this, I'll be glad to try
and help with the testcases.
This is a regression, at least for file:/// URLs.  I'm pretty sure we used to
not send file:/// referers.  (Bug 16442 is marked as fixed for M14.)
Keywords: regression
mscott: 

i'm a bit torn: i see the merits of your solution... it is simple and direct.  
but, it's not exactly very customizable/dynamic.  maybe that's not such a big 
deal at the moment, so r/sr=darin.
I feel your pain Darin. I'm open to other alternatives too. I couldn't think of
any. 

Mitch, what's the exact list of protocols we need to be blocking besides the
ones I've listed here? I'm sure there are more....
The only ones I can think of to add are resource: and chrome:. Will these ever
be used as referrers?

r=mstoltz with that addition.
In response to Darin's comment, if you want to take the time, you could always
create a (hidden) pref specifying the list of schemes. That would make it easier
to change the list later.
a=dbaron for trunk checkin (on behalf of drivers)
fix checked in. To verify, generate an HTTP protocol log. From mail, click on a
message that has an http url in it. Now click on the http url. Do this for
mailbox, news, snews, imaps, file (create a test file url with a http url in it)
urls. Verify in the http log that when we open up the http url, we don't pass in
these urls as the referrer.
Status: NEW → RESOLVED
Closed: 23 years ago
Resolution: --- → FIXED
*** Bug 88331 has been marked as a duplicate of this bug. ***
Verified on linux for file and imap, (as part of duping bug 88331)
question - why list all the protocols which should be ignored by 
nsHTTPChannel::SetReferrer instead why not make a list of the protocols
which should be allowed - this is a much more robust approach in the
face of new protocols being added.


After confirmed with Scott, changing QA Contact to Ben.
QA Contact: huang → benc
I'm sending this back to mailnewqa and here's why:

My thinking in taking this back was that this was a Necko feature that we could
whitebox test. Bradley tells me I'm wrong (this is why I do black box and he
does white box...)

So in that light, this is a blackblox mailnews bug that needs to be tested. I
spent about an hour today trying to create a message that sent a mailbox: or
imap refer to my test server, and I didn't log anything in Netscape 6.0 or in
last weeks FC.

Since there are no steps (or sample mail) here, I think mailnews qa will need to
produce a test case, and test it.

I have configured a special instance of a web server on www.packetgram.com:81.
The log file is configured to log basic info and the refer line:

http://www.packetgram.com/pktg/mozilla/bugzilla/83038/referlog.txt

So, I suggest that someone create some test cases that point to
http://www.packetgram.com:81/ and see what the web server logs.
If you links to other types of content (like images) let me know.

re: black list vs. white list...

Bradley had some strong feelings about how this should work.
QA Contact: benc → huang
> Bradley had some strong feelings about how this should work. 

I did? All I said was that programatically scripting a click on a link from a
mailnews window wasn't an option. I could send you a mail which loads a url in
an iframe or something, and loads a cgi on a server wich tells you if there was
a referrer or not (and thats how I verified part of this bug...)
Over to neetworking. This bug has nothing to do with mailnews. Please stop
re-assigning it to mailnews qa. The change was to our http protocol
implementation. We block the referrer url for http for the following url schemes:
file
mailbox
imap
chrome
resource
snews
news

I listed steps to QA this bug in an earlier comment when I marked the bug fixed.
 This is not a mailnews bug. It's a http bug. If you aren't the QA contact for
changes to the HTTP module then please assign it to the right person for http.
Thanks Ben.
QA Contact: huang → benc
mscott: The problem blocking verification is that I can't get the problem to
happen in Netscape 6, so I can't really verify it.

STEPS:
Using Netscape 6.0.1, Windows NT
Create an email with a link in it (http://www.packetgram.com:81/)
Send the mail to myself.
Open the mail.
Click on the link:
Look at the refer.log entries that are created:

208.12.40.226 "24/Jul/2001:12:17:41 -0700" "GET / HTTP/1.1" "-"
208.12.40.226 "24/Jul/2001:12:18:19 -0700" "GET /launch.html HTTP/1.1" "-"
208.12.40.226 "24/Jul/2001:12:18:19 -0700" "GET /banner.html HTTP/1.1" "-"

Furthermore, I launch.html and banner.html are gotten as frames from getting
"/", and they have no referers either.

Here's a log from getting the same home page via a new window, you can compare
the entries and see how getting launch.html and banner.html were refered from
getting "/"...

205.217.228.69 "24/Jul/2001:00:22:30 -0700" "GET / HTTP/1.1" "-"
205.217.228.69 "24/Jul/2001:00:22:31 -0700" "GET /launch.html HTTP/1.1"
"http://www.packetgram.com:81/"
205.217.228.69 "24/Jul/2001:00:22:31 -0700" "GET /banner.html HTTP/1.1"
"http://www.packetgram.com:81/"

Chrisof: can you forward me the email that resulted in this firewall log?


UPDATE: 
NS6.01 RTM was to far back, so mscott asked me to look at builds before and
after the checking (okay, yeah, I was in a hurry and just ran what I had...)

Mozilla 0.9.2 (post checkin):

208.12.40.226 "24/Jul/2001:13:47:47 -0700" "GET /banner.html HTTP/1.1"
"http://www.packetgram.com:81/"
208.12.40.226 "24/Jul/2001:13:47:46 -0700" "GET / HTTP/1.1" "-"
208.12.40.226 "24/Jul/2001:13:47:47 -0700" "GET /launch.html HTTP/1.1"
"http://www.packetgram.com:81/"
reproduced problem in mozilla 0.9.1, so it looks fixed 0.9.2 and later.

208.12.40.226 "24/Jul/2001:14:20:33 -0700" "GET / HTTP/1.1"
"imap://dredd.mcom.com:143/fetch%3EUID%3E/INBOX%3E58207"
208.12.40.226 "24/Jul/2001:14:20:33 -0700" "GET /banner.html HTTP/1.1"
"http://www.packetgram.com:81/"
208.12.40.226 "24/Jul/2001:14:20:33 -0700" "GET /launch.html HTTP/1.1"
"http://www.packetgram.com:81/"
+makingtest. I'll make a more complete test matrix in the future.
Status: RESOLVED → VERIFIED
Keywords: makingtest
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: