Closed Bug 8743 Opened 25 years ago Closed 25 years ago

Cookie not set error ?

Categories

(Core :: Networking: Cookies, defect, P3)

Product:
Core
Component:
Networking: Cookies
Platform:
x86
Windows NT
Type:
defect

Tracking

()

Status:
VERIFIED FIXED

People

(Reporter: slok00, Assigned: morse)

References

(
URL
)

Details

Using M7.

Site visited : http://mail.yahoo.com
M7 Preference set : enable Java, enable JavaScript, accept all cookies

error from site : An Error Occurred Setting Your User Cookie

Tried changing preference to :
accept only cookie that got sent back to originating server

error : loaded the frameset for Yahoo! mail.
Left frame is ok, right frame with error saying login session expired
and give possible reasons of cookies not set correctly.

Tried changing preference to :
Disable all cookies

error : same error as above when preference is set to accept only cookies
that got sent back to originating server.
Status: NEW → ASSIGNED
Target Milestone: M8
*** Bug 7111 has been marked as a duplicate of this bug. ***
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
This is a nasty problem that was fixed in 4.5 but apparently the fix never made
it to 5.0.  Here's what was happening.

There is a serious logic bug in the useage of cookies whereby a site such as
a.b.co.nz can set cookies for the entier .co.nz domain.  To fix that we decided
to strictly enforce the cookie spec whereby a site could not set a cookie for a
domain two levels up (i.e., a.b.co.nz could set it for .b.co.nz but not for
.co.nz).  This was really just a band-aid for the particular bug report that we
had -- it really didn't fix the problem (because b.co.nz could still set cookies
for all of .co.nz) but it did let us close out the bug report which was already
posted on the web.

Lo-and-behold, we discover that our fix, although the correct thing to do, was
breaking sites such as yahoo.com.  Reason being that these sites were themselves
violating the cookie standard and setting cookies for domains that don't start
with dots.  In particular, in yahoo mail, site f12.mail.yahoo.com sets a cookie
for the domain mail.yahoo.com rather then .mail.yahoo.com.  Because the leading
dot in the domain is missing, we interpret this as trying to set a cookie for
.yahoo.com and that was being caught be our new fix and being rejected.

Bottom line is that the bug is in yahoo.com setting cookies incorrectly (no
leading dot in domain name).  But we can't now go breaking all the sites that
are doing this wrong and got away with it in our earlier browsers, so we had to
back out our fix.  It was backed out of 4.5 but was forgotten about in 5.0.  So
I just backed it out.  (Sorry about the .co.nz -- you are now in trouble again.)

Incidentally, all is not lost.  When we backed out the change, we did leave the
more-strict cookie test in but put it under control of a preference.  That
preference is not in the preference panel but can be set by hand for anybody who
is concerned about .co.nz.  The preference is network.cookie.strictDomains which
should be set to true to get the strict cookie testing.
FWIW, the bugsplat bug related to this issue is 338501 (and its duplicate,
namely 340166).  You can read all about it, how the band-aid fix was introduced,
and then backed out.  I checked the band-aid fix into the 5.0 codebase on
December 31, 1998 and into the 4.x (nova) tree on Jan 5, 1999.  Then jpm backed
out of nova on Feb 11 but that back-out never made it to the 5.0 codebase.
Thank you.  This is exciting--now I know why I couldn't check my mail at Yahoo!
This means, for my purposes Mozilla is now dogfood ready, since it will deal
with practically all of my essential browsing demands.

One question: Where can people like me outside of Netscape's firewall see this
bugsplat report you allude to?
The bugsplat report for 338501 is very informative but also very lengthy.
However, since this bug report is now fixed and closed, there shouldn't be many
more comments made here so I'll close it out by flooding it with the complete
set of comments from bugsplat 33801.  Turns out that there is a limit to the
number of characters that can be entered in a comment and the bugsplat report
exceeds that limit.  So I'll need to split it into two parts and enter each
part into a separat comment.  I hope this works.  Here the first part:

*****************************************

The problem is related to the Netscape web browser and the cookie mechanism.
Customer have built a middleware Web-to-Host Unix server solution for their
client in New Zealand.

The middleware gets requests from the web server, gets the information and sends
back a HTML page to the web server which in turn sends it back to the user who
requested it.  The middleware logic uses cookies as a means of keeping the
session information for a particular user (mainly to be able to identify a user
uniquely during a  session).  When a user requests a dynamic page from the Web
server, a cookie is generated (e.g. Name=Session ID,
Value=nz19981001043592a005r9) and sent to the web browser.  The cookie is
expected to reside in the web browser's memory until the Web browser exit.  They
do not set any expiration such that the cookie never gets set in the
"cookies.txt" file.  On all subsequent requests from this user, the cookie is
sent by the Web browser to the web server where upon the middleware identifies
the user and the user request is fulfilled.

All of this mechanism is working on all of the web browsers until recently they
had following problem:

The cookie sent by the web server is NOT getting  set in the web browser.  The
options "Enable Cookies" and "Warn me before accepting a cookie" are both turned
ON such that the user does see that the cookie has arrived, the user clicks
OK but the cookie is not set.  We know this because on the subsequent requests,
the web server does not receive the cookie !!  This causes the web server to not
identify the user and yet another cookie is sent to the browser.  And the
problem goes in a loop !!  This clearly means that the cookie is not being sent
back to the web server.

An observation seen by the user is that if the "cookies.txt" file is removed,
then the cookie gets set in the browser and does indeed get sent on the
subsequent requests and everything starts working fine.

All of this has led us to believe that some combination of "cookies.txt" and the
Netscape  browser is causing the Netscape Browser to not send the cookie back to
the web server.

Here is how you can re-create the problem: Assuming you are on Windows NT and
Netscape Browser 4.06

1) Replace your "cookies.txt" file with the customer's "cookies.txt" file. You
can get it here: /u/van/public_html/sabre/cookies.txt

2) Please turn ON the Cookies and also turn ON the warning before accepting the
cookie on Edit-Preferences on the Netscape browser.  Using these settings you
can actually see the cookie coming in to the browser.

3) Go to http://www.airnz.com/

4) On the Welcome/Home page you have to "Select your Home".  Please select "New
Zealand" as your home country.

5) On the next page (which has a picture of a lady and a message starting with
"Travel the World ...") select ROUTES/SCHEDULES option on the top left corner.

6) On the next page (which has a picture of a beach and a message starting with
"There are few true beautiful ...") select "Schedules" on the bottom left
corner.  This will prompt you with a cookie acceptance window.  The cookie
contents will be something like Name=Session ID, Value=nz19981001043592a005r9.

7) Please accept the cookie.

8) Now you get the form where you can select cities to get a Flight Schedule.
Please select the dep/arv city codes e.g.  Auckland (AKL) and Brisbane (BNE)
and click on the "One Way" button.

9) Now if you get the same page back with a cookie, you have re-created the
problem.  What happened here is that the first cookie did not get set or sent
back to the Web server.  The web server thought this person to be a new user
and tried setting the cookie again, looping the user on the same page !!
 - On the other hand if you get a flight schedule after clicking on "One Way",
this means you could not recreate the problem.

10) If you are able to re-create the problem, now you can repeat the same
exercise above but remove the "cookies.txt" file from you directory.  This
should eliminate the problem.

After further investigation, we have narrowed down the problem to the following
line the cookies.txt:

.co.nz  TRUE    /       FALSE   942189160       INTERSE 192150897172547

If the above line is removed, this corrupted cookies.txt file become good again.
From this, it looks like some web site in New Zealand is sending out an invalid
and the browser is accepting it.  This cookie may be sitting on all these
user's cookie file and causing the problem.  It also appears that every time the
user (with above mentioned cookie) goes to any of the *.*.co.nz site, above
cookie is sent and not the one from that server.

From our understanding of cookies, from:

http://developer1.netscape.com:80/docs/manuals/communicator/jsguide4/cookies.htm
#1003254

Determining a Valid Cookie

When searching the cookie list for valid cookies, a comparison of the domain
attributes of the cookie is made with the domain name of the host from which
the URL is retrieved.

If the domain attribute matches the end of the fully qualified domain name of
the host, then path matching is performed to determine if the cookie should be
sent. For example, a domain attribute of royalairways.com matches hostnames
anvil.royalairways.com and ship.crate.royalairways.com.

Only hosts within the specified domain can set a cookie for a domain. In
addition, domain names must use at least two or three periods. Any domain in the
COM, EDU, NET, ORG, GOV, MIL, and INT categories requires only two periods; all
other domains require at least three periods.

PATH=pathName specifies the URLs in a domain for which the cookie is valid. If a
cookie has already passed domain matching, then the pathname
component of the URL is compared with the path attribute, and if there is a
match, the cookie is considered valid and is sent along with the URL request.
For example, PATH=/foo matches /foobar and /foo/bar.html. The path "/" is the
most general path.

---

So, the likely source of the cookie entry is some site at the .co.nz level.

The reason the domain cookies have a minumum length is that these are the
minimum levels at which DNS subdomains are delegated. For example, requiring
"two periods" means that you can set a cookie for "netscape.com." if you are a
host in netscape.com.

It prevents you from setting cookies at the com level, which would allow you to
set cookies for another company. This would be a terrible security hole, our
site could push cookies that broke things in the microsoft.com domain.

The reason non-three letter "main top-level domains" get three periods is that
there is an extra layer of delegation (for each country name). So, this means
that you should have cookies of at least three parts: <3rd.2nd.top>. This means
that a host in "host.net.taiwan" would be prevented from pushing cookies to
clients that would affect the rest of "net.taiwan".  So, there are two problems,
some people have servers pushing out cookies at a level that they shouldn't.

The .co.nz cookies are invalid. However, the client should recognized that
the cookies are invalid and reject it. If the client accepts these bad cookies,
there could potentially be a security hole as described in the above scenario.



------- Additional Comments From hannigan  10/15/98 13:23 -------

Assigning to anthonyd for triage and testing

Who is the customer?


------- Additional Comments From valeski  10/15/98 14:33 -------

I'm just getting into this.
1. This looks very bad.
2. Considering someone is just noticing this, it goes to show that
international weblications aren't being used/built too often.

------- Additional Comments From valeski  10/15/98 14:40 -------

If someone can come up with a mulit-level domain name server we can test with
that would be good.

------- Additional Comments From valeski  10/15/98 14:47 -------

we're sending the INTERSE (from some other server) cookie and the sessionID
cookie to thier server.

------- Additional Comments From valeski  10/15/98 14:48 -------

Workaround (does not address the security issue)

Parse the sessionID cookie string that we send back out of the entire cookie
string. They're just expecting the entire cookie string to consist of their
sessionID cookie, but, we're actually sending.

INTERSE=xxx;sessionID=yyy.

------- Additional Comments From valeski  10/15/98 14:51 -------

I'm testing this with a day old 4.5 build

------- Additional Comments From valeski  10/15/98 14:59 -------

How the hell our online docs (this is the root/most standard one
http://www.netscape.com/newsref/std/cookie_spec.html) were written to account
for these cases and there is no code to back them up is beyond me. We don't
check for this case.

------- Additional Comments From valeski  10/15/98 15:18 -------

The probable fix for this is to simply do what the spec says. The fix is easy,
but, testing it is not. This functionality has been there since 1.0 and changing
it would break someone without a doubt.


------- Additional Comments From vanh  10/15/98 15:49 -------

To answer hannigan's question, customer is Sabre Group.

------- Additional Comments From valeski  10/15/98 16:04 -------

Brendan confirmed my suspicions. This is an ancient bug and has been debated
many times on several fronts.

1. We'll break too many people if we fix it.
2. Anyone inside a second level domain ".ca.us" can "sort of/sometimes" be
trusted with the cookies of others in that domain.

It's a privacy issue, and a serious one at that. We should not hold 4.5 for this
as the ramifications of the fix are unknown and we need a better understanding
of it.

The fix is to do what the spec says we already do.

------- Additional Comments From valeski  10/15/98 16:10 -------

no need for the multi domain testbed. I hacked my client to mimic it.

------- Additional Comments From valeski  10/15/98 16:11 -------

hat's off to pual macquiddy for noticing the cookie location is bad. it should
be vanh instead of van

------- Additional Comments From morse  10/15/98 16:13 -------

I think we will still have a problem if we implement what the spec says --
namely checking for three periods for domains not in the catagories mentioned.
For example, a typical domain for a school is
   http://Bellmore-merrick.k12.ny.us/
That means that if some site sets a cookie for .k12.ny.us, then this would match
any k12 school in New York.  So we haven't solved the problem.

------- Additional Comments From valeski  10/15/98 16:13 -------

my bad, it's paul :)

------- Additional Comments From leger  10/15/98 16:48 -------

Moving to 4.51.  Please feel free to put on 5.0 if that's where it should be.
Missed the nova train.

------- Additional Comments From benc  10/15/98 21:02 -------

RE: k12.ny.us problem.

According to the spec this is okay. I guess you are going to have to let it go.
Domain cookies are inherently bad if you ask me, if it's just something you
can't live with, I would make it yet another setting.

I missed the concept of domain level cookies the first time I read the spec, so
having only realized they exist for about a two days, I am still in shock. No
matter what level you restrict the domain level cookies, you are always betting
that the web developers are beholden to the hostmasters that delegated name
space if a conflict occurs.

For now, I think we should do the spec, then deal with the big picture problem
(if possible). Maybe someone will update the cookie spec as fallout to the re-
write of DNS needed for IPv6, or just get rid of it. The whole idea is kind of
unwieldy. Maybe the spec needs to be re-written so the domain level cookies are
triplets: sourcehost,label,value. This would allow a host to received a domain
level cookie while still being able to filter out hosts that are not acceptable.


------- Additional Comments From jar  10/16/98 01:33 -------

Unless I'm missing the issue, this is not a security problem.  If a server app
is silly enough to set the DOMAIN to be a small enough subset (postfix) of the
server host name, then the cookies that are set will be spammed across all
similarly postfixed servers. Such a silly server app deserves the
loss-of-privacy of the cookies *IT* sets. Similarly, anticipating such spam, a
server app must not assume that no other cookies would appear during an HTTP
interaction (this seems to be the bug in this bug report... it is just that this
server app is not coded with defensive coding... and gets bent out of shape when
a spam-cookie appears.).

The goal of the "must have n dots" restriction in the spec was to prevent folks
from spamming all the servers.  It would be a shame if you could cause tons of
stuff to be sent to all *.com servers.... imagine how bad the performance would
then be if the cookies were large!  An attacker could make your browser really
drag... and bother all the servers that you visited in such a domain.  The bad
news is that it is nearly impossible to define such a rule stating that the
"postfix must have at least n dots" and hope to block spam (accidental or
intentional).

The critical point is that cookies that are set from well written server apps
will never face this problem... as no "evil" server can exist that can capture
its cookies.  Simply put: Poorly written apps can give away their cookies.  Well
written apps can defend their cookies by using nearly full host names in the
cookie setting.

The sad thing is that cookie-spammers (folks that diminish client performance by
setting cookies that are sent to many sites, for no use other than wasteing
bandwidth) can cause cookies to be sent to many sites... and there is not that
much that can *automatically* be done about it.  We could try to verify via DNS
that the postfix of the hostname really corresponds to a computer... rather than
just being a large diverse domain name... IMO, this is the only method that has
a chance at succeeding.  Trying to list postfixes, and count dots, is surely a
loosing adventure.

Am I misunderstanding the problem????  Aren't all cookies that are valid sent to
a given host... an hence wouldn't "spam cookies" be ignorable (if the server app
was coded defensively)??

Thanks,

Jim

------- Additional Comments From vanh  10/23/98 12:32 -------

I forwarded valeski's workaround to the customer. They came back with some
questions, can someone please help me address them:

Workaround (does not address the security issue)

Parse the sessionID cookie string that we send back out of the entire cookie
string. They're just expecting the entire cookie string to consist of their
sessionID cookie, but, we're actually sending.

INTERSE=xxx;sessionID=yyy.

Questions from customer:

We checked in our code and we can parse the cookie string. Few questions
regarding the cookie being sent from the web browser.

1) Confirmation that our cookie is definitely being sent along with the faulty
cookie.

2) Confirmation that the cookies string will always have any one of the
following syntax when one or more cookies are being sent (Our "SessionID=yyy"
cookie string may appear anywhere in the complete cookie string).

 a) sessionID=yyy - No problem case

 Problem cases with current parsing:

 b) AnyCookieName1=xxx;sessionID=yyy
 c) AnyCookieName1=xxx;sessionID=yyy;AnyCookieName2=aaa
 d) AnyCookieName1=xxx;sessionID=yyy;AnyCookieName2=aaa;AnyCookieName3=bbb
  and so on ..
 e) AnyCookieName1=xxx;AnyCookieName2=aaa;sessionID=yyy;AnyCookieName3=bbb
  and so on ..

3) Is there a dot "." at the end of the string ?

 Please let us know.  Thank you very much for all the help.



------- Additional Comments From valeski  10/23/98 13:24 -------

1. As long as they continue setting their cookies as they have been (i.e. using
their current domain attribute value).
2. Confirmed.
3. there is no dot.

------- Additional Comments From vanh  10/28/98 09:34 -------

customer to workaround the problem by parsing their cookies as valeski
suggested. Awaiting customer's feedback.

------- Additional Comments From vanh  11/02/98 22:03 -------

Customer feels they have enough information to work-around the problem for now.
They will parse their cookies as suggested. However, they would like Netscape to
pursue a permanent fix to this matter and would like to know when we'll fix this
permanently (Assuming that there is one).

------- Additional Comments From hannigan  11/03/98 09:48 -------

Since workaround has been accpted, I propose we change TS level to TS2.
Comments?

Ray, if none by close of biz today, reduce to TS2.

------- Additional Comments From vanh  11/03/98 10:57 -------

reduced to TS2.

------- Additional Comments From raym  11/10/98 15:49 -------

judson , are we still on for 4.51 time frame??

------- Additional Comments From gagan  11/10/98 16:33 -------

Jud's gone and in light of the schedules, I'm marking this for 5.0

------- Additional Comments From jar  11/10/98 17:21 -------

Note that Judson's last day was last friday.

------- Additional Comments From morse  11/10/98 17:59 -------

Well, much as I hate to admit it, I guess I'm next in line for this bug now that
Judson is no longer with us.  I know what Judson had intended to do here and
will make the fix in the raptor (5.0) codebase.  Am reassigning this to myself.

Here is the fix I'm going to make.  Reject setting of domain cookies unless the
host is in the lowest-level domain.  In other words, if the url of the host is
a.b.c.d.e, then the only domain it can set a cookie for is .b.c.d.e; it cannot
set a cookie for .c.d.e or for d.e.  Yes, this will break some existing site
implementations, but it will close all security (privacy?) holes.  If anyone
objects to this fix, speak now or forever hold your peace.

Alternatively we could do nothing about this because, as jar rightfully points
out, this is really neither a security nor a privacy hole but rather a means of
allowing a poorly-written site to spam its cookies.  So, at worst, it's a
performance issue.  If people want to take this viewpoint and consequently not
make any changes, speak up quickly.

------- Additional Comments From jar  11/10/98 21:06 -------

The change that you describe is actually (potentially) worse than we have now.

Visit the site:  http://monkeys.com

Would you allow him to set cookies in ".com"?   If so, it would be worse than
what we have now :-(.

------- Additional Comments From morse  11/10/98 23:28 -------

Oops, jar's right.  I was sloppy and didn't state all the conditions.  In a
private message that I received from Judson when he was investigating this,
he told me that the new cookie spec is described in a draft rfc 2109 which can
be found at

   http://www.cis.ohio-state.edu/htbin/rfc/rfc2109.html

In particular, section 4.3.2 of that document states the conditions.  Roughly
speaking, it is what I said above with the added condition that the request is
rejected if the requested domain does not contain an embedded dot.  Here is the
official wording:


4.3.2  Rejecting Cookies

   To prevent possible security or privacy violations, a user agent
   rejects a cookie (shall not store its information) if any of the
   following is true:

   * The value for the Path attribute is not a prefix of the request-
     URI.

   * The value for the Domain attribute contains no embedded dots or
     does not start with a dot.

   * The value for the request-host does not domain-match the Domain
     attribute.

   * The request-host is a FQDN (not IP address) and has the form HD,
     where D is the value of the Domain attribute, and H is a string
     that contains one or more dots.

   Examples:

   * A Set-Cookie from request-host y.x.foo.com for Domain=.foo.com
     would be rejected, because H is y.x and contains a dot.

   * A Set-Cookie from request-host x.foo.com for Domain=.foo.com would
     be accepted.

   * A Set-Cookie with Domain=.com or Domain=.com., will always be
     rejected, because there is no embedded dot.

   * A Set-Cookie with Domain=ajax.com will be rejected because the
     value for Domain does not begin with a dot.



------- Additional Comments From gagan  11/12/98 13:52 -------


Some comments from Jud-

morse's last comment clarifies why the fix is good. This fix is the right thing
to do. Note: (please put this in the bug) rfc2109 is
*not* the new cookie spec it is the current *proposal* and we can suggest
changes/additions to it via the ietf.

Jud


------- Additional Comments From granrose  12/30/98 11:37 -------

I'm writing a statement on this for techsup.  For the record, are we planning on
implementing the entire proposed RFC 2109 spec, or just the rules specified in
section 4.3.2?

------- Additional Comments From morse  12/30/98 11:55 -------

Just the rule specified in section 4.3.2.

------- Additional Comments From morse  12/31/98 13:51 -------

I have the fix ready and have tested it out.  Now what do I do with it?  The
nova tree is closed and has been since October.  Somebody please advise.

BTW, I have just checked the fix into the 5.0 tree.  That closed out bug report
329292.

------- Additional Comments From morse  12/31/98 14:02 -------

My previous comment referred to the 5.0 bug as 329292.  That should have said
329392.

------- Additional Comments From jpm  12/31/98 14:05 -------

Steve,

Can you please come to the 4.51 bug review meeting on Tue 1/5 in Twinkies, Bld.
22, 2nd Floor?

Thanks,

Jussi-Pekka

------- Additional Comments From jpm  01/05/99 11:14 -------

Marking as 4.51in as per the BRB 1/5/99.


------- Additional Comments From morse  01/05/99 11:18 -------

Approved for check-in at this mornings Nova bug-review meeting.  Fix was checked
in.  Resolution marked as FIXED.

------- Additional Comments From hannigan  01/12/99 13:07 -------

edwyn, can we test for this ? if unclear, pls reach out to morse to develop
valid test cases....

------- Additional Comments From morse  01/12/99 13:35 -------

Yes, we can test for this.  The test is in the URL listed in the URL field at
the head of this bug report.

------- Additional Comments From abenes  01/12/99 16:44 -------

verified fixed against Jan 11, 1999 Win32 build . I used the testcase outlined
in bug #329392 and it appears to be using the non-generic domain versus the
generic domain co.nz as expected. one more note web sites that rely on generic
domains to retrieve cookies such as yahoo's web mail client do not work under
this new implementation.

------- Additional Comments From paulmac  01/12/99 17:23 -------

Edwyn, can you be a little more specific. What broke?

------- Additional Comments From morse  01/12/99 18:38 -------

I think I can answer that (What broke).  Edwin is saying that yahoo's webmail is
relying on domains that are more than one level up from the host -- i.e., host
w.x.y.z is setting a cookie for domain .y.z.  In that case the bug is in yahoo's
webmail since they are in violation of the cookie standard.  But we are going to
get dinged for that and not yahoo!  What to do?  We either are in compiance with
the cookie spec and thereby break yahoo's webmail.  Or we are not in compliance
and have the so-called privacy hole announced publicly on the web.  We are in a
lose-lose situation!

------- Additional Comments From hannigan  01/12/99 19:22 -------

Your right - this was discussed at a bug meeting and it was determined that this
was the best scenario to be in .....


------- Additional Comments From morse  01/12/99 19:34 -------

I agree.  It's just unfortunate (or maybe it's fortunate) that we are breaking
such a popular site as yahoo.  Maybe somebody should notify them and give them a
chance to modify their cookie usage before our fix is released.

------- Additional Comments From paulmac  01/13/99 11:19 -------

So I'm not a cookie expert, but this bug dealt with non-generic domains, not
generic domains like yahoo.com. Why are we breaking them? The privacy issue did
not deal explicitly with generic domains. Is there not separate code for the two
different types of domains, due to different assumptions of the security needed?

Am I completely off-base here? I think more discussion is needed. I can't
believe we really made a conscious decision to break the #1 site on the
internet. This would be a nightmare.

------- Additional Comments From jpm  01/20/99 12:32 -------

Here's how Yahoo sets their mail.yahoo.com cookie:

Send 518 bytes on stream 4.
<00000779< GET /config/mail?.intl= HTTP/1.0
<0000079B< Referer: http://login.yahoo.com/config/login?.src=ym&.lg=us&.don
<000007DB< e=http%3a//edit.my.yahoo.com/config/mail%3f.intl=
<0000080E< Connection: Keep-Alive
<00000826< User-Agent: Mozilla/4.5 (Macintosh; U; PPC)
<00000853< Host: edit.my.yahoo.com
<0000086C< Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
<000008A9< image/png, */*
<000008B9< Accept-Encoding: gzip
<000008D0< Accept-Language: en
<000008E5< Accept-Charset: iso-8859-1,*,utf-8
<00000909< Cookie: Y=v=1&n=eul9jjol82p7c&l=1k337p8bb0_rqq/o&p=m1v2rqr0110g&
<00000949< lg=us; T=z=y5ip2Ay/3p2AkNK1V/LSRqpMgYyMDczME8xNzI-
<0000097D<

Receive 348 bytes on stream 4.
>0000114E> HTTP/1.0 302 Found
>00001162> Date: Wed, 20 Jan 1999 19:28:54 GMT
>00001187> Location: http://f5.mail.yahoo.com/py/ymTop.py?y=1&.rand=ak59ldt
>000011C7> 1kbrff
>000011CF> Content-type: text/html
>000011E8>
>000011EA> <HEAD><TITLE>Document moved</TITLE></HEAD>
>00001215> <BODY><H1>Document moved</H1>
>00001233> The document has moved <A HREF="http://f5.mail.yahoo.com/py/ymTo
>00001273> p.py?y=1&amp;.rand=ak59ldt1kbrff">here</A>.<P>
>000012A2> </BODY>

Send 535 bytes on stream 5.
<0000097F< GET /py/ymTop.py?y=1&.rand=ak59ldt1kbrff HTTP/1.0
<000009B2< Referer: http://login.yahoo.com/config/login?.src=ym&.lg=us&.don
<000009F2< e=http%3a//edit.my.yahoo.com/config/mail%3f.intl=
<00000A25< Connection: Keep-Alive
<00000A3D< User-Agent: Mozilla/4.5 (Macintosh; U; PPC)
<00000A6A< Host: f5.mail.yahoo.com
<00000A83< Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
<00000AC0< image/png, */*
<00000AD0< Accept-Encoding: gzip
<00000AE7< Accept-Language: en
<00000AFC< Accept-Charset: iso-8859-1,*,utf-8
<00000B20< Cookie: Y=v=1&n=eul9jjol82p7c&l=1k337p8bb0_rqq/o&p=m1v2rqr0110g&
<00000B60< lg=us; T=z=y5ip2Ay/3p2AkNK1V/LSRqpMgYyMDczME8xNzI-
<00000B94<

Receive 1232 bytes on stream 5.
>000012AA> HTTP/1.1 200 OK
>000012BB> Date: Wed, 20 Jan 1999 19:29:22 GMT
>000012E0> Server: Apache/1.3.1 (Unix)
>000012FD> Set-Cookie: YM.Login=id%3d%241%24rm%24pUlz.VNQzG3qrkgxdRO1J1%26s
>0000133D> id%3d3vOiUStEoZWc%250a%26ts%3d%253d%25a6%2500%25b6%25cc%250b%252
>0000137D> 8%250a%25d5%253f%255bpY; path=/; domain=mail.yahoo.com;
>000013B5> expires=Sat, 30 Nov 1999 00:01:00 GMT
>000013DC> Set-Cookie: YM.Pref=farm%3d5%26silo%3dms51%26v%3df%26email%3dbud
>0000141C> dhzilla_100%40yahoo.com%26head%3dbrief%26fwd%3dattach%26fontsz%3
>0000145C> dnormal%26msgwidth%3d72%26order%3ddown%26inc%3d50%26goto%3dmsg;
>0000149C> path=/; domain=mail.yahoo.com; expires=Sat, 30 Nov 1999
>000014D4> 00:01:00 GMT
>000014E2> Connection: close
>000014F5> Content-Type: text/html

Notice that there is a HTTP 302 Redirect happening from edit.my.yahoo.com to
f5.mail.yahoo.com. Perhaps we're still comparing the original host when doing
the
domain/cookie comparison?


------- Additional Comments From jpm  01/20/99 12:33 -------

Reopening as P0 Test Stopper.


------- Additional Comments From jpm  01/20/99 12:34 -------

*** Bug 340166 has been marked as a duplicate of this bug. ***

------- Additional Comments From morse  01/20/99 14:10 -------

It's fine to reopen this bug but what should I do about it?  As I mentioned
previously, were are in a lose-lose situation.  We have three choices:

1. Keep that fix that I made.  Adv: it solves the problem for the demo that it
publicly out there on the web.  And it gets us into conformance with the cookie
spec.  Disadv: it breaks yahoo webmail and possibly other sites as well that
were not in conformance with cookie spec.

2. Go back to the old way.  Adv: yahoo webmail and other existing sites still
work.  Disadv: site can set cookie for entire co.nz domain.

3. Do more sophisticated parsing on domain.  Adv: fixes problem without breaking
anything.  Disadv: requires a complete enumeration of all domain suffixes that
do not end in country designation (i.e., .com, .gov, .edu, etc) -- if ever a new
suffix is created, browser will fail.
and here's the second part of bugsplat report 338501:

***************************

------- Additional Comments From morse  01/20/99 14:18 -------

On second thought, maybe solution #3 is not that bad.  Is it not the case that
all country designations are two letters whereas all the non-country final
suffixes (.com, .gov, etc) are three letters?  If so, then I can require the
domain name to contain at least one embedded dot if the final suffix has three
letters and two embedded dots if the final suffix is two letters.  Then we can
revert back to our original logic which allowed host w.x.y.z to set a cookie for
the .y.z domain even though that is in violation of the cookie spec.  I think
this will make everybody happy.  Comments?

------- Additional Comments From leger  01/20/99 14:53 -------

I think #3 would be best for now.

------- Additional Comments From jpm  01/20/99 14:57 -------

Let's discuss this at length at the 4:30 meeting today (1/20, Twinkies, B22.)


------- Additional Comments From marek  01/20/99 15:29 -------

adding srilatha to the cc list

------- Additional Comments From jpm  01/20/99 16:18 -------

Here's a really kludgy hack around this:

Index: mkaccess.c
===================================================================
RCS file: /m/src/ns/lib/libnet/Attic/mkaccess.c,v
retrieving revision 1.99.8.1.18.9
diff -c -3 -r1.99.8.1.18.9 mkaccess.c
*** mkaccess.c  1999/01/15 18:28:47     1.99.8.1.18.9
--- mkaccess.c  1999/01/21 00:14:36
***************
*** 1447,1455 ****
--- 1447,1464 ----
   *    cookies for the entire .co.nz domain.
   */

+ /*
+  * Hack for domain_from_header="mail.yahoo.com", cur_host="f5.mail.yahoo.com"
+  */
+                       if (domain_from_header[0] != '.') {
+                               domain_length++;
+                       }
                        cur_host[cur_host_length-domain_length] = '\0';
                        dot = XP_STRCHR(cur_host, '.');
                        cur_host[cur_host_length-domain_length] = '.';
+                       if (domain_from_header[0] != '.') {
+                               domain_length--;
+                       }
                        if (dot) {
                                TRACEMSG(("host minus domain failed no-dot
test."
                                  " Domain: %s, Host: %s", domain_from_header,
cur_host));


------- Additional Comments From leger  01/20/99 17:02 -------

Per 4.51 M2 bug mtg today, a decision was made today regarding a fix which
affects yahoo.com mail.  Though the 4.51 eng team will make it work, the
implementation should be made clear with CPD Marketing should the press make
something of this later.  Adding appropriate marketing folks to this bug.

------- Additional Comments From paulmac  01/20/99 18:31 -------

I don't know what the final decision was at the meeting today, but I asked Lou
Montulli (original code and spec writer) for comments about the proposed fix,
because I knew there was some history with this bug. Here's what he says.

------------------------------------------
That was a solution I proposed to the committee a few years ago, but it doesn't
turn out to work well.  There are lots of domains that don't end in the big six
and still only have two dots and should be valid. It's likely to break several
foreign sites.

Your trying to quickly solve a problem that I have been thinking about for over
4 years.  I seriously doubt that you are going to find a good solution for it
any time soon.  JG and I knew about the problem when I designed the system in
'94, and there hasn't been a serious problem with it in all that time.  The only
plausible solution to this problem is a strong reporting system that allows
trace back or a signature/approval system, both of which cause large complexity
issues.
-----------------------------------------------------------
So has anyone thought of calling Microsoft and asking what they're doing about
this? A common solution might be good.

------- Additional Comments From morse  01/20/99 20:07 -------

Just to clarify things, the problem with yahoo mail is not what I thought.  It
is not the case that w.x.y.z is trying to set a cookie for domain .y.z.  Instead
it involved a site that was setting a cookie for a domain not starting with a
dot.  That is a totally unrelated problem which just happened to be uncovered by
the fix that I made.  jpm's hack that he's about to check in compensates for
that by having the browser be less stringint in checking for valid cookies.
This totally fixes the yahoo problem.

Be aware that we are still susceptible to the problem I just described.  If any
site is doing that, the new fixes will break that site.  Let's keep our fingers
crossed.

------- Additional Comments From jpm  01/20/99 20:24 -------

Fix checked in:

/m/src/ns/lib/libnet/Attic/../../../mcom/lib/libnet/mkaccess.c,v  <--
mkaccess.c
1.99.8.1.18.10; previous revision: 1.99.8.1.18.9


------- Additional Comments From abenes  01/25/99 12:22 -------

verified against jan 25 1998 m2 re-spin on win32

tested air new zeland web site and yahoo web site both are working as expected.

------- Additional Comments From leger  01/25/99 13:13 -------

My understanding is that this is a cross platform fixed.  Someone needs to
verify Mac and Unix before marking Verified/Fixed.  is this indeed cross
platform?

------- Additional Comments From leger  01/26/99 08:15 -------

This has been Verified fixed on the Mac with the Jan 25 build by glynn.
hannigan, we need a verification noted here for Unix please.

------- Additional Comments From hannigan  01/27/99 20:18 -------

edwyn, pls verify for unix asap

------- Additional Comments From dprice  02/09/99 03:07 -------

Apparently the fix was a little too restrictive.  I'll be adding a hidden pref
to disable it.


------- Additional Comments From morse  02/09/99 07:55 -------

*** Bug 342279 has been marked as a duplicate of this bug. ***

------- Additional Comments From morse  02/09/99 08:04 -------

Would somebody please be more specific here.  In bug 342279 jpm said that the
problem is because "The array is indexed at offset [-1]".  I have no idea what
array you are talking about.

Second jpm also said in that report (and dprice echoed that above) that the
problem could be fixed by introducing a "new hidden pref".  I don't understand.
If this pref disables the fix and is hidden, then isn't that the same as
removing the fix?  What have we gained?  If we remove or disable the fix, aren't
we back to the original problem?  I'm confused and must be missing something
here.

------- Additional Comments From dprice  02/11/99 18:05 -------

I have checked in the code for the new preference.  It is
network.cookie.strictDomains

Index: mkaccess.c
===================================================================
RCS file: /m/src/ns/lib/libnet/Attic/mkaccess.c,v
retrieving revision 1.99.8.1.18.11
diff -r1.99.8.1.18.11 mkaccess.c
57a58
> static const char *pref_strictCookieDomains = "network.cookie.strictDomains";
1301c1302,1303
<
---
>     Bool   pref_scd = FALSE;
>
1451,1453c1453,1455
<  * #338501: Fix for domain cookies not starting with a dot.
<  * These cookies are invalid according to the spec, but they're out there...
<  * Example: mail.yahoo.com.
---
>  *  Although this is the right thing to do(tm), it breaks too many sites.
>  *  So only do it if the restrictCookieDomains pref is TRUE.
>  *
1455,1471c1457,1474
<                       if (domain_from_header[0] != '.') {
<                               domain_length++;
<                       }
<                       cur_host[cur_host_length-domain_length] = '\0';
<                       dot = XP_STRCHR(cur_host, '.');
<                       cur_host[cur_host_length-domain_length] = '.';
<                       if (domain_from_header[0] != '.') {
<                               domain_length--;
<                       }
<                       if (dot) {
<                               TRACEMSG(("host minus domain failed no-dot test.
"
<                                 " Domain: %s, Host: %s", domain_from_header, c
ur_host));
<                               FREEIF(domain_from_header);
<                               FREEIF(cur_path);
<                               FREEIF(cur_host);
<                               return;
<                       }
---
>             if ( PREF_GetBoolPref(pref_strictCookieDomains, &pref_scd) < 0 ) {

>                pref_scd = FALSE;
>             }
>
>             if ( pref_scd == TRUE ) {
>                          cur_host[cur_host_length-domain_length] = '\0';
>                          dot = XP_STRCHR(cur_host, '.');
>                          cur_host[cur_host_length-domain_length] = '.';
>                          if (dot) {
>                                 TRACEMSG(("host minus domain failed no-dot tes
t."
>                                    " Domain: %s, Host: %s", domain_from_header
, cur_host));
>                                 FREEIF(domain_from_header);
>                                 FREEIF(cur_path);
>                                 FREEIF(cur_host);
>                               return;
>                          }
>
>             }


------- Additional Comments From abenes  02/16/99 12:10 -------

here are the results for Win32... there is one case where
preference.cookie.strictDomains is not defined where behavior is questionable.
until this issue is clarified i will leave this bug marked resolved fix.

with the hidden preference.cookie.strictDomains set to TRUE
1.mail.yahoo.com - does not function as expected since this site does not follow
the Cookies Specification.
2.www.airnz.co.nz - this site works as expected due to fix in cookies code.

with the hidden preference.cookie.strictDomains set to FALSE
1.mail.yahoo.com - functions as 4.5 as expected
2.www.airnz.co.nz - site functionality fixed does not conflict with co.nz
cookie. due to a specific site fix.

without the hidden preference.cookie.strictDomains in prefs.js (i'd expect this
to function similarly to preference.cookie.strictDomains set to FALSE)

1.mail.yahoo.com - functions as 4.5 as expected
2.www.airnz.co.nz - this site does not work. the cookie for this site is not
saved or set.


------- Additional Comments From abenes  02/16/99 16:42 -------

marking bug as fixed. issue with hidden preference missing was due to bad form
data input sent to server not with the code checked in. with correct input data
behavior is similar to user_pref(network.cookie.strictDomains,false).

------- Additional Comments From anthonyd  02/19/99 15:11 -------

verified as of 2-19-99.
*** Bug 6170 has been marked as a duplicate of this bug. ***
See also bug report 9422 which was just filed.  Now someone is complaining again
about the .co.nz problem.  Of course we can't fix that without breaking yahoo
mail (as laboriously described in this bug report), so I had to close out 9422
as "WON'T FIX".
Status: RESOLVED → VERIFIED
verified mail.yahoo.com now working correctly with 7/10 builds
The issue of the three-dot cookie problem was raised again in the press and I 
had to research it to see why it was a problem.  Since this bug report has all 
the details of the other known cookie problems (yahoo mail problem, .co.nz 
problem), I figured it would be a good archive to add what I learned about the 
three-dot problem here.  This way all the problems are described in one place.  
A description of the three-dot problem can be found at 
http://cookiecentral.com/bug/index.shtml

-------------------------

Here's a description of the three-dot problem:

Site A makes an http request to set a cookie.  That request specifies
(1) that it is a domain cookie and (2) that the value of the domain is
"...".  Now of course that's a garbage domain but the browser doesn't
catch that.  It dutifully sets the cookie having the specified domain
name.

Site B has a url of http://xxx.../yyy where xxx is a valid hostname
(such as people.netscape.com) and yyy is a valid path on that host (such
as morse).  Of course the correct url should be http://xxx/yyy but for
some reason the trailing dots don't seem to bother anyone.  Either the
browser is stripping them off before it makes is DNS lookup, or the DNS
is ignoring trailing dots.

So what happens when the browser sends an http request for site B?  Well
it needs to attach any cookies for B to that request.  So it looks
through its cookie list to see if it has any host (i.e., non-domain)
cookies for host xxx... and it doesn't find any.  It also looks through
the cookie list for any domain cookies that match the domain that site B
is in.  Well, what domains are B in?  Assuming that the url was
http://people.netscape.com.../morse, then it will be in the domain
".netscape.com...", ".com..." as well as  "...".  Note that the third
domain is really nonesense -- it's an artificact of that fact that we
look for all strings starting with dot in the host name and saying that
those are the possible domains.  Now, lo-and-behold, the domain name of
"..." which site B appears to be in matches the domain name of the
cookie that site A set.  So we release site A's cookie to site B.

What just happened?  First a site set a cookie with a bogus domain name
("...") and we allowed it.  Second, another site had a bogus host name
(ending in "...") and we allowed that too.  These two combinations
together caused us to release site A's cookie to site B.  Note that this
is not like the .co.nz problem in which a malicious site was releasing
its cookies to all legitimate sites in the .co.nz domain.  Rather in
this scenerio both site A and site B have to be cooperating by both
doing unorthodox things.

Just like the .co.nz problem, this is not a security or privacy attack. 
In both cases the site is giving out its cookies and not receiving
cookies from an unsuspecting site.

My guess is that we should be able to have the browser nip this one
right in the bud.  First off, I need to know if trailing dots are the
only things that can slip through the DNS lookup and still get to the
intented site?  If so, we need merely to check for any trailing dots in
the domain name when setting a cookie.  Alternately we could check for
any trailing dots in the hostname before releasing the cookie.  Or
probably we can check for both.

But of course I made a similar statement in the past saying we could
solve the .co.nz problem, and by doing so we later learned that the
solution broke yahoo.mail.  So I'm not recommending that we jump right
in and make this fix.  Just that we should know that there is a possible
fix in case this becomes a big problem in the press.

*** Bug 56545 has been marked as a duplicate of this bug. ***
*** Bug 90500 has been marked as a duplicate of this bug. ***
*** Bug 115175 has been marked as a duplicate of this bug. ***
*** Bug 131509 has been marked as a duplicate of this bug. ***
*** Bug 163222 has been marked as a duplicate of this bug. ***
You need to log in before you can comment on or make changes to this bug.