44041 - WWW-Authenticate: compatability w/ multiple lines

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect, P3)

Description

[Brian Hunt]

In General: When attempting to connect to this web page using other browsers
(NS, IE), the authentication works fine.

Mozilla: Nothing happens.  The server is IIS/4 with (basic) BASE64
authentication.  This link is only temporary, but I may be able to provide a
replacement when this one goes inside a DMZ (thus you won't be able to connect
to it anyway) this week.  STDERR receives "Error loading URL
http://gyrus.cs.unb.ca/mast_xr" but nothing more.

I didn't trace the packets, but if someone things it would be helpful, it can be
done.  This bug was marked as a blocker since Beta2 shouldn't go out unless Moz
can authenticate with BASE64/IIS. (if that turns out to be an isolation of the
problem; I'm speculating a bit)  There might be more to this than I comprehend -
feel free to fill me in on why this wouldn't work. :)  (although it does work in
NS4.7)

Comment 1

[Mitchell Stoltz (not reading bugmail)]

Reassigning to networking and Gagan.

Comment 2

[rcummins]

HTTP header for http://gyrus.cs.unb.ca/mast_xr/ is:

HEAD /mast_xr HTTP/1.0

HTTP/1.1 401 Access Denied
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="131.202.33.81"
Content-Length: 644
Content-Type: text/html

The "WWW-Authenticate: NTLM" seems to be throwing Mozilla, it doesn't even try
to render the page much less ask for username/password. I suspest "NTLM" is not
a valid value, since multiple valid "WWW-Authenticate"s seem to work. Was able
to duplicate the problem on Win95 build 2000062914, and on another web server at
http://www.burlco.lib.nj.us/moz/realm/ - NC 4.72 handles this "correctly" - or
at least it handles it gracefully.

Comment 3

[Eliot Landrum]

Confirmed on Linux 2000070308.

Tried the URL with a trailing slash and without.

I do not get an error on STDERR:

Document: Done (0.851 secs)
Document http://gyrus.cs.unb.ca/mast_xr/ loaded successfully

Document: Done (1.024 secs)
Document http://gyrus.cs.unb.ca/mast_xr loaded successfully

Yet no dialog or content.

Comment 4

[justin]

I don't believe mozilla considers multiple WWW-Authenticate lines.
It just takes the first, which isn't the proper RFC behavior.
I've recently done some work in the area, so I'll fix this soon.

Comment 5

[justin]

Attachment: This patch causes mozilla to look at multiple WWW-Auth headers and then use the first it can handle

Comment 6

[justin]

This bug was the result of mozilla's incomplete handling of www-auth headers.
It didn't consider the possibility of multiple headers being sent by the server.

I've fixed this (in the above patch), so that it goes through each header, in 
the order received (as per RFC specification), and sees if there's an auth 
service for each type.

This makes both Brian Hunt and rcummins example URLs work fine now (it falls 
through to the Basic auth type).

Also, while I was in the neighborhood, I changed the password prompt message to 
make it work like 4.x did. For example, mozilla was prompting with:

"Enter username for ${WWW-Authenticate header value}"

It now prompts with:

"Enter username for ${realm} at ${host}"

Comment 7

[justin]

Oh, btw, that patch applies in netwerk/protocol/http/src

Comment 8

[Peter "jag" Annema]

I'm seeing this on Win 95, 2000-07-28-08-M18, and from the type of code 
suspecting this is All All, so marking so.

Adding a few keywords. No need to let this rot away. At first glance the patch 
looks okay but someone more appropriate will have to provide the r=.

Comment 9

[Gagan]

nominating and recommending a plus.

Comment 10

[Gagan]

ok.

Comment 11

[justin]

I just noticed this has approval and review keywords.
Does that mean I can land it? If so, who do I list for r=/a=?

Comment 12

[Peter "jag" Annema]

Actually, review and approval keywords mean that they're desired. They should be
taken off the keywords list once done / granted. Also, (the request for)
approval shouldn't be up there until the review has been done. My mistake and
correcting.

Comment 13

[Gagan]

ok justin, r/a=gagan 
go for it! and thanks! :)

Comment 14

[Gagan]

fix checked in.

Comment 15

[Gagan]

*** Bug 48047 has been marked as a duplicate of this bug. ***

Comment 16

[Paul Wyskoczka]

Updating QA Contact to [e-mail]

Comment 17

[John Unruh]

Verified fixed. 

Comment 18

[Yrjänä Rankka]

Choosing the first authentication method you can handle is a step into the right
direction, but still not how it should be. I believe the rfc states that one
should always choose the method providing strongest security that the client
implements. i.e. if we know how to do basic and digest and the response headers
contain both challenges for basic and digest, we should choose to use digest,
regardless of order in which the challenges are presented.

Comment 19

[Steffen Pietsch]

Ups,
here: Win98, Mozilla M18 from 20001010

Following URL request 2 authenticate header.
  1. digest auth
  2. basic auth
http://www.berlinonline.de/wissen/computer/linux_tips/os/.bin/authtest.cgi

After applying the user/pw:  "test"  "test"  (without the quotes)
shows which authentication scheme was used.

!! Mozilla does not take the first (digest) but only the second (basic) !!
(Or ist the digest-patch from june not yet in M18??)
Steffen

Comment 20

[John Unruh]

Steffen and yrjana - please open a new bug based upon your last comments that 
the most secure method should be used. I'm leaving this one as verified fixed, 
since the original bug is fixed - "Unable to enter basic authentication"

Comment 21

[justin]

Steffen: The Digest auth code is not in; that's mostly my fault, as I have not
updated my patch (for Digest auth) for the changes I made to fix this bug
(support for multiple WWW-Auth lines).

Junruh and Yrjänä: The current implementation for handling multiple www-Auth
lines is correct. The RFC says the the client should use methods in the order
listed by the server, not that the client should determine the most secure
method and use that. This makes sense as 1) the server should be capable of
ordering by security and 2) the server may prefer one particular method over
another with similar (or even slightly more) security, such as an NT ISS box
wanting NTLM before Digest auth.

The relevant bit from RFC 2068 (aka HTTP/1.1):

   15.2 Offering a Choice of Authentication Schemes

   An HTTP/1.1 server may return multiple challenges with a 401
   (Authenticate) response, and each challenge may use a different
   scheme.  The order of the challenges returned to the user agent is in
   the order that the server would prefer they be chosen. The server
   should order its challenges with the "most secure" authentication
   scheme first. A user agent should choose as the challenge to be made
   to the user the first one that the user agent understands.

Comment 22

[benc]

->http

Comment 23

[justin]

The issue of choosing an appropriate authentication mechanism came up
in the digest auth bug discussion, and I was about to point out that
the RFC says follow the server's order, not make the client decide.
Well, in fact, it says both. Here are the two relevant sentences from 
RFC2068, both in section 15.2, separated by a mere paragraph:

"A user agent should choose as the challenge to be made
 to the user the first one [in the server's order] that the 
 user agent understands."

and 

"For this reason, the client should always use the strongest 
 scheme that it understands from the choices accepted."

Of course, the second one only applies to man-in-the-middle attacks,
and if there is a man in the middle, he can just remove all but
Basic auth headers, so it doesn't help. Considering that, I believe
it's still best to use the server order, since a site might prefer
some wacky alternative (NTLM) and using something else (Digest) might
hinder the client's abilities in their system.

Comment 24

[An-Cheng Huang]

Actually, RFC 2068 is obsoleted by 2616, which doesn't mention this any more.
Instead, section 15.2 of RFC 2068 is now sections 4.6 & 4.8 of RFC 2617,
which says "MUST choose the strongest" (no conflicting statements).

However, I agree that this doesn't help much with the MITM problem (a better
solution is to disable "insecure" schemes such as Basic).  Also it involves 
problems such as ranking the strength of the authentication schemes (well,
is Digest "stronger" than NTLM?).  And lastly, there isn't anything other 
than Basic at the moment anyway.  So maybe this can wait until Mozilla 
supports multiple authentication schemes.

Comment 25

[Brian Hunt]

I am not entirely sure this has been fixed.  When encountering a web page with
headers:

WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="192.168.1.17"

Mozilla seems to default as broken, never attempting basic auth.  I am not sure
if the problem is related to the fix applied here, but this seemed like a good
place to start.

Note: the web server once used for testing is now offline.

Comment 26

[Brian Hunt]

Apologies for the spam.

I have confirmed that this is a problem with the debian binary for Mozilla, and
a bug report is being sent upstream.  (In my own defense, 3 versions of Mozilla
were tested, cvs, 0.9.7 and 0.9.5, where only 0.9.5 worked; all on debian.)

Remarking as closed.

Comment 27

[John Unruh]

Verified per reporter's comment.

Comment 28

[benc]

John: can you send Tom the test server URL that you verified? Are there public
servers using NTLM? I want to add that to our test suites.

Comment 29

[John Unruh]

I verified this based on the reporter's comment, as stated directly above. If
this is not a dupe of another bug or should be reopened, please do so. Also, I
don't know of any public servers using NTLM.

Comment 30

[benc]

I've got this working internally now, so it is part of our HTTP testcases.

Comment 31

[benc]

*** Bug 48469 has been marked as a duplicate of this bug. ***