281851 - (httpauthorder) CVE-2005-2395 Wrong scheme used when server offers both Basic and Digest auth [rfc2617 obsoletes rfc2068]

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P5)

Description

[dori eldar]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

When connecting to a web server that offers both HTTP Basic authentication and
HTTP Digest authentication, Firefox selects to perform Basic authentication
rather then the more secure Digest authentication scheme. This behavior is in
contrast to RFC 2617 requirement, which describes HTTP authentication:
------------------------------------------------------------
4.6 Weakness Created by Multiple Authentication Schemes

   An HTTP/1.1 server may return multiple challenges with a 401
   (Authenticate) response, and each challenge may use a different
   auth-scheme. A user agent MUST choose to use the strongest auth-
   scheme it understands and request credentials from the user based
   upon that challenge
------------------------------------------------------------
 

Reproducible: Always

Steps to Reproduce:
1. Setup a Web Server to support both Basic and Digest authentication mode in
parallel i.e. IIS server in a windows domain
2. Use firefox to connect to the web-server.
3. send user/pass credentials, when Firefox prompts the for credentials.
4. Use a network sniffer to to track traffic.
5. analyze sniffer dump, note that Firefox sends the user's credential to the
wb-server, specifying HTTP Basic scheme. 

Actual Results:  
from network sniffer:

// first un-authenitcated request send from client
Frame 4 (452 bytes on wire, 452 bytes captured)    
Ethernet II, Src: 00:d0:59:cd:24:38, Dst: 00:0c:f1:33:e2:ff
Internet Protocol, Src Addr: 192.168.2.2 (192.168.2.2), Dst Addr: 192.168.2.94
(192.168.2.94)
Transmission Control Protocol, Src Port: 1646 (1646), Dst Port: http (80), Seq:
1, Ack: 1, Len: 398
Hypertext Transfer Protocol
    GET / HTTP/1.0\r\n
        Request Method: GET
    Host: 192.168.2.94\r\n
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5)
Gecko/20041107 Firefox/1.0\r\n
    Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    \r\n

...

// server sends un-authorized response indicating support for both basic and digest
Frame 5 (1514 bytes on wire, 1514 bytes captured)
Ethernet II, Src: 00:0c:f1:33:e2:ff, Dst: 00:d0:59:cd:24:38
Internet Protocol, Src Addr: 192.168.2.94 (192.168.2.94), Dst Addr: 192.168.2.2
(192.168.2.2)
Transmission Control Protocol, Src Port: http (80), Dst Port: 1646 (1646), Seq:
1, Ack: 399, Len: 1460
Hypertext Transfer Protocol
    HTTP/1.1 401 Access Denied\r\n
        Response Code: 401
    Server: Microsoft-IIS/5.1\r\n
    Date: Thu, 10 Feb 2005 21:33:02 GMT\r\n
    WWW-Authenticate: Basic realm="corp.intel.com"\r\n
    Content-Length: 4431\r\n
    WWW-Authenticate: Digest qop="auth", realm="corp.intel.com",
nonce="964f5a7cd20d6b8f10246120000062720c8897eaebb6bbccd1a7a2999dd0"\r\n
    Content-Type: text/html\r\n
    \r\n

...

//client send an authenticated response specifying HTTP Basic scheme
Frame 16 (487 bytes on wire, 487 bytes captured)
Ethernet II, Src: 00:d0:59:cd:24:38, Dst: 00:0c:f1:33:e2:ff
Internet Protocol, Src Addr: 192.168.2.2 (192.168.2.2), Dst Addr: 192.168.2.94
(192.168.2.94)
Transmission Control Protocol, Src Port: 1647 (1647), Dst Port: http (80), Seq:
1, Ack: 1, Len: 433
Hypertext Transfer Protocol
    GET / HTTP/1.0\r\n
        Request Method: GET
    Host: 192.168.2.94\r\n
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5)
Gecko/20041107 Firefox/1.0\r\n
    Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    Authorization: Basic ZG9yaTpkb3Jp\r\n
        Credentials: dori:dori
    \r\n


Expected Results:  
Firefox should select to use HTTP Digest authentication scheme, when presented
with both Digest and Basic authentication scheme, as required in RFC 2617

Comment 1

[Daniel Veditz [:dveditz] out until Nov 10]

->Core:Networking

Comment 2

[Boris Zbarsky [:bzbarsky]]

So the problem seems to be that nsHttpChannel::GetCredentials will go with the
first challenge it sees...

Comment 3

[Chris Nebergall ([e-mail])]

(In reply to comment #2)
> So the problem seems to be that nsHttpChannel::GetCredentials will go with the
> first challenge it sees...

Yes.  Mozilla follows the older RFC 2068 (HTTP 1.1 RFC)

   "An HTTP/1.1 server may return multiple challenges with a 401
   (Authenticate) response, and each challenge may use a different
   scheme.  The order of the challenges returned to the user agent is in
   the order that the server would prefer they be chosen. The server
   should order its challenges with the "most secure" authentication
   scheme first. A user agent should choose as the challenge to be made
   to the user the first one that the user agent understands."

Does IIS always return Digest AFTER Basic authentication in the WWW-Authenticate
headers or did that take some hacking of the Metabase?  What browers are smart
enough to handle this correctly?

Valid RFE, probably not security sensitive.

Comment 4

[dori eldar]

(In reply to comment #3)
> Yes.  Mozilla follows the older RFC 2068 (HTTP 1.1 RFC)

IMHO it should follow the more secure option manadated in RFC 2617, which 
obsoletes RFC 2068
 
> Does IIS always return Digest AFTER Basic authentication in the WWW-
Authenticate headers or did that take some hacking of the Metabase?

This is the default behavior of IIS 5.0 I was told IIS 6.0 places digest first 
by I didn't verify.

>What browers are smart enough to handle this correctly?

Seen the same problem with IE 6.

Comment 5

[Boris Zbarsky [:bzbarsky]]

Yeah, this doesn't seem to be security-sensitive.

Comment 6

[Daniel Veditz [:dveditz] out until Nov 10]

Forgive me if I tightened the summary too much. Default buglists only show the
visible portion of the textbox up there.

I agree that we should the confidential flag.

Comment 7

[Vidar Haarr (not reading bugmail)]

Attachment: version 0.1

I made this patch in an attempt to learn a bit more about array and string
usage in the Mozilla API, and I hope I can get some help to evolve the patch
into something that resembles proper usage ... If you'd like to do that via.
e-mail or IRC, I'd be happy to do so instead.

Anyway, some things I'm not happy about:
* The "static" priority array should probably live in some longer-lived object;
maybe the HttpHandler?
* The "static" priority array could be a string list pref, like
'networking.http.auth.schemePriorityList' with a comma-seperated, prioritized
from 0->X, list. Or something ... The idea is that if someone (say, Netscape
:-) makes 2-3 new HTTP authentication schemes and they want one of them to have
a priority between basic and digest, they can easily just modify this pref. As
it is now, all unknown schemes get a random priority higher than the known
ones.
* challengesList and schemeList is really just a map of scheme <->
fullChallengeString. I have no idea if there is a better way of doing this ...
* The strings here are just insane. Why do I have to go through all that
trouble (nsDependentCString(nsCString->get())) when I want to check
nsCStringArray.IndexOf? According to common sense I should be able to just pass
in a nsCString to IndexOf, since it is a CStringArray ... Of course, that
'common sense' is there to fill the void that is my knowledge of the Mozilla
String API and C strings in general :-)

Comment 8

[Christian :Biesinger (don't email me, ping me on IRC)]

>Why do I have to go through all that
>trouble (nsDependentCString(nsCString->get())) when I want to check
>nsCStringArray.IndexOf? 

What was wrong with *your_cstring?

Comment 9

[Daniel Veditz [:dveditz] out until Nov 10]

I don't see any comments from Darin here, but out of band I think he told me we
still follow the old spec for server compatibility. Possibly enough servers we
care about have evolved since that time that we can fix it.

The following from 3APA3A was posted on Bugtraq today
http://securityfocus.com/archive/1/410580/30/0/threaded

Comment 10

[Juha-Matti Laurio]

A Normal severity still after that Bugtraq posting or is there anything new
information?

Comment 11

[Vidar Haarr (not reading bugmail)]

Attachment: version 0.2

* Moves the default priorities into a pref and reads it in nsHttpHandler.

Comment 12

[Mike Schroepfer]

Darin - can you comment on risk level for this patch given upcomming b2 release?

Comment 13

[Darin Fisher]

I'm not sure what to think about this bug.  It's possible that Digest
authentication can be more secure than NTLM, or vice versa, depending on what
mode of NTLM the server supports.  In that case, what should we do?  I really
prefer giving servers the choice here since it gives "smart" server admins the
flexibility to do what's best for users.  However, I understand that it would be
wise for Firefox to prefer Digest auth to Basic auth even if the server lists
Basic auth first.

At first glance, the patch looks pretty good.  I'll probably have more comments
once I look it over more closely.

Given that IE also implements the older RFC, it makes it a bit more risky to
consider changing our behavior at this point in time.  What about Opera and
Safari?  What do they do?

Comment 14

[Vidar Haarr (not reading bugmail)]

(In reply to comment #13)
> Given that IE also implements the older RFC, it makes it a bit more risky to
> consider changing our behavior at this point in time.  What about Opera and
> Safari?  What do they do?

According to comment 4, IE has the same problem as Mozilla here. But, according
to <http://securityfocus.com/archive/1/410580/30/0/threaded> (ref. comment 9),
"Internet Explorer offers [the] strongest authentication."

Comment 15

[Darin Fisher]

> According to comment 4, IE has the same problem as Mozilla here. But, according
> to <http://securityfocus.com/archive/1/410580/30/0/threaded> (ref. comment 9),
> "Internet Explorer offers [the] strongest authentication."

Can we get a definitive answer to this question then?

Comment 16

[Vidar Haarr (not reading bugmail)]

The I.E. I have access to here is version 6.0.2900.2180.xpsp_sp2_gdr.050301-1519
(IE6, WinXP SP2), and it chooses basic auth on
<http://www.security.nnov.ru/files/atest/all.asp> (which is where, according to
<http://securityfocus.com/archive/1/410580/30/0/threaded>, I.E. "offers
strongest authentication".)

So it would seem to me that
<http://securityfocus.com/archive/1/410580/30/0/threaded> is wrong and comment 4
is right.

Comment 17

[Darin Fisher]

Vidar: Thanks for testing that.  I'm not sure what to do now.  Hmm...

Comment 18

[Christopher Nebergall]

(In reply to comment #13)
> I'm not sure what to think about this bug.  It's possible that Digest
> authentication can be more secure than NTLM, or vice versa, depending on what
> mode of NTLM the server supports.  In that case, what should we do?  I really
> prefer giving servers the choice here since it gives "smart" server admins the
> flexibility to do what's best for users.

IIS and other servers will probably always offer NTLM before Digest because of
the autosend abilities of IE and Mozilla in Windows.  So whether we let mozilla
decide or the server decide the result will probably be the same.

Comment 19

[Vidar Haarr (not reading bugmail)]

(In reply to comment #13)
> Given that IE also implements the older RFC, it makes it a bit more risky to
> consider changing our behavior at this point in time.  What about Opera and
> Safari?  What do they do?

Opera 8.5 build 7700 on Windows selects Basic, just like IE and Mozilla
currently does without attachment 197260 [details] [diff] [review].

It might be worth noting that neither IE nor Opera present other authentication
methods if you click Cancel in the Basic dialog. Mozilla presents one dialog for
every auth scheme in sequence where applicable.

Comment 20

[Vidar Haarr (not reading bugmail)]

(In reply to comment #13)
> Given that IE also implements the older RFC, it makes it a bit more risky to
> consider changing our behavior at this point in time.  What about Opera and
> Safari?  What do they do?

I don't have access to Safari, but tried Konqueror, and it presented me with a
user/password dialog that said "Site: NTLM at www.security.nnov.ru", so it would
seem that it uses the (presumed) strongest method by default.

gandalf tested it too, and it said "[e-mail] at
www.security.nnov.ru" there, so I presume NTLM is for some reason disabled in
his build, but it picked digest over basic.

Clicking Cancel did not provoke an auth dialog for a "lesser" scheme (like
Mozilla does).

Comment 21

[Darin Fisher]

Since Firefox cannot know if NTLM is better than Digest or vice versa, I think Firefox should simply prefer anything over Basic authentication.  When deciding between other authentication schemes, Firefox should just select them in the order specified by the server.  (Negotiate does not necessarily mean Kerberos!)

Comment 22

[Vidar Haarr (not reading bugmail)]

(In reply to comment #21)
> Since Firefox cannot know if NTLM is better than Digest or vice versa, I think
> Firefox should simply prefer anything over Basic authentication.  When deciding
> between other authentication schemes, Firefox should just select them in the
> order specified by the server.  (Negotiate does not necessarily mean Kerberos!)

I agree. This means, AFAICS, that attachment 197260 [details] [diff] [review] can be used as attached except changing the pref network.http.auth-scheme-priority-list to read only "basic" and only adding schemePriorityList.AppendCString(NS_LITERAL_CSTRING("basic")).

Comment 23

[Darin Fisher]

Attachment: v1 patch

This is an alternate patch that only implements re-ording "basic" auth challenges.  It attempts to do the right thing if given multiple basic auth challenges.  I haven't fully tested this patch, so any help testing it would be greatly appreciated.  The patch includes some cleanup and simplification of some related code.

Comment 24

[Vidar Haarr (not reading bugmail)]

Comment on attachment 204614 [details] [diff] [review]
v1 patch

My testing shows that the patch presents me with other schemes before "basic" in the order specified by the server.

Would it be an idea to keep the pref as proposed in comment 22?
It would make it easier to make policy changes at a later date, and also for embedders to make their own decision.

I'm not saying I see an absolute need for it, but I don't see any drawbacks either.

Comment 25

[Wyllys Ingersoll]

(In reply to comment #21)
> Since Firefox cannot know if NTLM is better than Digest or vice versa, I think
> Firefox should simply prefer anything over Basic authentication.  When deciding
> between other authentication schemes, Firefox should just select them in the
> order specified by the server.  (Negotiate does not necessarily mean 
Kerberos!)
> 

While "Negotiate" does not necessarily absolutely mean "Kerberos", in almost 
all practical cases in use today, it almost always means something stronger than 
"Basic".   Microsoft uses Negotiate to choose between NTLM and Kerberos.
I think it *is* safe to put Basic behind Negotiate when sorting by security.

We ran into a problem just recently where an Apache server was sending 
"Negotiate" followed by "Basic" and the browser always prompts for Basic, 
ignoring the "Negotiate" extension which was the opposite of what was intended -
 which was to use Negotiate if possible, else fall back to Basic.

For the rare case where someone might prefer Basic over other choices, use
the preference option, but I think the default should be that Basic is
always used as a last resort.

Comment 26

[Jason Duell]

Does anyone know if either of the two proposed patches for this bug ever landed?

I'm adding this to my list of likely network bugs to work on.  No guarantee when I'll actually get to it.

Comment 27

[Jo Hermans]

(In reply to comment #26)
> Does anyone know if either of the two proposed patches for this bug ever
> landed?
> 
> I'm adding this to my list of likely network bugs to work on.  No guarantee
> when I'll actually get to it.

No, no patches were added to the code. You can check at <http://mxr.mozilla.org/mozilla/source/netwerk/protocol/http/src/nsHttpChannel.cpp>

Comment 28

[Greg Inozemtsev]

This bug is still present in FF 3.5.1. IE 7 and 8 follow the RFC.

Comment 30

[cmitasch]

Any news on this bug, it's a serious security related issue and opened for quite a long time!

Comment 31

[Sam Varshavchik]

For unrelated reasons I did some recent testing on how Firefox implements several aspects of RFC 2617, including the one reported by this bug (which I wasn't aware of when I did my testing). I found a number of issues, and while searching Bugzilla found this entry. In addition to this bug, I also found a few other problems.

RFC 2616 section 14.47 specifies the WWW-Authenticate header as follows:

WWW-Authenticate  = "WWW-Authenticate" ":" 1#challenge

According to section 2.1, this syntax specifies that WWW-Authenticate contains one or more comma-separated "challenge" elements, and "challenge" gets defined in RFC 2617.

RFC 2617 section 1.2 defines the "challenge" element as follows:

challenge   = auth-scheme 1*SP 1#auth-param

In other words, the "challenge" elements consists of "auth-scheme" (which gets defined as word) one or more spaces, then one or more parameters (defined, roughly, as name=value tuples, elsewhere).

As I interpret this, RFC 2616 and RFC 2617 permit a single WWW-Authenticate header to specify multiple challenges, and the following header seems legal to me, according to RFC 2616 and RFC 2617:

WWW-Authenticate: Basic realm="Test Realm", Digest realm="Test Realm", algorithm=MD5, domain="/", nonce="1350785623", qop="auth"

The fact that this is a valid header is unambiguously confirmed by section 1.2 of RFC 2617:

   Note: User agents will need to take special care in parsing the WWW-
   Authenticate or Proxy-Authenticate header field value if it contains
   more than one challenge, or if more than one WWW-Authenticate header
   field is provided, since the contents of a challenge may itself
   contain a comma-separated list of authentication parameters.

In my testing, I've set up a test server and pointed Firefox 16.0.1 to it. When Firefox received a response with the above header, it prompted for authentication, but responded with a Basic authentication scheme.

If I reverse the order of the schemes, and have the custom server respond with the following header:

WWW-Authenticate: Digest realm="Test Realm", algorithm=MD5, domain="/", nonce="1350785798", qop="auth", Basic realm="Test Realm"

In this case, Firefox fails to prompt for authentication entirely.

I also tested the Elinks 0.12 browser. In both of the above cases, elinks prompted for authentication and responded with the MD5 Digest authorization scheme.

Additional testing:

RFC 2617 specifies only MD5 as the hash mechanism for the Digest authorization scheme, however RFC2617 does not prohibit other schemes from being used. Section 3.2.1 of RFC 2617 states:

algorithm
     A string indicating a pair of algorithms used to produce the digest
     and a checksum. If this is not present it is assumed to be "MD5".
     If the algorithm is not understood, the challenge should be ignored
     (and a different one used, if there is more than one).

I am using gcrypt, and it was fairly trivial for me to implement Digest using SHA1, or any other hash method implemented in gcrypt, in place of MD5. So I tried the following headers:

WWW-Authenticate: Basic realm="Test Realm", Digest realm="Test Realm", algorithm=MD5, domain="/", nonce="1350787263", qop="auth", Digest realm="Test Realm", algorithm=SHA1, domain="/", nonce="1350787263", qop="auth"

... and several other permutations. The results were fairly consistent: Firefox uses Basic authorization scheme if it's the first one that appears in the WWW-Authenticate header. If something else appears, even Digest/MD5, which Firefox otherwise implements, Firefox fails to authenticate. Elinks always used Digest/MD5, no matter what its position was, in the header, ignoring the SHA1 challenge.

When I change the server to use a separate WWW-Authenticate header for each challenge, Firefox actually ignores the header with the SHA1 scheme, if it's the first one, and picks either Digest/MD5 or Basic, whichever one sees first, consistent with this bug report.

WWW-Authenticate: Digest realm="Test Realm", algorithm=SHA1, domain="/", nonce="1350787799", qop="auth"
WWW-Authenticate: Digest realm="Test Realm", algorithm=MD5, domain="/", nonce="1350787799", qop="auth"
WWW-Authenticate: Basic realm="Test Realm"

In this case, Firefox responds with a Digest authorization scheme, and ignores the SHA1 header. When the order is SHA1/Basic/MD5, Firefox uses Basic.

In this respect, Firefox's behavior is the same as Elinks'.

Comment 32

[Huzaifa Sidhpurwala]

Any idea if this is going to be fixed in Firefox?, or we just want to keep the original behaviour?

Comment 33

[Daniel Stenberg [:bagder]]

Sam's point about problems with multiple auths in the same header line is bug 669675.

Comment 34

[David Woodhouse]

I am still seeing Firefox choosing to use Basic auth, when it's offered first before the preferable NTLM and Negotiate options. Any prospect of this ever being fixed?

Comment 35

[abderrahimpro]

I have the same problem. 

In my case, Users must be authenticated to Squid, to have access to internet.

IE and Chrome choose negotiate over basic, firefox not. (In Squid logs, i found that firefox tries Basic and negotiate at the same time).  

There is any solution to force firefox to choose negotiate over basic.

This is my squid config :

### basic authentication
auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid3/users
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours

### negotiate kerberos and ntlm authentication
auth_param negotiate program /usr/local/bin/negotiate_wrapper -d --ntlm /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=??????.COM --kerberos /usr/lib/squid$
auth_param negotiate children 10
auth_param negotiate keep_alive off

### pure ntlm authentication
auth_param ntlm program /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=??????.COM
auth_param ntlm children 10
auth_param ntlm keep_alive off

......

Comment 36

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 37

[gstrauss]

Patch submitted for review: https://phabricator.services.mozilla.com/D106241
Related bug: https://bugzilla.mozilla.org/show_bug.cgi?id=472823 SHA 256 Digest Authentication

Comment 38

[Mathew Hodson]

It sounds like this should depend on bug 472823. Is this fixed now that that bug is?

Comment 39

[Valentin Gosu [:valentin] (he/him)]

Yes, this should be fixed now.

Comment 40

[Mathew Hodson]

The code in bug 669675 and bug 472823 landed in Firefox 93, so I think this is fixed now.