270447 - HTTP Digest Authentication with MD5-sess algorithm is recalculated with new cnonce for every request

Status: NEW

(Core :: Networking, defect, P5)

Description

[Scott Moxham]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

When using HTTP Digest Authentication (RFC 2617) and algorithm is set to
MD5-sess,  Mozilla recalculates A1 for every request using a different cnonce
each time. According to the RFC "If the "algorithm" directive's value is
"MD5-sess", then A1 is calculated only once - on the first request by the client
following receipt of a WWW-Authenticate challenge from the server. It uses the
server nonce from that challenge, and the first client nonce value to construct
A1". This means that I can't use this algorithm on my server as MS IE6
calculates A1 correctly ONCE and then changes the cnonce on every request
meaning that I can't calculate it every time like mozilla does without it
failing on IE.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.

Comment 1

[Jo Hermans]

I've implemented Digest-support in a SIP/H323 SoftSwitch for my day-time job (a
major telecommunications company), and we've seen this before. Several
SIP-phones have this bug.

But RFC2617 never tells you that you *have* to use the same cnonce value every
time. It's true that if you use the same one, the server has to calculate H(A1)
only once, which saves some cpu-time. But if the server detects a different
cnonce, it has to recalculate the hash anyway. Or, if the server never stores
the hash, or has no access to it (load-balancing servers !), it has to calculate
it again anyway. Note that the main purpose of cnonce is that the client can
also verify the authentication of the server (in Authentication-Info).

But ok, if IE6 stores H(A1) for the session, and sends a different cnonce for
the next requests (, 3, ...), we might have to follow. I'm not using that
technique myself (b/c different request are handled by different servers, even
in 1 session), I'm using recalculating H(A1) whenever cnonce changes
(nonce+cnonce indicates the session).

Reporter, are you sure MS IE6 really has this behaviour ? I'm not able to verify
it myself (not running Windooze). Does this creates problems with websites ?
MD5-sess is not used much.

Comment 2

[Scott Moxham]

My understanding of the RFC 2617 spec is that H(A1) is only calculated once
using the original cnonce but this is then hashed together with several other
things for each request including the new cnonce and nonce-count. The point in
this is that H(A1) can be calculated by a 3rd party server and stored without
needing to request this value for every request.

It is possible that I've misunderstood the RFC (it wouldn't be the first time)
but I don't think I have. I have implemented it on a web server in the way I
have described and it works perfectly on MS IE and Apple's Safari (although
Safari seems to reuse the same cnonce all the time).

Comment 3

[Jo Hermans]

You haven't misunderstood RFC2617 section 3.2.2.2 : H(A1) with the cnonce can
really be used as a sessionkey. But this requires that both the client and the
server store this key. Note that section 3.3 (3th para) doesn't tell you that
the client has to store the cnonce value too (that's a mistake).

RFC2617 isn't very clear that the cnonce used in H(A1) is the one from the first
request, while the request-digest in the next requests will use the 'exported'
value (proving that you're really the same one that has send the first request).

The problem is that this doesn't work very well for load-balancing servers,
which might not be in possesion of the H(A1). But that's not a problem for the
client (in this case, MD5-sess should not be offered by th server).

Comment 4

[Jo Hermans]

(changing Hardware/OS to All/All)

I think the correct location to fix this is at
nsHttpDigestAuth::GenerateCredentials
<http://lxr.mozilla.org/aviarybranch/source/netwerk/protocol/http/src/nsHttpDigestAuth.cpp#314>
: we should remember the H(A1) in the sessionstate for the first request, and
reuse it in the next requests (without changing cnonce). So we need to move the
allocation of the sessionstate to after H(A1) is computed (but keep incrementing
the noncecount above).

AFAIK, Apache doesn't yet implement MD5-sess. Reporter, what are you using this
to etst against ? Neon ?

Comment 5

[Scott Moxham]

I am actually using a custom web server that my company has developed in house
for use on our web-enabled databases. Unfortunately these databases aren't
publically available for testing.

Comment 6

[Ray Morris]

Having implemented a server which supports Digest Authentication
and works with Mozilla and also other browsers, I think I can help
clear up some obvious confusion.  Our server works with Mozilla
when MD5 is specified.  When using MD5-sess, Mozilla does NOT
work, whereas other browsers do work.  The whole point of MD-sess
is for the server to be able to calculate the correct response
once per session and I think a careful reading of the protocol
makes that rather clear.  To address specific comments others 
have made:

> that H(A1) is only calculated once using the original cnonce 
> but this is then hashed together with several other
> things for each request including the new cnonce and nonce-count.

The nonce-count is never hashed with anything at all, unless 
we've totally missed some part of the RFC.  If something in
RFC says the request should include a hash of nonce-count 
with something else please point out that section.
Same thing with the cnonce. 

> But RFC2617 never tells you that you *have* to use the 
> same cnonce value every time.

The cnonce is defined as part of the definition of the 
authorization header - the header sent by the browser.
The relevant part of the definition of what the authorization 
header is says:
-------
   If the "algorithm" directive's value is "MD5-sess", then A1 is
   calculated only once - on the first request by the client following
   receipt of a WWW-Authenticate challenge from the server.  It uses the
   server nonce from that challenge, and the first client nonce value to
   construct A1 as follows:
--------
In case it's not clear enough that A1 MUST be calculated that way when
the RFC says "It uses... the first client nonce value to construct A1",
it makes it even more clear by saying "A1 is calculated only once - 
on the first request by the client following receipt of a WWW-Authenticate
challenge from the server".  In neither sentence does it say that the
user agent MAY use the same cnonce for the session, it says twice that the 
user agent DOES use the same cnonce.  I don't see how it could be more
clear that A1 is defined as a hash of the first cnonce value.
If the cnonce value were difference each time, you'd have MD5, which 
is clearly specified as a different option.  There would be no need
for the RFC to even mention MD5-sess if it were essentially exactly
the same as MD5 as some seem to think.

> this requires that both the client and the server store this key.
For most practical uses, the server would probably remember the key.
To be very clear and precise, though, the protocol requires only 
that the client remember the cnonce, as it is supposed to pass that
cnonce back to the server as part of ever authorization header.

IE6 does exhibit the very strange behavior of using two entirely
different cnonce values in the same request.  As it turns out, 
this doesn't have much practical consequence though precisely because
the whole reason most people use MD5-sess is as a session key as
outlined in the RFC and mentioned in comment #3, where the server 
stores the response value as session ID and never has any need to 
inspect the cnonce values passed in later requests (they are not 
part of any hash).  It is my opinion, then, that IEs behavior in 
this respect a) broken and b) inconsequential and therefore 
Mozilla should NOT emulate this behavior of using two different
cnonce values for the same request, one to calculate A1 and a
different one to pass directly in the header.  Rather we should 
recognize the point made in comment #3 here and section 3.2.2.2 
of the RFC that the idea is to have a session key and Mozilla 
use the cnonce value from the first request in all further requests.

Comment 7

[Ray Morris]

Strike the last line in the second paragraph of my comment, that was
an editing error.  This line should not be there:
---
Same thing with the cnonce.
---

Comment 8

[Jo Hermans]

I was speaking from experience when implement HTTP-Digest for VOIP servers. Most
current SIP-phones don't correctly implement MD5-sess (or not at all), even
though they claim they do. That's why my day-time company still doesn't support
MD5-sess in our VOIP SoftSwitch. Neither do our customers. And we're don't even
plan to fix it in the near future (b/c we can't rely on a good fallback to
normal MD5).

One of the most common errors is that every new Authorization uses a new A1
(calculated with a new cnonce, not the old one), basically starting a new
session everytime. Other clients never change the cnonce in every request
(that's not required in RFC2617), but this defeats the purpose of using MD5-sess
at all. And don't get me started about the bad quality of nonces in general - a
well known SIP-phone uses its timestamp as the cnonce.

But those are problems for the servers. For the clients it's much simpler, we
just have to implement the standard correctly.

Comment 9

[Darin Fisher]

-> default owner

Comment 10

[Ingo Struck]

Ray Morris wrote this:

====
In case it's not clear enough that A1 MUST be calculated that way when
the RFC says "It uses... the first client nonce value to construct A1",
it makes it even more clear by saying "A1 is calculated only once - 
on the first request by the client following receipt of a WWW-Authenticate
challenge from the server".  In neither sentence does it say that the
user agent MAY use the same cnonce for the session, it says twice that the 
user agent DOES use the same cnonce.
====
The first part is right. The second ist not. The client MAY use any cnonce
at any time to calculate the response, but NOT to recalculate the session key.

In the subsequent text I will use the following terms:
nonce0 := the first nonce the server sent for a "session"
cnonce0 := the first nonce the client sent for a "session"

The rfc clearly states that the session key is calculated only once.
That means that both server and client need to remember HA1 during the session.
Within algo=MD5 this is simple:
  HA1=md5(user:realm:password)
this could be assumed to be constant during any "session" (because neither of user, realm, and password change), it is not clear why the spec assumes that HA1 has to be recalculated for each request/response.

The rfc now requires that for algo=MD5-sess
  HA1=md5(md5(user:realm:password):nonce0:cnonce0)
and that this is only calculated once, i.e at the start of a session.

But the cnonce used by the client MAY vary from request to request, because it does not affect HA1 (neither within MD5 nor within MD5-sess) in any way.
It only affects the response:

response = md5(HA1:nonce:cnonce:qop:HA2)

In fact the requirement both for server and for client do not differ:
for each session both sides have to know and consistently use the initial
value of HA1, both for the MD5 and MD5-sess algorithm. It is ONLY
a performance penalty for a server to deal with b0rken clients who recalculate
the session key with every new cnonce issued (the response needs to be calculated twice and the session key needs to be recalculated as well).

Bottom line:
To recalculate the session key with every new cnonce is
- a clear violation of the spec
- a performance penalty for the client (because it costs nothing to store a session key but two md5 operations to recalculate it), but this penalty should
not affect a b0rken client much
- a severe performance penalty for the server because it has to recalculate each response for b0rken clients (i.e. three surplus md5 operations per request)

Comment 11

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 12

[Adam_5Wu]

14 year later... I hit this bug again when trying to implement a proper digest authentication on an MCU HTTP server.
Internet Explorer still uses a non-changing cnonce for each auth session -- you could call them lame, but at least they aren't broken. :P

Comment 13

[Mathew Hodson]

The code around this area was updated in bug 1705659, bug 472823, and bug 669675. Is this issue still reproducible?

Comment 14

[Vit Hampl]

(In reply to Mathew Hodson from comment #13)

The code around this area was updated in bug 1705659, bug 472823, and bug 669675. Is this issue still reproducible?

I haven't tried it, but from a brief look into current code of nsHttpDigestAuth, there is no reason to think of it being resolved. The solution would contain storing (and reusing) result of CalculateHA1 into sessionState in a similar manner as nonce_count, but there is currently no such code.

Comment 15

[Evgeny Grin [:karlson2k]]

18 years later...
The bug is still actual and needs to be fixed.
Here you can find recorded communication between Firefox and IIS: https://github.com/curl/curl/pull/9074#issuecomment-1177963896
Obviously the Firefox's implementation for MD5-sess does not work correctly and requires fixing.

Currently Firefox (like Chrome) succeed with the first request with fresh server nonce, but always fail with the second request in the same session when using IIS with Digest Auth.

Currently cnonce is always regenerated for each request (which is correct), but A1 (and H(A1)) is regenerated as well with the new cnonce value (which is wrong for MD5-sess).
https://github.com/mozilla/gecko-dev/blob/54571ce8b0e25ad9be9feb6587a775454616c8ac/netwerk/protocol/http/nsHttpDigestAuth.cpp#L323-L336

The calculated H(A1) value is used only for one-time calculations for the response and not stored anywhere.
https://github.com/mozilla/gecko-dev/blob/54571ce8b0e25ad9be9feb6587a775454616c8ac/netwerk/protocol/http/nsHttpDigestAuth.cpp#L492-L534
As it was mentioned in previous comment, the code must remember H(A1) value in the same way, as it is implemented for nonce-count.

The similar bug for Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1344488