228062 - NTLM authentication fails with mod_ntlm, mod_ntlm reports "missing/corrupt NTLM header"

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect)

Description

[Jan Ploski]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Linux 2.6.0-test9 i686) Opera 7.11  [en]
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6b) Gecko/20031208

I set up mod_ntlm v.0.4 today to try it out. MSIE 6 can authenticate just fine
in my setup. When I try accessing the same page with Mozilla 1.6b, mod_ntlm logs
the following errors:

[Wed Dec 10 18:07:28 2003] [error] [client 192.168.0.1] creating new 
ntlm_connection 134977044 28747                          
[Wed Dec 10 18:07:28 2003] [notice] [client 192.168.0.1] got auth_line 
"TlRMTVNTUAABAAAAB4IIAAAAAAAgAAAAAAAAACAAAAA="
[Wed Dec 10 18:07:28 2003] [error] [client 192.168.0.1] ntlm_decode_msg failed: 
type: 1, host: "", user: "", domain: "", error: 3
[Wed Dec 10 18:07:28 2003] [error] [client 192.168.0.1] missing/corrupt NTLM 
header 134977044 28747

mod_ntlm then gives up, without even trying to contact the Samba server that
I specified as domain controller. From the fact that MSIE works, I conclude
that it is a problem with Mozilla, not mod_ntlm.

Here is the relevant mod_ntlm configuration snippet:

        AuthType NTLM
        NTLMAuth on
        NTLMAuthoritative on
        NTLMServer 127.0.0.1
        Require user jpl


Reproducible: Always

Steps to Reproduce:
Try accessing a location protected by the Apache module mod_ntlm 0.4 using
the specified version of Mozilla.

Actual Results:  
The NTLM popup dialog reappears in Mozilla, messages are logged in Apache's
error log.

Expected Results:  
The NTLM popup dialog should disappear, the protected page should become 
displayed.

I can provide packet capture logs if requested.

Comment 1

[Darin Fisher]

Jan: thank you for the bug report.  do you have a way to post a test case for
this bug on the internet?

can you possibly post the Type1 message from a successful IE authentication? 
Please do not post the second message sent by IE since it will contain your
password (encrypted in a possibly weak manner).  the Type1 message is the first
thing IE sends in response to a NTLM challenge.  for example, you quoted that
mozilla sent:

  "TlRMTVNTUAABAAAAB4IIAAAAAAAgAAAAAAAAACAAAAA="

i'd really like to see what IE is sending in this case.  thanks!

Comment 2

[Jan Ploski]

MSIE is sending "TlRMTVNTUAABAAAAB4IIoAAAAAAAAAAAAAAAAAAAAAA=" and succeeds.

As for the public test case, it might be difficult: it seems that I get some
other (probably unrelated) errors from mod_ntlm 0.4 when I change Apache's
server name from 192.168.0.1 to remotejava.dyndns.org - then MSIE is not being
successfully authenticated either. I guess it would not make a good test case.

Note that when testing with Mozilla, I was running it on the same host as
the web server (i.e. 192.168.0.1), while the successful MSIE accesses were
coming from 192.168.0.13.

Comment 3

[Darin Fisher]

here's IE's Type1 message decoded:

bin> ./ReadNTLM 'TlRMTVNTUAABAAAAB4IIoAAAAAAAAAAAAAAAAAAAAAA='
signature =
    0x4e 0x54 0x4c 0x4d 0x53 0x53 0x50 0x00    NTLMSSP.
message type =
    0x01 0x00 0x00 0x00                        ....
flags =
    0x07 0x82 0x08 0xa0                        ....
    0x00000001 (NegotiateUnicode)
    0x00000002 (NegotiateOEM)
    0x00000004 (RequestTarget)
    0x00000200 (NegotiateNTLMKey)
    0x00008000 (NegotiateAlwaysSign)
    0x00080000 (NegotiateNTLM2Key)
    0x20000000 (Negotiate128)
    0x80000000 (Negotiate56)

and here's mozilla's Type1 message:

bin> ./ReadNTLM "TlRMTVNTUAABAAAAB4IIAAAAAAAgAAAAAAAAACAAAAA="
signature =
    0x4e 0x54 0x4c 0x4d 0x53 0x53 0x50 0x00    NTLMSSP.
message type =
    0x01 0x00 0x00 0x00                        ....
flags =
    0x07 0x82 0x08 0x00                        ....
    0x00000001 (NegotiateUnicode)
    0x00000002 (NegotiateOEM)
    0x00000004 (RequestTarget)
    0x00000200 (NegotiateNTLMKey)
    0x00008000 (NegotiateAlwaysSign)
    0x00080000 (NegotiateNTLM2Key)

seems like the only difference is that IE sends those Negotiate128 and
Negotiate56 flags, which don't actually have any bearing on the protocol.
oh well, maybe moz should just send them as well.

eric: do you have any opinion/thoughts on this?

Comment 4

[Darin Fisher]

>Note that when testing with Mozilla, I was running it on the same host as
>the web server (i.e. 192.168.0.1), while the successful MSIE accesses were
>coming from 192.168.0.13.

can you please try running mozilla on 192.168.0.13 as well?

Comment 5

[Jan Ploski]

I tried the Windows version of Mozilla 1.6b running from 192.168.0.13.
Message sent: "TlRMTVNTUAABAAAAB4IIAAAAAAAgAAAAAAAAACAAAAA=".
On the server side, the same error occured as in the previous tests.

Comment 6

[Malcolm Rowe]

darin: Flags aren't the problem here.  The two messages, in full:

IE:
4E544C4D53535000 01000000078208A0  NTLMSSP.........
0000000000000000 0000000000000000  ................

Mozilla:
4E544C4D53535000 0100000007820800  NTLMSSP.........
0000000020000000 0000000020000000  ................

The second line of the messages is domain length/domain offset/host length/host 
offset. I guess they're not relevant here, so Mozilla and IE both set the 
lengths to zero. Mozilla sets the offsets to '2', and that's causing mod_ntlm 
to fail when extracting the hostname and domainname (hence the error code 
returned from ntlm_decode_msg: 3, indicating that both ntlm_msg1_gethostname 
and ntlm_msg1_getdomainname failed).

For the life of me, I can't see *why* it works when offset = 0 and length = 0, 
but fails when offset <> 0, but I must be overlooking something simple (or 
looking at a different version of mod_ntlm).

Comment 7

[Jan Ploski]

I downloaded mod_ntlm 0.4 from SourceForge yesterday:
$Id: ChangeLog,v 1.3 2003/02/21 02:17:14 casz Exp $

Comment 8

[Jan Ploski]

I looked at mod_ntlm's source code, and the problem is indeed buried there:
the flags "Negotiate Domain Supplied" and "Negotiate Workstation Supplied" are
ignored by mod_ntlm (it seems that their meaning was not clear at the time the
code was written). mod_ntlm always tries to extract the host and domain name
from a type 1 message, and uses the "32" supplied by Mozilla as an offset in
both cases (MSIE sends "0"). The unsatisfied check of offset < srclen causes
the extraction attempts to fail.

Why does it work with offset == 0? Well, in this case the aforementioned
check is satisfied, and the length is also 0, so a valid empty string is
"extracted" by ntlm_extract_string then.

I am going to file a bug report or patch for mod_ntlm concerning these flags,
but it would not hurt if Mozilla behaved the same way as MSIE even in this
murky case, would it?

A quick hack of mod_ntlm to work around the failure allowed Mozilla to
authenticate successfully in my test setup. Thanks for your attention.

Comment 9

[Jan Ploski]

I submitted a bug report for mod_ntlm:

https://sourceforge.net/tracker/index.php?
func=detail&aid=858331&group_id=4906&atid=104906

Comment 10

[Malcolm Rowe]

Ah. That answers my question. I mentally mistranslated '20000000' (little-
endian) into '00000002' rather than '00000020', which is, of course, just past 
the end of the buffer. mod_ntlm is at fault here, but it would be nice to 
change Mozilla so that it passes zeroed offsets if the relevant flags are 
unset, if that's not too hard to achieve.

(Confirming based on comments).

Comment 11

[Darin Fisher]

yeah, there are two sides to this bug.  i think moz should send offset=0 for
empty security handles just like IE.  but, it certainly seems that mod_ntlm
should be ok with offset=length since IIS is ok with that.  patch coming up...

Comment 12

[Darin Fisher]

Attachment: v1 patch

ok, here's the patch for mozilla.  this should do the trick.

Comment 13

[Darin Fisher]

Comment on attachment 137274 [details] [diff] [review]
v1 patch

brian: can you please do the honors here.  this is a very simple patch, and it
is one that i think we should consider for mozilla 1.6 final.  thx!

Comment 14

[Darin Fisher]

any idea how prevalent mod_ntlm is?  is this a show-stopper?

Comment 15

[Darin Fisher]

fixed-on-trunk for 1.6 final.

Comment 16

[Malcolm Rowe]

Oddly, the checkin doesn't seem to be visible via bonsai, though lxr has it 
just fine. Did the checkin go in ok?

Comment 17

[Darin Fisher]

malcolm: i see it in bonsai:

http://webtools.mozilla.org/bonsai/cvsquery.cgi?module=allrepositories&branch=&dir=&file=&who=darin%25meer.net&sortby=Date&hours=2&date=week

.. and i double checked cvs directly to make sure, and it's definitely there ;-)

Comment 18

[Malcolm Rowe]

mea culpa. I was looking at the SeaMonkeyAll module - doh!

Comment 19

[Olivier Cahagne]

just to let you know trunk build 20031213 Win2k still work ok with NetApp filers
using NTLM (bug 226639). I'm not marking it verified but so far, NTLM still work
ok for me.