245742 - Apparent memory corruption after successfully authenticating using NTLM

Status: RESOLVED FIXED

(Core :: Networking, defect, P1)

Description

[[e-mail]]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/125.2 (KHTML, like Gecko) Safari/125.7
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040606 (CVS MOZILLA_1_7_BRANCH and on 1.7 RC2)

When I successfully authenticate against my work's NTLM-authenticating Outlook Web Access on 
Microsoft Exchange 5.5 on IIS 3 (which is why I'm reluctant to give the URL), Mozilla crashes when I 
proceed further with navigation and at non-repeatable points in the code (using gdb). To the user, this 
is just a crash, and I had submitted a TalkBack through TalkBack a week or so ago. Using gdb, it looks 
like the most common crash is "Program received signal EXC_BAD_ACCESS, Could not access memory.," 
though "Program received signal SIGABRT, Aborted." comes up sometimes, too.

I'm currently trying to debug this, but I though I'd at least put this bug here in case if y'all were 
planning on releasing Mozilla 1.7 soon. I would really like to have this bug fixed first, and please let me 
know if you're working on this one, too (I'll post more details as I find out more about this bug). I'm 
going to try to recompile with ElectricFence and see what might be writing out of bounds..

Thinking that the bug might be in mozilla/security/manager/ssl/src/nsNTLMAuthModule.cpp, I pulled 
down revision 1.6, which was used in Mozilla 1.6 (NTLM on the same site works properly in 1.6), and 
re-made mozilla/security/manager/ssl. Unfortunately, the crash still occurs, so the bug might be 
elsewhere.

Reproducible: Always
Steps to Reproduce:
1. Log in to a web site using NTLM
2. Navigate to another page
Actual Results:  
At the 9km view, Mozilla crashes. Inside of gdb, the crash seems to occur on non-repeatable code 
segments, potentially indicating memory corruption in the NTLM authentication process somewhere.

Expected Results:  
Quite simply, Mozilla should navigate to the requested page and not crash ;-).

Talkback crash ID: TB52990Z
Mac OS X Panther 10.3.3 and 10.3.4
PowerPC G4 450MHz (one CPU), 1GB RAM

about:buildconfig

Build platform
target
powerpc-apple-darwin7.4.0

Build tools
Compiler 	Version 	Compiler flags
gcc 	gcc version 3.3 20030304 (Apple Computer, Inc. build 1495) 
	-Wall -W -Wno-unused -Wpointer-arith -Wcast-align -Wno-long-long -fpascal-strings -no-cpp-
precomp -fno-common -I/Developer/Headers/FlatCarbon -pipe
c++ 	gcc version 3.3 20030304 (Apple Computer, Inc. build 1495) 
	-fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual 
-Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -fpascal-strings -no-cpp-
precomp -fno-common -fshort-wchar -I/Developer/Headers/FlatCarbon -pipe -I/usr/X11R6/include

Configure arguments
--enable-macos-target=10.3 --enable-official-branding --with-nss --disable-tests --enable-
optimize --enable-crypto

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: TB52990Z

Comment 2

[[e-mail]]

I now have ElectricFence 2.2.2 compiled and installed as a dylib and am now
recompiling Mozilla with ElectricFence (I had to hack ElectricFence's Makefile a
bit for OS X).

As an aside, the build process initially failed when I configured it with
ElectricFence. Apparently, xpidl's use of libIDL, which I'm presuming is
precompiled elsewhere outside of Mozilla, causes ElectricFence's free() function
to say that the requested memory was not allocated with malloc() and crash
xpidl. I had to add -lm before $(LIBIDL_CFLAGS) on the line in
mozilla/xpcom/typelib/xpidl/Makefile that starts with CFLAGS and re-make it to
get the build to not crash while running xpidl so that it uses the system's
memory functions instead of ElectricFence's.

If all goes well, I hope to have a better idea of the root cause later tonight..

Comment 3

[[e-mail]]

Unfortunately, ElectricFence was unable to detect any problems. Perhaps a
stronger tool is necessary.. I'm going to try NTLM on Mozilla 1.7 RC2 on another
computer to confirm that it's not something crazy with this particular computer
or OS. I'm going to attach two different runs of the program in gdb along with a
stack trace on the crash as text files below.

Comment 4

[[e-mail]]

Attachment: gdb log of the first run after recompiling with ElectricFence

Comment 5

[[e-mail]]

Attachment: gdb log of the second run after recompiling with ElectricFence

Comment 6

[[e-mail]]

I tried Mozilla 1.7RC3 on both this and another Mac system and encountered this
crash on both systems. Both systems are OS X, so it might be OS X-only. I
submitted a TalkBack trace on the other system, and that ID is TB81308Z.

I tried Mozilla 1.7RC3 on a Windows NT system and could not reproduce this
crash. I have not yet tried Linux, but I'm beginning to suspect that this might
only happen in Mac OS X.

It seems like this one might be difficult to pinpoint, but I'll try anyway.
Hopefully, someone who knows more about what might cause this can help out
and/or fix it :-). I'm hoping that this can get fixed before 1.7 final..

Comment 7

[Darin Fisher]

> Unfortunately, the crash still occurs, so the bug might be elsewhere.

The stack traces to the crash appear in layout.  I understand that memory
corruption could have occured outside of layout, leaving layout to stumble over
a corrupt heap or what not, but I don't see anything to confirm that it is
definitely the NTLM module corrupting memory.  (If you had a Linux build, then
you could try running valgrind on a debug build, which would give you a precise
look at the source of the corruption.)

That said, it seems to me that another possibility here is that the page we get
back after authenticating is somehow leading to a crash.  It could be anything
really... we need more information it seems.

Comment 8

[[e-mail]]

I tried Mozilla 1.7RC3 for Linux on a Fedora Core 2 system and the NTLM scenario
that crashes the OS X version worked properly in Linux. So it looks like this is
probably an OS X-only issue. I use Mozilla on OS X a lot, so I'm going to try to
find the culprit in my free time today.

Comment 9

[[e-mail]]

> That said, it seems to me that another possibility here is that the page we get
back after authenticating is somehow leading to a crash.  It could be anything
really... we need more information it seems.

I think you're correct because http://earth.sfaf.org/ with NTLM works but
http://earth.sfaf.org/exchange/ somehow does not. I'm going to try loading up
that page without NTLM and see what happens. So this potentially might not be an
issue with NTLM per-se..

Comment 10

[[e-mail]]

Okay, this is weird. When I use a shorter account name with a shorter password during the NTLM logon, 
everything works fine and no crashes come up. When I use the Administrator logon, which has a 
password that is shorter than mine, things seem to work, though it did crash while I was exiting 
Mozilla. With my account, slawrance, which has a long password, it crashes soon after logging in 
consistently, though in inconsistent locations.

I wonder if there might be a bug in the BIG_ENDIAN handling. I walked through that code a couple days 
ago and couldn't find anything wrong with it, but I'll look again..

Comment 11

[[e-mail]]

Here's what I could find so far. It looks like longer passwords increase the likeliness of an immediate 
crash. Shorter passwords may postpone a crash for a long time. In one case, it crashed while I was 
quitting Mozilla. I set up a test server on an unused, unimportant Windows NT 4.0 box I have at home 
and was able to reproduce the crash using my local NT server.

Here are the URLs for the test server. Please feel free to try it out on Mac OS X. I set it up with multiple 
authentication mechanisms to show that the page does not lead to a crash in Mozilla when used with 
cleartext or no authentication. (feel free to e-mail me directly if you encounter any problems bringing 
up the test site)

NTLM: http://ns2.moonlightdesign.org/exchange1
None: http://ns2.moonlightdesign.org/exchange2
Cleartext: http://ns2.moonlightdesign.org/exchange3

I set up sample accounts that demonstrate how password lengths affect the NTLM module. It looks like 
NT 4.0 Server only supports passwords up to 14 characters when creating users in the User Manager. 
As a result, the accounts that I set up with more than 14 characters have their passwords truncated at 
14.

Account: Password
----------------
moz: moz
mozilla: mozilla
mozillatest: mozillatest
mozilla1: 1
mozilla2: 12
mozilla3: 123
mozilla5: 12345
mozilla9: 123456789
mozilla10: 1234567890
mozilla11: 12345678901
mozilla12: 123456789012
mozilla13: 1234567890123
mozilla14: 12345678901234
mozilla15: 12345678901234
mozilla20: 12345678901234

While testing these, I actually got a quick crash with mozilla1. I wonder if the username length has any 
effect, or if what I'm seeing is random.  On shorter accounts like moz, you may need to reload the page 
several times before it crashes. It's weird. Please try it out and confirm on OS X if possible. Thanks in 
advance :-).

It's unfortunate that valgrind doesn't work on OS X :-(.

Comment 12

[[e-mail]]

Ignore Comment #9; when I tested http://earth.sfaf.org/, I was using a shorter account name. I was 
able to replicate the crash by logging in with my account and then reloading the page.

I'm wondering if it could have something to do with multiple threads using the NTLM module on OS X 
because the test pages I put up and http://earth.sfaf.org/exchange/, which are the same, have several 
images on the page that load at the same time in Mozilla. Other pages like http://earth.sfaf.org/ don't 
have as many images and are mostly text. I tried turning off image loading in Mozilla and was able to 
get farther into the logon in http://earth.sfaf.org/exchange, but still crashed. I then tried turning off 
Pipelining, and still got the crash. With HTTP 1.0 instead of 1.1, it's the same thing. When I turned off 
Keep-Alives, however, it would keep asking for my login without displaying a page unless if I press 
Cancel, and then it would display Access Denied on the page.

I'm probably approaching this problem incorrectly, but hopefully some of the information I posted here 
will help others identify and fix the bug :-).

Comment 13

[Darin Fisher]

-> me

Comment 14

[Darin Fisher]

javier: are you able to repro this crash?  see comment #11 for testcases.  thx!

Comment 15

[jhp (no longer active)]

Yes, I can recreate it.  And as Steven pointed out, sometimes in crashes when
bringing up the page, other times when closing the browser.  Mostly crashes in
gklayout, but sometimes crashes in necko and i18n.  This should be fun!

Comment 16

[chris hofmann]

scaning talkback reports for 1.7 rc3 I saw this...

     Count   Offset    Real Signature
[ 1   nsImageFrame::OnDataAvailable() 68e25b88 - nsImageFrame::OnDataAvailable() ]
 
     Crash date range: 08-JUN-04 to 08-JUN-04
     Min/Max Seconds since last crash: 48 - 48
     Min/Max Runtime: 48 - 48
 
     Count   Platform List 
     1   [Darwin 7.4.0]      
 
     Count   Build Id List 
     1   2004060805
 
     No of Unique Users         1
 
 Stack trace(Frame) 

	 nsImageFrame::OnDataAvailable()
[/builds/release/1.7/mozilla/layout/html/base/src/nsImageFrame.cpp  line 1211]  
 
     (81308)	URL: http://earth.sfaf.org/exchange
     (81308)	Comments: Logged in to an Exchange 5.5 Outlook Web Access web mail
system running on Microsoft IIS 3 on NT4 using NTLM Authentication and got a
crash after bringing up the inbox (my other computer wouldn't even get that far
 but this system is faster). Please
     (81308)	Comments:  attach to Bugzilla item #245742. This helps confirm that
it's not something weird with my original computer  though it's still the same
platform (OS X Panther 10.3).

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Comment 17

[jhp (no longer active)]

Steven, I can no longer get access to the NTLM test site.  I get "401.2
Unauthorized: Logon Failed due to server configuration".  Could you take a look?

Comment 18

[jhp (no longer active)]

Never mind, I found what was wrong.

Comment 19

[Simon Fraser [no longer active]]

Do we know when this crash started? Another way to home in on it would be to do
binary search by build date, then inspect the checkins for the suspect date.

Comment 20

[jhp (no longer active)]

Hmm, this stack seems to come up quite a bit when I'm debugging:

Call [2] [arg=26]: thread_a000a1ec |0x0 | _dyld_start | _start | main |
main1(int, char**, nsISupports*) | nsAppShellService::Run() | nsAppShell::Run()
| nsMacMessagePump::DoMessagePump() | nsMacMessagePump::GetEvent(EventRecord&) |
0x927fdb80 | 0x927ece68 | 0x927d92b4 | 0x927d9100 | 0x927f8b90 | 0x927e892c |
0x927e4a34 | 0x927d25a4 | 0x927d23ec | 0x927e30a4 | 0x927d6a0c | 0x927d25a4 |
0x927d2330 | _md_EventReceiverProc | PL_ProcessPendingEvents | PL_HandleEvent |
0x2cf00a4 | nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) |
nsInputStreamPump::OnStateStart() | nsHttpChannel::OnStartRequest(nsIRequest*,
nsISupports*) | nsHttpChannel::ProcessResponse() |
nsHttpChannel::ProcessAuthentication(unsigned) |
nsHttpChannel::GetCredentials(char const*, int, nsCString&) |
nsHttpChannel::GetCredentialsForChallenge(char const*, char const*, int,
nsIHttpAuthenticator*, nsCString&) |
nsHttpChannel::GenCredsAndSetEntry(nsIHttpAuthenticator*, int, char const*, char
const*, int, char const*, char const*, char const*, nsHttpAuthIdentity const&,
nsCOMPtr<nsISupports>&, char**) |
nsHttpNTLMAuth::GenerateCredentials(nsIHttpChannel*, char const*, int, unsigned
short const*, unsigned short const*, unsigned short const*, nsISupports**,
nsISupports**, char**) | nsNTLMAuthModule::Init(char const*, unsigned, unsigned
short const*, unsigned short const*, unsigned short const*) | 0x252cea54 |
nsSubstring::Assign(unsigned short const*, unsigned) |
nsSubstring::ReplacePrep(unsigned, unsigned, unsigned) |
nsSubstring::MutatePrep(unsigned, unsigned short**, unsigned*) | 0x2d13790 |
malloc | malloc_zone_malloc

Debugging right now to see if some string is getting overrun.

Comment 21

[jhp (no longer active)]

Attachment: patch

WriteUnicodeLE is expecting the size of the string, not size of buffer.  Input
size was twice as necessary, so the for loop in WriteUnicodeLE was going off
the end of the buffer.

Comment 22

[Darin Fisher]

Comment on attachment 150564 [details] [diff] [review]
patch

r+sr=darin

thanks javier!!

Comment 23

[Darin Fisher]

Comment on attachment 150564 [details] [diff] [review]
patch

This a good low-risk patch for 1.7

Comment 24

[tor]

Comment on attachment 150564 [details] [diff] [review]
patch

a=tor for 1.7 branch.  Please get in this in ASAP to make the 1.7 builds.

Comment 25

[Darin Fisher]

fixed-on-trunk, fixed1.7, fixed-aviary1.0 (is there a keyword for this?)

Comment 26

[David :Bienvenu]

we're using the status whiteboard for fixed-aviary1.0

Comment 27

[[e-mail]]

Wow! I must have spent four or so hours mentally tracing through that and other
functions in the path, including md4sum, but totally did not catch that at all.
How silly I now feel; I need to sharpen my C/C++ bug finding abilities ;-) (I've
been programming in Java for too long anyway ;-).

Thank you very, very, very, very much for finding that and fixing it :-)!!!!!

I tested it out on my working copy and confirm that it works properly :-). Now I
can use my work's Outlook Web Access from home on Mozilla 1.7 :-)!!

As an aside, I can leave the test system up in case if it's needed as a test
case for further before/after testing. If it's needed or not, please let me know
:-). I'm guessing that I can take down my NTLM test system on or before Mozilla
1.7 final's ship date..

Comment 28

[beanladen]

Is this fix also the solution for bug #238379 (crasher after NTLM with Solaris) ?

Comment 29

[Darin Fisher]

(In reply to comment #28)
> Is this fix also the solution for bug #238379 (crasher after NTLM with Solaris) ?

possibly.  it affects any big endian platform.

Comment 30

[beanladen]

*** Bug 238379 has been marked as a duplicate of this bug. ***

Comment 31

[Olivier Cahagne]

*** Bug 237809 has been marked as a duplicate of this bug. ***

Comment 32

[Olivier Cahagne]

*** Bug 245991 has been marked as a duplicate of this bug. ***

Comment 33

[beanladen]

Problem is fixed in newest Solaris8 build (13-Jun), both mail and password 
protected pages work again with NTLM. Thanks, Darin !

Comment 34

[Darin Fisher]

> protected pages work again with NTLM. Thanks, Darin !

Thank Javier instead! :-)

Comment 35

[beanladen]

Oh, sorry, Javier, of course those thanks should be routed to you...Blinded by the
light...

Comment 36

[[e-mail]]

As a follow-up, the test server is now offline.

Thanks again for fixing this issue in time for Mozilla 1.7 :-)!