80457 - crash or hang loading page in nsHttpChunkedDecoder

Status: VERIFIED FIXED

(Core :: Networking, defect)

Description

[James Green]

cvs build from today, having removed dist/bin/component.reg and created a new
profile, I'm hanging loading the checkins page.

This prolly hangs the product on tons of other pages too.

Comment 1

[James Green]

Ugh, ok try reassigning again -- cache messed up.
Darin is apparently being paged on this, reassigning to him. Adding smoketest
keyword since this is being seen by at least one other person and it's highly
likely to be hit during smoketesting.

Comment 2

[R.K.Aa.]

RH7.1 without compat egcs (using gcc 2.96-81) so perhaps a fringe case, but for
me moz crash when loading the checkin page. Doesn't crash on bug-page.
Pulled at 05/12/2001 01:32 bonsai time, clobbered and built.

Comment 3

[R.K.Aa.]

Attachment: same page w. problems, but doesn't look like netwerk (backtrace)

Comment 4

[R.K.Aa.]

sorry: WITH compat-egcs (6.2)

Comment 5

[James Green]

Yeah the page crashed for me too with an overnight build (checked out from cvs
shortly after the carpools). So I decided to pull 'n' build again today see if
it fixed it, but it hangs now.

I've given it to darin since cls is seeing hangs/crashes on this page too, and
his backtrace indicated networking. cc him.

Comment 6

[R.K.Aa.]

Hmm. Crashed twice in a row with that backtrace. Deleted cache - tried again -
same crash. But now, after first surfing a lot of other sites, the page loaded
once just fine. There's trouble today though - can't send attachments in
bugzilla, crash when i reply to ng postings.. will repull

Comment 7

[James Green]

Ought to mention that yesterday I upgraded to gcc 2.95.4 from .3. I'm also
building with -O2.

Comment 8

[R.K.Aa.]

rebuilt - i still crash but the backtrace changed. Now it looks more like
network related.

Since we don't use egcs this might not be a blocker. Someone on egcs should
confirm this.

As far as i understand it, mozilla isn't guaranteed to even build with gcc 2.95
and above - far less run - since it's designed for an old compiler

Comment 9

[Darin Fisher]

investigating...

Comment 10

[R.K.Aa.]

Attachment: new backtrace

Comment 11

[James Green]

Darin: are you able to reproduce?
Anyone tried with a nightly?

Comment 12

[K Carter]

I crash trying to load this page on Windows 98 with the latest nightly (the one
posted May 12 at 11:32) Windows Installer.  Sorry I don't have the build ID - I
reverted to a previous build because of the crashing.  This is on an old profile.

Comment 13

[Alex]

I first noticed this with a 2001051104 release nightly that was put on the FTP 
server between 5pm to 6pm on May 11. The build with the exact same Build ID, but 
put on the server much earlier on the same day, DID NOT experience this crash. 
Tested on Windows 98 SE.

This might not be related to this bug at all, but in the first 2001051104 
context menus were completely broken, while the later 2001051104 had them 
semi-functional.

Comment 14

[Darin Fisher]

Attachment: this patch should fix the crash inside the chunked decoder

Comment 15

[Dan Rosen]

r=dr

Comment 16

[R.K.Aa.]

bug 80481 reports a crash on the same page in Windows. Several talkbacks
referred to there. Dup?

Comment 17

[R.K.Aa.]

and bug 80478 seems to be a dup. Same backtrace as in "new attachment" here.

Comment 18

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

FWIW, this patch fixes the frequent crashes I was seeing yesterday loading
tinderbox build logs, but it doesn't fix the fact that the logs don't finish
loading.

Comment 19

[R.K.Aa.]

i've seen a lot of pages seemingly never finishing to load lately:
even if all content is on page the throbber keeps on going.
Is that what you see, or is content actually missing?

Comment 20

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Content is missing whenever I select "View Full Log" after clicking on one of
the "L" links in http://tinderbox.mozilla.org/showbuilds.cgi?tree=SeaMonkey .

Comment 21

[James Green]

R.K.Aa: The problem of pages seemingly never finishing to load is covered by
multple bugs in bugzilla, and in fact TWO tracking bugs for them, so they are
seperate.

My build from yesterday morning (prior to tree closure) did work fine iirc -- or
at least I would have reported this then. So this is definitely a recent regression.

Comment 22

[cls]

With the patch, I'm seeing a hang loading the tinderbox logs with disk cache
turned off (via prefs).  And it didn't finish loading on top of that.  Turning
disk cache back on doesn't seem to make a difference.

Comment 23

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Does the "hang" you're seeing involve full CPU usage?  If so, it may be fixed by
the event supression patch to nsSocketTransport Darin emailed me yesterday evening.

Comment 24

[James Green]

I just built (linux) from the tip, with Memory and Disk cache turned on, and the
bonsai page now _crashes_ for me quite a way into the load, I'd say about five
pages worth and still coming.

Re: did it hang with 100% cpu before - no it didn't. Mozilla froze and took no
cpu time at all. ps ax | grep mozilla showed two entries, when I normally expect
5 or more which may or may not be significant (right now I've got five in total).

Comment 25

[R.K.Aa.]

modifying summary so people find this bug

Comment 26

[Jus]

I am seeing this under Windows XP build 2001051208. Change OS to ALL?

Comment 27

[Stephen Anderson]

*** Bug 80481 has been marked as a duplicate of this bug. ***

Comment 28

[Stephen Anderson]

I've had a lot of crashes associated with this bug and generated a lot of
talkbacks.  (See bug 80481)  I don't know if it adds anything to the discussion
but I just had the crash again and it was a little different.  All my previous
crashes happened in the same (wish i could remember) dll.  This latest happened
in a different (one of the javascripts) dll.   Anyway, if it's helpful, its
talkback TB30371522G

Comment 29

[Matthias Versen [:Matti]]

*** Bug 80472 has been marked as a duplicate of this bug. ***

Comment 30

[James Green]

Per bug 80472, this is not confined to bonsai, re-summarising for better
searches -- sorry for spam :(

Anyone seeing this on Mac?

Comment 31

[David Caswell]

Semiverified, Linux tar.gz 2001051206...http://www.isnnews net and
http://www.linuxgames.com crash as well. I'm not COMPLETELY certain this is the
same bug, but this trouble came up at or around the same time as the blocker, so
I'm guessing 80% chance. (Can't do a stack trace right now to completely
confirm; GDB isn't cooperating...)

 What's odd is that sometimes the processes don't completely die on segfault;
one or two idle processes remain that have to be killed manually. I don't get
that...

Comment 32

[Dan Rosen]

i'm (still) seeing this, probably because the fix isn't in :) adding cc.

exact same stack trace for me as the "new backtrace" attachment, by the way.
also, i can confirm david caswell's observation that some processes don't die
cleanly after the segfault, fwiw.

Comment 33

[Dan Rosen]

Some additional info that might or might not be useful:

In the debugger, after the initial crash (which yielded the backtrace indicating
nsHttpChunkedDecoder), I did a "next" and received a *new* segfault with a
different trace:

#0  0x40276033 in PR_SetThreadPrivate
    (index=1, priv=0x0) at prtpd.c:165
#1  0x40113c3b in nsAutoLockBase::~nsAutoLockBase
    (this=0xbedffa3c, __in_chrg=0) at nsAutoLock.cpp:262
#2  0x4015e06a in nsAutoLock::~nsAutoLock
    (this=0xbedffa3c, __in_chrg=2) at ../../../../dist/include/nsAutoLock.h:141
#3  0x401182aa in nsThreadPool::GetRequest
    (this=0x81632d0, currentThread=0x82a5a50) at nsThread.cpp:596
#4  0x40119080 in nsThreadPoolRunnable::Run
    (this=0x82a5a38) at nsThread.cpp:834
#5  0x40116bf5 in nsThread::Main
    (arg=0x82a5a50) at nsThread.cpp:106
#6  0x402903ee in _pt_root
    (arg=0x82a5ac8) at ptthread.c:198
#7  0x402b0c8e in pthread_start_thread_event
    (arg=0xbedffc00) at manager.c:274

It looks like |self| is junk:

(gdb) print self
$1 = (PRThread *) 0x33090909

(gdb) print self->privateData
Error: Cannot access memory at address 0x3309092d

(gdb) print self->tpdLength
Error: Cannot access memory at address 0x33090929

This might explain why some processes don't die -- perhaps our threads are
freaking out before they can kill them...? I don't know. Hope this helps :)

Comment 34

[Dan Rosen]

FYI, I get another new segfault in our event queue, following the last one in
threads. gdb just died on me, but I get the feeling we have some underlying
safety issues in threads, which are probably the subject of a different bug.

Comment 35

[Stephen Moehle]

*** Bug 80483 has been marked as a duplicate of this bug. ***

Comment 36

[Stephen Moehle]

*** Bug 80480 has been marked as a duplicate of this bug. ***

Comment 37

[Darin Fisher]

Attachment: fixes the crash and the hang

Comment 38

[Bradley Baetz]

r=bbaetz

Comment 39

[Matthias Versen [:Matti]]

*** Bug 80605 has been marked as a duplicate of this bug. ***

Comment 40

[Simon Fraser [no longer active]]

darin: i note these lines in the patch:

-        p = PL_strstr(buf, "\r\n");
+        p = PL_strnstr(buf, "\r\n", count);

can we always be sure that the data has CRLF linebreaks here? Isn't is the case 
that some pages may be served with just CR, or just LF breaks?

Comment 41

[R.K.Aa.]

Just pulled and built after applying the patch: Crash is gone loading bonsai,
and loading tinderbox "L" links now complete.

Loading URL from bug 80472 i still see another crash - stream/plugin related.
Backtrace in bug 80504.

Comment 42

[Darin Fisher]

simon: i was going by the spec here.. assuming servers would send \r\n, but
perhaps that is too optimistic of me.  it looks like the old code accepted
either \n or \r\n.  i'll submit a new patch, which does the same.

Comment 43

[Darin Fisher]

Attachment: revised patch to make the chunked decoder more robust

Comment 44

[Johnny Stenback (:jst)]

sr=jst

Comment 45

[Matthias Versen [:Matti]]

With this patch, bug 80472 and bonsai is fixed (win32)

Comment 46

[Darin Fisher]

fix checked in

Comment 47

[James Green]

Tinderbox and Bonsai pages work following patch on the tip. VERIFIED on linux.