130359 - Hang while fetching IMAP mail over SSL [Mac]

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect, P2)

Description

[FZiegler]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:0.9.9) Gecko/20020310
BuildID:    2002031005



Reproducible: Sometimes
Steps to Reproduce:
1. Launch Mail & News, connect to IMAP server
2. Select message in thread pane


Actual Results:  Since upgrading to 0.9.9 (today):
(1) Some messages display OK.
(2) Some display only the first few lines (and nothing in "View Source")
(3) Some cause hang (spinning beachball, 80% CPU, have to force quit)
(4) Some cause complete crash (see backtrace attached)

Expected Results:  All messages should download and display.
(0.9.8 and subsequent nightlies gave me none of those problems, except maybe (2)
sometimes.)

If that matters:
• The IMAP account is over SSL
• I'm using 3-pane view
• Backtrace below was when downloading a mulitipart message
  (sourceforge mailing list digest), but I also got crashes
  on simpler messages.

Comment 1

[FZiegler]

Attachment: Stack trace

Comment 2

[John G. Myers]

Appears to be parsing a bodystructure, probably with a literal in it.  An IMAP
telemetry log would be helpful.

user_pref("imap.io.mac.logging",  true );

in the preferences file is how one enables this on Mac Classic, I don't know if
that would work on OS X.

Comment 3

[FZiegler]

This keeps happening with all post-0.9.9 builds, and it's pretty bad. To tell
the truth, it's kept me from really using them for two weeks now.

My account above is a bit oversimplified: what happens in fact is that Mozilla
*stalls* when trying to download the message (if I let it, it keeps spinning for
over a half hour), and the actual *crash* only happens when I hit the "stop"
button. The backtrace is always the same.

As you suggest, it is choking on the BODYSTRUCTURE of those digests -- most
likely because of the long lines, 408 through 411, in the logs below.

Comment 4

[FZiegler]

Attachment: imap log with build 2002-02-11 (works)

Successful download of a digest with pre-0.9.9 build

Comment 5

[FZiegler]

Attachment: imap log with build 2002-03-26 (crashes)

Failed download of the same digest with post-0.9.9 build

It chokes on line 410, another try stopped at 409.

Comment 6

[FZiegler]

Attachment: Different Mac, Newer Build, Same Crash

Stack trace with build 2002-03-31, installed on another machine.
(iBook 500 MHz; previous one was on an iMac.)

Scenario and message being fetched are the same as in comment #3.

Comment 7

[FZiegler]

Attachment: imap log, hangs while fetching flags

Moreover it's also starting to hang while fetching the message flags.
This is after a few more messages arrived in the mail spool. The attached log
shows:

line  420: 1st attempt to fetch flags for INBOX, stalls at message 181
line  836: 2nd attempt to fetch flags for INBOX, stalls at message 192
line 1252: 3rd attempt to fetch flags for INBOX, stalls at message 192
line 1288: flags successfully fetched for another mailbox (sent-mail)
line 1703: 4th attempt to fetch flags for INBOX, stalls at message 192.

The stalls can continue for minutes, I have to hit the stop button -- but here
it doesn't cause a crash. Again, all of this is consistently reproducible under
both builds 2002-03-31 (stalls at message 181, then 192) and 2002-03-31 (stalls
at message 182, then 192).

Those messages have nothing special, an earlier build such as 2002-02-11
fetches everything without problems.

Comment 8

[FZiegler]

Sorry, above should read:

2002-03-31 (stalls at message 181, then 192) and 2002-03-26 (stalls
at message 182, then 192).

Comment 9

[David :Bienvenu]

Several things: 

1. Could we get access to the message with the big body structure response that
causes the crash? It looks like you're using a UW server so we'll either need to
get access to that server, or copy the message to a test UW server and hope the
problem happens here. 

2. I'm not sure if the hang is related - did you press stop in that session?
That would explain the (null) at the end of the log. Other than that, my only
guess is that there's something going on with the mac networking or event
processing that's causing us to stall - maybe a race condition that we're now
losing.

It would really help if we could narrow down when this problem started
happening. This is a problem on the .9.9 branch, which was cut on 03/02/02 and
took changes until 03/11/02. Does Mozilla have archived nightly builds you could
try? I can't find them on the web-site, unfortunately. You've tried later trunk
builds and they still have the problem. Cc'ing a couple mac people and a
networking person in case they know about any similar problems.

Comment 10

[FZiegler]

Attachment: message that caused the above crashes

As per bienvenu's request. This is the very message involved in the previous
logs and stack traces; several others from the same mailing list also cause the
same crash.

Comment 11

[FZiegler]

Attachment: imap response lines where it stalls

In the imap logs of attachment #76731 [details] and attachment #76732 [details], the BODYSTRUCTURE
response lines (408 thru 411) are cut short. Here are the complete lines
obtained manually from the server.

As you can see they are quite long, especially the first one at 4347
characters. (But as I said in comment #5, Mozilla only chokes on the second or
third.)

Comment 12

[David :Bienvenu]

I've tried this message on our IMAP server, accessed by the win2K client, and it
looks like our client returns the same long body structure lines (or at least,
it returns 4 lines that get truncated by the log) and I don't hang/crash. Which
makes me suspect something mac-specific (rather than server-specific).

Comment 13

[David :Bienvenu]

could you try it with SSL turned off? Also, this message crashes for you without
pressing stop?

Comment 14

[John G. Myers]

On OS X 0.9.9 I've seen several cases where IMAP/SSL stalls while reading large
amounts of data from the server.  One thing that triggered it was downloading
captions from my 13,000-message folder.  Every 800-1200 captions or so, the
connection would stall.  I'd have to dismiss the messenger window and start
again with a new IMAP/SSL connection.

Comment 15

[David :Bienvenu]

That was JF's experience as well - turning on SSL on Mac OS X made fetching the
problem message stall. Who knows who we should contact about SSL issues on Mac OS X?

Comment 16

[FZiegler]

> 1. Could we get access to the message with the big body structure
> response that causes the crash? It looks like you're using a UW server
> so we'll either need to get access to that server, or copy the message
> to a test UW server and hope the problem happens here.

Yes, it's a vanilla UW server:

[gauss:~]$ rpm -qi imap
Name        : imap                         Relocations: (not relocateable)
Version     : 2000                              Vendor: Red Hat, Inc.
Release     : 9                             Build Date: Thu Apr  5 13:51:03 2001
Install date: Tue Jul  3 12:01:49 2001      Build Host: porky.devel.redhat.com
Group       : System Environment/Daemons    Source RPM: imap-2000-9.src.rpm
Size        : 2079581                          License: University of
Washington's Free-Fork License
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL         : http://www.washington.edu/imap/
Summary     : Server daemons for IMAP and POP network mail protocols.


The problem does NOT happen with mac.com's imap server (Netscape Messaging
Multiplexor), where I also uploaded this message. It's not over SSL, though.

Curious note: in attachment #77090 [details], I took the message straight from my
/var/spool/mail/$USERNAME. If on the other hand I try to extract it by doing
"View > Message Source" in Mozilla, then I do NOT get the whole thing.

Instead it is cut short after "exit c" on line 828 (of 945). This happens at the
exact same place on both servers (UW and mac.com's Netscape). And this place
happens to be just about the 32768 th character in the output. Coincidence?...

Comment 17

[FZiegler]

In reply to comment #9:

> 2. I'm not sure if the hang is related - did you press stop in
> that session? That would explain the (null) at the end of the log.

I agree that this hang (of comment #7, in fetching FLAGS) is of a different kind
than the one that leads to the crash (of comment #3, in fetching BODYSTRUCTURE).
In fact I've just noticed that someone else already filed the former as bug
133401, where it may perhaps best be discussed.

(Yes, I did press stop, and that's when the line with "(null)" appeared. Before
that, it was just stalling on the preceding line.) 

Comment 18

[FZiegler]

In reply to comment #9:

> It would really help if we could narrow down when this problem
> started happening. This is a problem on the .9.9 branch, which was
> cut on 03/02/02 and took changes until 03/11/02.

I've not tested builds between 2002-02-11 (which exhibits neither kind of hang)
and 0.9.9, but of course we could. One important difference in 0.9.9 is that a
digest's messages now appear in the "attachment pane" with full titles, rather
than just numbers. Where/when was the code introduced for this? I can't seem to
find it. 

Another thing I can say is that those hangs and crashes are only with CFM
builds. In Mach-O the networking works differently (IIUC), and I verified
yesterday that a crypto-enabled Mach-O build has none of those problems.

Comment 19

[Jean-Francois Ducarroz]

>One important difference in 0.9.9 is that adigest's messages now appear in the
>"attachment pane" with full titles, rather than just numbers. Where/when was the
>code introduced for this? I can't seem to find it.

It's was part of a fix I did a while ago. I use the subject for inline messages
instead of putting the part number.

Comment 20

[FZiegler]

In reply to comment #13:

> could you try it with SSL turned off?

Well, I did, but on another server (at mac.com, no hang, no crash). I can try on
the very same UW server using stunnel in place of NSS, would that be of interest? 

> Also, this message crashes for you without pressing stop?

No... without pressing stop it just stalls forever, as I was saying in comment
#3. Then I press stop, and it crashes.

Comment 21

[FZiegler]

In reply to comment #19:

> It's was part of a fix I did a while ago. I use the subject for
> inline messages instead of putting the part number.

What I was wondering is, at what point does Mozilla memorize those subject
lines? Is it already while parsing the BODYSTRUCTURE, or later? With these
ultra-long lines from the imap server, might there be some buffers filling up
faster than when it was just the part numbers?

Comment 22

[Jean-Francois Ducarroz]

it's done by mime using the message/rfc822 data received from the protocol
service (IMAP, POP3, NNTP), this data has always been there, even before I
decide to use it for the attachment name, therefore it should not matter in that
case.

Comment 23

[David :Bienvenu]

>Well, I did, but on another server (at mac.com, no hang, no crash). I can try on
>the very same UW server using stunnel in place of NSS, would that be of interest? 

We're already way past my knowledge of SSL on the mac - it might be of interest
to somebody, so it's worth a try - more data never hurt.

Also, I don't know what CFM and Mach-O are.

Comment 24

[FZiegler]

> this data has always been there, even before I decide to use
> it for the attachment name, therefore it should not matter in
> that case.


OK, so that's not it. (I like having those subject lines, by the way! Much nicer
than before.)

I've just found the following report, which seems to indicate that this was a
frequent crasher in Linux talkbacks at some point. Exact same stack trace: 

bug 123026

Comment 25

[FZiegler]

Attachment: Latest crash

All right, the crash (and any hangs) disappear if I turn off Mozilla's built-in
SSL and use tunnelling instead (OpenSSL + stunnel).

Then I tried to revert to using Mozilla's SSL. (Before that, I put another copy
of that message by itself in a new folder "test", so that I could try and
download it even if a listing of my inbox failed due to the hangs of comment
#7.) I went about it in two different ways, which gave two different results:

1) First, I went back to Mozilla's SSL in the SAME SESSION, by just putting
back the old values in "Edit > Mail & Newsgroups Account Settings". That way,
there was still no problem! Flags were fetched OK, and the message downloaded
fine from both my inbox and the new "test" folder.

2) But second, I quit and relaunched Mozilla in a NEW SESSION (still
2002-03-31, still set to use Mozilla's SSL). Then everything was back to the
ugly behavior: The inbox listing stalled, so I had to stop it. The "test"
folder listing went OK, but things stalled again when I tried to download the
message I had copied there. And when I stopped that (by closing the mail
window, for a change), it gave me again the same crash and same stack trace
(attached).

(Note: as the stack trace shows, this is on an iMac with OS X 10.1.1. In case
you wonder, the crash of comment #6 was with the latest, 10.1.3; so I don't
think the OS version is a factor.)

Comment 26

[FZiegler]

In reply to comment #12:

> I've tried this message on our IMAP server, accessed by the win2K
> client, and it looks like our client returns the same long body
> structure lines (or at least, it returns 4 lines that get truncated
> by the log) and I don't hang/crash. Which makes me suspect something
> mac-specific (rather than server-specific).

Yes, I forgot to say that the mac.com server also returned those long lines.
Out of curiosity:

1) was this with SSL, or just plain imap?
2) does View Message Source give you the same as me? (cf. comment #16)

Comment 27

[David :Bienvenu]

I tried it with and w/o SSL - it worked both ways.

View Message Source does seem to truncate the message for me (though not at the
same place as you, I think), if I do it from the IMAP message (SSL or not).
Viewing the source of the local message works fine. I suspect this is unrelated,
though it is a bit odd.

Comment 28

[FZiegler]

Same crash, same stack trace, still happening with 2001-04-11 (Mac OS X Carbon
CFM build)...

To further corroborate comment #12, I verified that the Linux PPC build has no
problems fetching the same message from the same (imap ssl) server. So this
seems indeed purely a Mac (+ imap + ssl) problem.

I see only post-0.9.9 builds at http://ftp.mozilla.org/pub/mozilla/nightly/. Has
anyone tried to test this, as Bienvenu suggested in comment #9, with the builds
between 2002-02-11 (which works) and 0.9.9 (which doesn't)? 

Comment 29

[FZiegler]

Attachment: imap log with CFM build 2002-04-11, using Mac OS 9

Using the same build (2002-04-11-05 Carbon CFM, Mozilla 1.0.0 branch) under Mac
OS 9.

I can't fetch the test message either, it hangs at the same place. In today's
build the "stop" button won't even respond, the only way to proceed is to just
close the Mail & News window. (This is true under OS X too.)

If I close the window, the behavior is less predictable than under OS X (where
Moz crashes every time). Under OS 9 it stays alive and the browser still works,
but I can never open another Mail window -- trying that only gives an empty
line in the "Window" menu. To get back into Mail I have to quit (sometimes
force quit) Mozilla.

The imap log (attached) has one extra line of possible interest, which says:
"PARSER:Internal Syntax Error: %s:"

Comment 30

[FZiegler]

Attachment: imap log with Mac build 2002-04-11-05 (non-Carbon)

For completeness, just tried the today's pure OS 9 build
(http://ftp.mozilla.org/pub/mozilla/nightly/latest-1.0.0/MacMozillaFullInstall.sea.bin)


Behavior & imap log (attached) exactly the same as with the CFM build (previous
comment).

So in short, the problem seems specific to the Mac builds, Classic and CFM but
not Mach-O.

Comment 31

[David :Bienvenu]

*** Bug 133401 has been marked as a duplicate of this bug. ***

Comment 32

[FZiegler]

Using Apple's Sampler.app, one can see a little more of the stack. I'm attaching
one-minute call graphs taken as Mozilla spins while fetching flags (comment 7),
resp. bodystructure (comment 3). I abbreviated the branches that don't involve
nsImapProtocol, but note that we're hanging 99.7% of the time in them... 

Comment 33

[FZiegler]

Attachment: Call graph while fetching FLAGS

Comment 34

[FZiegler]

Attachment: Call graph while fetching BODYSTRUCTURE

Comment 35

[Simon Fraser [no longer active]]

Those traces look just like the issue in bug 102797 (lots of time spent in
WaitNextEvent, not enough servicing threads). That bug describes a slowdown,
however. This one is actual failure. I'm not sure Sampler is going to tell us
much here.

Comment 36

[FZiegler]

Well that gives me hope, since you have a near-fix for 102797. The reason I took
these samples is that someone on n.p.m.macosx said he was experiencing similar
hangs with *POP* over SSL. (The SSL part is not in his message, but he wrote me
to confirm it when I asked.)

So I was beginning to wonder how much of this is IMAP-specific, and if those
stack traces that stop at strlen() in nsIMAPGenericParser are not misleading.
Now, attachment 92990 [details] seems to show that it gets past that and waits inside
CreateNewLineFromSocket() -- same as with flags (compare the other attachment),
and iiuc, same as in bug 102797. 

Comment 37

[Karen Huang]

This bug should be specific for IMAP/SSL -- not the same as bug 102797.
Since w/ bug 102797 fix, I did the following verification on IMAP SSL for 
yesterday & today's trunk builds, I still can reproduce this problem....
For today's build, it is even worse than yesterday's build since it stop at 762
of my IMAP w/SSL 1062 Inbox msgs headers:

Before bug 102797 fix (08-05-08 trunk):
It takes 2 minutes 30 seconds for downloading IMAP/SSL Inbox 1000 msgs headers on
08-05 trunk build, run on G4/450Mhz

After bug 102797 fix (08-07-08 trunk):
It takes more time(even worse)..it stops at 762 of my IMAP 1062 Inbox msgs
headers and never complete for downloading IMAP msgs headers on IMAP SSL, 
run on G4/450Mhz

Comment 38

[Simon Fraser [no longer active]]

Who's gonna drive this bug?

Comment 39

[Jean-Francois Ducarroz]

I should be the one who take care of that issue but I am going soon on vacation
for 3 1/2 weeks and I have already some other big thing on my plate, I need to
see if I can free some time. And Bienvenu who own IMAP does not have a Mac!

Comment 40

[Dannie M Stanley]

I have been using trunk build 20020808 for most of the day, previous to that I
was running trunk build 20020807 and I have noticed a significant performance
increase and significant reduction in IMAP/SSL problems.  In fact I have only
noticed a problem similar to this one once in the last 4 hours (I get regular
mail, > 100 messages a day) and it was different enough to make me wonder if
this bug has been resolved.

My new problems seem to be related to cacheing... they are minor but indicate
perhaps something else is amis unrelated to IMAP/SSL specifically.

Comment 41

[David :Bienvenu]

Dannie, have you noticed any performance changes in the time it takes to read
local messages, i.e., messages in local folders? That would tell us if the
slowness reading an imap message for the second time is really just a problem
reading local messages. There were lots of changes to the file io code checked
in, completely unrelated to Simon's change.
If the other IMAP SSL problems are timing-related, then Simon's change could
easily change the timing and make the problems less likely to occur for some users.

Comment 42

[Dannie M Stanley]

I don't use any local folders.  However, I have unchecked the checkbox "Make the
messages in my Inbox available when I am working offline" and it seems to have
made a significant difference.  I.E. I don't get any of the symptoms I reported
in bug 102797

Comment 43

[David :Bienvenu]

right, can you create a local folder and copy some imap messages down to it and
see if they're as slow to load as the messages that have been downloaded for
offline use? especially the larger messages, like the 78 k message you mentioned
before.

Comment 44

[Dannie M Stanley]

I just created a new offline folder named "test".  I then copied 60 messages of
varying sizes to "test".  I did not notice a delay loading large messages (>
100KB) from "test" where the messages was primarily a small text message plus 1
or more attachments.  However I noticed a significant delay loading a 68KB all
plain text message of 11-12 seconds.  This appears to be consistent with my
testing yesterday when I had my folders available for offline access.

Comment 45

[Kai Engert (:KaiE:)]

This appears to be a Mac only bug. Unfortunately I don't have one.

Comment 46

[John Unruh]

Test results: Using a NEW profile both times. Getting inbox IMAP/SSL email in
order to get the password entered, then clicking on the trash folder and timing
the downloading of 771 headers from that folder.
8/5/02 commercial Mac OSX trunk build - 111.258 seconds
8/9/02 commercial Mac OSX trunk build - 3.15 seconds

Marking this fixed.

Comment 47

[Simon Fraser [no longer active]]

Whoa, not so fast soldier. Nothing was checked in that fixes this bug. I still
see it on Mac OS 9 (admittedly, I have not tried a new profile), so I'm sure
that the bug still exists. Try using IMAP/SSL all day, and I'm sure you'll see
issues.

Comment 48

[John Unruh]

Right, reopened. SSL/IMAP still stalls sometimes. I had forgotten to import my
cert for the second test.

Comment 49

[FZiegler]

*** Bug 149864 has been marked as a duplicate of this bug. ***

Comment 50

[FZiegler]

Transferring nsbeta1 keyword from bug 149864, and changing OS to 9.x as per
comments 30 and 47.

John, were you able to find out if any February nightlies have been archived?
Following Bienvenu (comment 9), it seems to me that backing out whatever patch
caused the bug to appear between 2002-02-11 and 2002-03-11 would provide the
easiest way out, especially if Mozilla decides to move on to Mach-O (cf. bug
158927, comment 14).

Failing that, is there any possibility that a few of these nightlies could be
recompiled and placed in .../nightly/experimental? I (and I'm sure others here)
would be more than willing to check them against the reproducible test cases I have.

Comment 51

[FZiegler]

Going out on a limb...

Might this have something to do with the patch in bug 121951 ?

Comment 52

[Andy Gerweck]

Another data point AND A WORKAROUND:

I'm using Mozilla 1.1 release and Mac OS X.2.  Any time mozilla transfers a
large amount of data (> a couple KB, roughly) from the IMAP daemon, it becomes
extremely slow or hangs.  This affects large messages, header collection and the
subscribe/unsubscribe to folders dialog.  It is 100% reproducible when using SSL
+ IMAP.

The workaround: don't let mozilla know it's using SSL.  There's a small
application called SSL Enabler available from Apple's download site (or probably
others).  It's really just a nice UI to stunnel, which lets you set up SSL
connections at the OS level.  You remap the IMAP port on your local machine to
the SIMAP port on the server and just tell Mozilla to talk to the local machine.
 OS X does the rest, and secure IMAP works fine on OS X.  This may seem
complicated, but SSL Enabler is very simple to use and includes explicit
instructions on this exact use case.

Comment 53

[David :Bienvenu]

Wow, that's very odd. I don't know what it would be about our SSL implementation
and its interaction with IMAP that would cause this. My guess would be some sort
of contention over the event queues, since IMAP uses xpcom proxies to send data
to the ui thread, and those use event queues, and I believe the SSL
implementation does the same, or something similar.

I did slightly tweak the imap code that reads lines from the sockets recently.
You might try a trunk build from today (8/30/02) and see if the problem still
occurs. My guess would be that it still does, but it wouldn't hurt to try.

Comment 54

[Simon Fraser [no longer active]]

My guess is that the cause of this bug lies in the Mac NSPR code, but I haven't
had a chance to look at it yet.

Comment 55

[bill]

I've noticed this bug showing up almost exclusively for messages that are 6-7k
in size.  As soon as you try to read one of those messages, you can no longer
read ANY more messages until you restart Mozilla.  I use SSL/IMAP all day long
and this is a real pain in the ****.

WORKAROUND:  sometimes (definitely not always) you can hit "escape" within a
second of selecting such a message and you'll see the entire message (headers
and body), the the spinner will stop throbbing, and you'll be able to continue
without error.  But this doesn't work most of the time.

Comment 56

[Antonio.Xu]

who can reproduce this bug on linux or solaris?

Comment 57

[FZiegler]

antonio, if you mean reproducing this exact bug -- as in the testcases of
comment 10 or bug 133401 -- then I don't think anyone could (except perhaps by
using Mac-on-Linux). See comment 28.

otoh, if you mean getting the same crash trace (cf. comment 24), then I don't
know; sdonner added an http://climate/reports/ URL the other day, is it showing
up there?

My impression (based on the call graphs, comment 32) is that the crash is only
an epiphenomenon here -- i.e. a separate problem, which gets exposed by this bug
and got exposed by something else in bug 123026. Would this make sense?

sfraser, did you get a chance to pursue the NSPR lead? did you mean the patch in
attachment 69356 [details] [diff] [review], or something else? 

Comment 58

[FZiegler]

Actually macsockotpt.c was patched a second time (attachment 73325 [details] [diff] [review])
in the Feb 11-Mar 11 interval during which this bug appeared.

Comment 59

[FZiegler]

*** Bug 138448 has been marked as a duplicate of this bug. ***

Comment 60

[FZiegler]

*** Bug 157782 has been marked as a duplicate of this bug. ***

Comment 61

[FZiegler]

*** Bug 168125 has been marked as a duplicate of this bug. ***

Comment 62

[FZiegler]

*** Bug 161151 has been marked as a duplicate of this bug. ***

Comment 63

[[e-mail] (gone - use [e-mail] instead)]

I'm not seeing this with build 2002-09-12-08 on Mac OS 9.2.2.  The server type
was an IMAP 4 Netscape Messaging Server 4.15 (patch 4).

Comment 64

[John G. Myers]

This is MacOS X specific.  Changing the OS field.

Comment 65

[[e-mail] (gone - use [e-mail] instead)]

Note: I just tried on OS X with the latest 2002-09-12-08 build as well, same
setup as my prior comment, and no crash yet.  I'll keep an eye out, though.

Comment 66

[John G. Myers]

I believe we should handle the crash in bug 123026 and have this bug focus on
the hang.  Any objection to removing the crash from the summary/keywords?

Comment 67

[John G. Myers]

I believe I see the bug in NSPR.  CheckPollDescEndpoints() is writing over any
out_flags set by CheckPollDescMethods().  Thus input data buffered in the SSL
layer is never causing PR_Poll() to immediately wake.

Comment 68

[[e-mail] (gone - use [e-mail] instead)]

I agree with John Myers, so I'm removing references to crashing from this bug
report, as bug 123026 deals specifically with that issue.

Comment 69

[David :Bienvenu]

wtc or jgmyers, what are the chances of a fix for the nspr problem?

Comment 70

[John G. Myers]

A fix is attached to bug 168831.

Comment 71

[Simon Fraser [no longer active]]

I'm sorry, I haven't had a chance to test the fix yet. If someone else could get
an OS X build to QA with this fix, and have them test all the usual suspects
(FTP,  HTTPS, IMAP/SSL etc etc), that would be great.

Comment 72

[Paul Fischer]

If someone can point me to a Mac OSX binary with this fix I would be happy to test it.

Comment 73

[John G. Myers]

I don't have the relevant CFM Mac compiler, so I can't make a binary.  sfraser?
wtc? All we need is an NSPR20.shlb compiled with the proposed fix for bug 168831.

Comment 74

[Kai Engert (:KaiE:)]

I am able to reliably reproduce this bug on OS X, with both the 29/9 trunk build
and my own debug build.

After applying the patch from bug 168831, my debug build does no longer hang.

Comment 75

[FZiegler]

Alleluia! Vive John Myers!

Comment 76

[Kai Engert (:KaiE:)]

Meanwhile I have an optimized build and it works, too.
Let me know if somebody wants me to send the compiled NSPR shared library (and
how I should send it).

Comment 77

[Kai Engert (:KaiE:)]

I received requests from several people to provide my working build.

However, I don't have yet a tool to build a distributable package. I have
started a process to obtain a license for DropStuff, but I'm not sure how much
time it will take before I receive it.

I'm new to the Mac world.

I have just tried to use MacZip - does not work, because it does not add the
real targets for symbolic links

I have also tried to use the command line on Mac OS X to create a tar archive,
but that doesn't work either, because the resource forks (whether a file is
executable or not) is lost.

So unless somebody can suggest me a freeware tool that I can use, we'll have to
wait for my DropStuff license, or we should check in 168831.

Comment 78

[Simon Fraser [no longer active]]

Kaie: DropStuff works even when unregistered; just hit the 'Not Yet' button in
the initial dialog. It resolves aliases when stuffing.

Comment 79

[John G. Myers]

I suggest just attaching your NSPR20.shlb to bug 168831.  People can then test
by replacing that one file.

Comment 80

[Kai Engert (:KaiE:)]

DropStuff: Yes, my DropStuff 7 eval version can create archives, but it adds
aliases as well. My prefs don't show any option mentioning aliases?!? When I
create the archives, it's only 5 MB big, that can't be right.

Disk Copy: This just stores aliases as well.

Attaching shared library to the bug: Are you sure that will work? I already
uploaded that single file to a web server, and I was told this file doesn't
work, errors with shared libraries are shown.

However, I just have the idea that I could create a disk image file containing
that single shared library - that disk image file would probably carry the
shared library file attributes correctly. Let me try that.

But another question: When I download the single mac-os-x file from
ftp.mozilla.org, what I get is a disk image containing a single 33 MB Mozilla
file. How will you be able to replace the shared library in there? Or are you
downloading a different build?

Comment 81

[Kai Engert (:KaiE:)]

FYI: You can try to get the single unprocessed raw binary shared library file from:
  http://www.kuix.de/misc/bug168831/

But I was told it doesn't work.

I'll try to upload a disk image with that single file to the same page shortly.

Comment 82

[Kai Engert (:KaiE:)]

Ok, we now have a SIT archive at
  http://www.kuix.de/misc/bug168831/
containing a full Mozilla build.

While the standard version of DropStuff 7 does not have an option to expand
aliases, I found DropStuff lite version 6.5.1, which does allow to do that!

Comment 83

[bill]

I tried reading every 6-9K message I had in my mailboxes, and with this debug
build I no longer see the "hanging" that I do with the 29/09 (or any older)
build.  happy day!

Comment 84

[FZiegler]

> However, I just have the idea that I could create a disk image file containing
> that single shared library

(Hmmm, whose idea was this? ;-)

It works! The .dmg.gz is small enough (116 k) that you could probably attach it
to this bug. I stuck the resulting NSPR20.shlb in its proper place in build
2002-09-29-03's app bundle, chmodded it to 777 just in case (so far as I can
see, this may not even be necessary):

[localhost:Contents/MacOS/Essential Files] fz% ls -l N*
-rwxrwxrwx  1 fz  unknown  226588 Sep 30 00:30 NSPR20.shlb
-rwxrwxrwx  1 fz  unknown   17426 Sep 29 09:55 NSRuntime.shlb
-rwxrwxrwx  1 fz  unknown  441252 Sep 29 09:56 NSS3.shlb
-rwxrwxrwx  1 fz  unknown  169224 Sep 29 09:56 NSSckbi.shlb
-rwxrwxrwx  1 fz  unknown  254656 Sep 29 09:56 NSStdLib.shlb

...and now for the first time ever, a CFM build managed to download my test
message (comment 10) without a hitch. (And no crash, even if I hit `Stop' before
it's done.) Oh, joy! I'll keep browsing with this build, look out for any
regressions, and hope this can be checked in soon. Thanks Kai, thanks again John! 

Comment 85

[FZiegler]

Just a precision: it works to put NSPR20.shlb in place simply by drag & drop, in
the Finder.

Comment 86

[FZiegler]

With this fix, I also see a significant speed improvement in downloading headers
and messages from news://news.mozilla.org

Comment 87

[John G. Myers]

It should also help out https: with persistent connections.

Comment 88

[Stephan Budach]

Hey guys,

nice work! There is almost no difference between running IMAP with or wothout
SSL to our CommuniGatePro server. It works like a charm and I cannot wait for
this fix to show up in the regular builds.

Great work, people!

Thanks,
Stephan

Comment 89

[Jean-Francois Ducarroz]

Why this bug is reopen? does the problem has been fixed?

Comment 90

[John G. Myers]

The problem has not been fixed.  There is a (likely) fix attached to bug 168831
which has not been landed.

Comment 91

[Charles Rosendahl]

Resolving this bug as fixed, so that it may be verified along with bug 168831.

Comment 92

[FZiegler]

The patch definitely fixed this for me. Should I mark it verified?

(Simon, John G.: is it possible to answer comment 51 now?)

Comment 93

[John G. Myers]

Comment #51 did correctly identify the patch that introduced the bug.

Comment 94

[lchiang]

Is this bug fixed on the branch as well?  I don't see any branch "markings" in
the keyword.  Thanks.

Comment 95

[Kai Engert (:KaiE:)]

This bug number tracks the symptoms in the mail client, but the only problem was
in the lower level NSPR library.

The underlying problem is tracked in bug 168831 and has been fixed on trunk and
branch, and is marked as fixed1.0.2.

Adding fixed1.0.2 to this bug, too.

Comment 96

[laurel]

*** Bug 177761 has been marked as a duplicate of this bug. ***

Comment 97

[[e-mail] (gone - use [e-mail] instead)]

I've checked the following OSs/builds:

Mac OS X 10.2, Windows 2000, RedHat 8.0.

2002-10-31-10 trunk.

The IMAP V4 server I tested against is * OK dredd.mcom.com IMAP4 service
(Netscape Messaging Server 4.15 Patch 4 (built Dec  7 2000))

Per my own testcases and the results of both the reporter and other parties in
this bug, marking this Verified FIXED.

Comment 98

[Stephan Budach]

Excellent work! I have just tested the latest trunk against CommuniGatePro
Server (IMAPS) and either Mac OS 9.2.2 and Mac OS X 10.2. It´s lightening fast
and a pleasure to use.

My thanks to all the people who have contributed to this bugfix.

Comment 99

[bill]

I can't find a message that makes it hang anymore.  Great job everyone!

Comment 100

[[e-mail] (gone - use [e-mail] instead)]

Verified using today's branch 1.0.2 2002-10-31 commercial build with SSL enabled
(port 993) using IMAP V4 (server type noted in comment 97) with both Mac OS X
10.1.5 and Mac OS X 10.2.

Replacing fixed1.0.2 keyword with verified1.0.2 keyword.

Comment 101

[Ingo Kappler]

Thanks to all of you, who fixed it!

I'm new to bugzilla. Is there any way to get this fixed version of mozilla 1.0.2
now, or have I to wait until it becomes officially released?

Comment 102

[Kai Engert (:KaiE:)]

Mozilla 1.0.2 is not released. But you can use the latest build from the 1.0
branch. Get it from: ftp://ftp.mozilla.org/pub/mozilla/nightly/latest-1.0/

Comment 103

[Ingo Kappler]

Thanks again. The nightly build works on Mac OS 9.2 just fine.

Comment 104

[Paul Fischer]

Everything looks fixes so far. I can read all my mailboxes through IMAPS on
RedHat 7.2 and Mandrake 7.3 standard IMAP servers. Great work on fixing the bug.
I really appreciate it. I've been using Mac's built-in mail program under OSX
10.2, but it doesn't do some of the really cool stuff Moz Mail does for me.

I am puzzled though that this is in Moz 1.0.1 and not in Moz 1.2b. Did I miss
something about the naming conventions? Is Moz 1.0.1 the second "real release"
of Moz and Moz 1.2b sort of misnamed and should have been Moz 1.0.1.2 or
something? If someone could clear this up for me I would really appreciate it.

Comment 105

[Kai Engert (:KaiE:)]

Paul, the fix is not in 1.0.1, it is an older snapshot that did not yet have the
fix. Future milestones like 1.0.2 or 1.2 final will contain the fix.

There is no such thing as a "real release", like you refer to it.

1.0 was a milestone.
1.0.1 was a improved version of the 1.0 milestone
1.0.2 will be another milestone, based on the 1.0 sources, containing additional
bug fixes

1.2 is a future milestone, not yet published, which will contain a lot of
changes since 1.0
1.2b, which means 1.2 beta, is a preliminary test release of the 1.2 milestone.

Comment 106

[FZiegler]

*** Bug 152432 has been marked as a duplicate of this bug. ***