Closed Bug 179504 Opened 22 years ago Closed 19 years ago

mark messages as non-junk hangs system caused by server + profile cache connection limit

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: Biesinger, Assigned: Bienvenu)

References

(Blocks 1 open bug)

Details

on my cvs build from the weekend (linux), marking several hundreds of mails as
non-junk hangs the entire system for some seconds.

I.e. I can't switch to another application, and can't do anything in Mozilla.
All I can do is move my mouse cursor.
looks like bug 135000 ...

It's certainly related; not sure whether it's exactly the same or not.
Status: NEW → ASSIGNED
I'm experiencing something similar on a nightly.  I'm operating on a IMAP mail
account, and "Mark Selected Messages As Junk" seems to hang Mozilla for a short
while.  Then it becomes usable, at least the non Mail components.  When I click
on a mail in the folder view, the message body does not get loaded into the
preview pane.  However, the title updates with the subject of the message and
the Mozilla name and Build ID tag.  Switching into another folder lets me
retrieve message bodies into the preview pane, but switching back to the
original folder results in an hourglass icon when hovering over the list.

I'm on:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3a) Gecko/20021118
(So this isn't just a Linux problem; if it's the same bug)
Sorry for missing my homework, it seems that comment #7 from bug 179162 mentions
the cause of the behavior I've seen.  The messages are being marked, there's
just not any UI to show the progress.  And I cannot retrieve other message
bodies in the same folder because that's a limitation of The Way Things Are Done.

Well, I hope this helps other people find the right bug.

I can imagine scoring local mail might lock up the ui a bit because we're
reading a lot of messages from disk - for imap, we should be bound by the
network and use less of the cpu. For imap, I checked in a change before
yesterday's build so that we'll chain the imap fetch requests so you should be
able to read other imap messages while the scoring is going on.
*** Bug 196211 has been marked as a duplicate of this bug. ***
Hi, is this bug still active, or is the performance problem being tracked on bug
179162 as well now?  It often takes as much as 20sec of intense disk IO on my
otherwise reasonably fast laptop to mark one single message as junk, during
which time all of Mozilla is locked up (mail and browser), and other apps are
obviously suffering if they too need to access the disk.  Then other times it
will take only a second or two.  Why are some marks so much more intensive than
others?

Regards, Ian
(In reply to comment #3)
> I'm experiencing something similar on a nightly.  I'm operating on a IMAP mail
> account, and "Mark Selected Messages As Junk" seems to hang Mozilla for a short
> while.  Then it becomes usable, at least the non Mail components.  When I click
> on a mail in the folder view, the message body does not get loaded into the
> preview pane.  

Confirmed. With Junk Controls on, Mozilla suddenly freezes when a mail is marked
as spam (by me or by Mozilla). Can't load messages, can't load folders. It takes
about two minutes to unfreeze.

With the option 'Move messages determined to be junk to Junk folder' switched
off, things work fine again - until I try to move a message to the Junk folder.
Same reaction: Mozilla freezes.

Weird thing is that I installed Mozilla 1.6 more than a weeks ago. This bug
showed its face only today.

System: Redhat 9, Mozilla 1.6
karin, are you using IMAP or POP3? The first thing I would look at is the size
of your training.dat file (it's in your user profile directory). If it's huge,
that might account for a freeze - there's a separate bug about limiting the
growth of this file.
(In reply to comment #9)
> karin, are you using IMAP or POP3? The first thing I would look at is the size
> of your training.dat file (it's in your user profile directory). If it's huge,
> that might account for a freeze - there's a separate bug about limiting the
> growth of this file.

IMAP. I don't think that my .dat file is particularly big (what's a normal size?
Mine is 270062k). But today I worked elsewhere and noticed something similar
(didn't have the time to fully investigate). That was on a machine that had a
fresh Fedora and Mozilla 1.6 install: the whole set-up was just a week old.

So I'd be prone to discard that expalnation.

Note: my Mozialla freezes when I move as little as _one_ mail to the Juk
forlder. So it is not the lack of a nice UI to mark the progress of moving: I
simply wouldn't call it progress :)


270062 Kb would be 270MB and would account for very large delay, not to mention
a rather staggering amount of memory use, so I'm hoping that number is off by a
couple orders of magnitude :-) . A normal size would be more on the order of a
few 100K. Mine is 2.3MB, but I've been marking things as junk for many months.

Training.dat gets read in the first time either a message is analyzed to see if
it's junk, or a message is marked as junk. It's written out every time you mark
 messages as junk or not junk, so having a huge training set would definitely
cause a freeze, since it's written out synchronously.
(In reply to comment #11)
> 270062 Kb would be 270MB and would account for very large delay, 

Eeeks, sorry: 270062k, 2,7MB. Seems doable. The other computer that I saw this
happening on cannot have yet acquired a big training.dat: it's freshly installed.


training.dat on the other computer discussed in comment #12 is 370174 bytes. 
And on Karin's machine it is 270062 bytes and nothing else. 
Solved. 

This is what happened re comment #12: In mozila mail, seven or eight different
IMAP accounts are configured (in the same profile) against the same IMAP server
running courier. Each of these accounts had the default of caching max 5
connections. Two different machines behind a firewall were using this
configuration. The server was configured to accept max 10 connections per IP.
Thus, as people moved from account to account and from mailbox to mailbox,
mozilla kept caching more and more connections, until it hit the limi of the
server. After that, mozilla would be unable to open any more mailboxes on either
machine until some of the previously cached connections would get dropped. 

Re comment #9, things are slightly different, but the problem is the same. One
IMAP account is configured caching the default of max 5 connections against a
courier server using the default MAXPERIP=5. That works fine, until a message
has to be moved to the junk folder. Under these circumstances, the connection to
Junk will always be the sixth one to be opened, running against the server's
limit every time and freezing mozilla until one of the previous connections gets
dropped. 

Of course, the problem in coment #12 was much worse, because i was caused by two
diferent machines working on the same server from behind the same IP, while
mozilla on the one machine can't drop superfluous connections on the other. 

This problem should be very common in corporate environments where role accounts
are opened at the same time by several different people, so it might merit some
attention. 

Proposed solution: If mozilla runs into "connection refused" from an IMAP server
to which it is already connected, it should pre-emptvely drop some unused cached
connections. 

that's great that you've tracked this down. However, I don't know why that
marking a message as junk should bypass the normal connection cache limit. Can I
get an imap protocol log showing this?

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

Connection refused is not an error that the imap code actually receives, afaik -
I think it's turned into something more generic by the time the imap code gets
it, but I'll look.
Assignee: dmose → bienvenu
Status: ASSIGNED → NEW
Component: Filters → Networking: IMAP
Product: MailNews → Core
(In reply to comment #16)

While looking for something else, I stumbled on this today. I'm almost two years
late, but I might reply anyway.

I just tested with Thunderbird 1.0.7 caching 5 connections against a courier
0.50 on MAXPERIP=5. No problems whatsoever. Having moved around and opened more
than five folders, junk is still moved to the junk folder without hangs.

My guess is that junk filtering somehow bypassed the logic in Mozilla mail which
said "if I am already caching my max and the user tries to open yet another
folder, I must release an old connection before making a new one". Either by
design or by accident this is not the case in modern-day Thunderbird and all
works well. 

I don't have Mozilla installed. If you think it's worth the trouble testing it,
I could; otherwise I guess this bug can be closed with FIXED_ITSELF ;)
Thanks.  Marking INVALID
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → INVALID
Summary: mark messages as non-junk hangs system → mark messages as non-junk hangs system caused by server + profile cache connection limit
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.