Thunderbird 3.0.3 on OSX uses too much network bandwidth

RESOLVED INCOMPLETE

Status

MailNews Core
Networking
--
critical
RESOLVED INCOMPLETE
8 years ago
8 years ago

People

(Reporter: Cliff Wells, Unassigned)

Tracking

({perf})

1.9.1 Branch
x86_64
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: closeme 2010-12-15 [needs v3.1 or trunk test][needs protocol log])

Attachments

(1 attachment)

(Reporter)

Description

8 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.366.2 Safari/533.4
Build Identifier: 3.0.3

Single user of TB 3.0.3 consumes 10-20Mbit of bandwidth for long periods of time for no apparent reason.

Disabled all sync features
Disabled Global Search Index
Disabled Spotlight

The user's IMAP mailbox size is ~3.3GB, but the user transfered ~50GB over several hours before the problem was noticed.


Reproducible: Always

Steps to Reproduce:
1. Install TB 3.0.3 on OSX
2. Point it to an IMAP server
3. Pay your huge bandwidth bill



I had initially assumed that this was related to sync, but disabling that didn't fix it.  Then I thought maybe search indexes, but disabling those didn't fix it either.

For the time being, we are banning TB 3.0 from our organization, including setting up a mail proxy to detect TB 3.0 and prevent it from accessing our IMAP servers.   We simply can't afford to let our users run it.   If a single user can easily move over 50GB per day out of a 3.3GB mailbox, then we'd very quickly no longer be able to afford hosting.

I'd equate "we lost our hosting" to "causes you to lose data", hence the "Critical" severity below.
(Reporter)

Comment 1

8 years ago
Created attachment 437420 [details]
Bandwidth utilization of single user, note *normal* utilizaton near the right side where TB was terminated.
(Reporter)

Updated

8 years ago
Attachment #437420 - Attachment description: Bandwidth utilization of single user → Bandwidth utilization of single user, note *normal* utilizaton near the right side where TB was terminated.
(Reporter)

Comment 2

8 years ago
I should also describe the IMAP server:

CentOS 5.4 (Linux mail 2.6.18-2-pve #1 SMP Mon Feb 1 10:45:26 CET 2010 i686 athlon i386 GNU/Linux)

courier-authlib-0.59.3-10
courier-authlib-pgsql-0.59.3-10
courier-imap-4.3.1-1
courier-authlib-devel-0.59.3-10
Cliff :
Can you confirm that mail.server.default.autosync_offline_stores is false ?
Are you using Lightning ?


David what would it be helpful for log here in order to figure out why we are consuming so much bandwidth ?
Component: General → Networking
Product: Thunderbird → MailNews Core
QA Contact: general → networking
Version: unspecified → 1.9.1 Branch

Comment 4

8 years ago
an imap protocol log would help, once he verifies that he did indeed use the config editor to turn off autosync.
(Reporter)

Comment 5

8 years ago
I've already moved that user back to TB 2.0 (he's head of that company and I can't use him for testing =).   I don't have a Mac myself, but I'll see if I can hijack another one to get a better report.

Interesting data point: I discovered another Mac user in the building using TB 3.0.3 without this problem.   The only differences I am able to discern are 

User with issue:
1) OSX 10.6
2) Turned on sync features, then turned them off

User without issue:
1) OSX 10.5
2) Never turned on sync features

This (rather unscientifically) leads me to think it may be an issue with 10.6 *or* somehow, once you've turned sync on, it doesn't turn off.

Comment 6

8 years ago
This is issue is highly unlikely to be mac-specific, or 10.6-specific. toggling mail.server.default.autosync_offline_stores to false is very likely to fix this. I think it's more likely that turning off sync doesn't really turn it off, in the sense that folders will still be configured for offline use and will still be autosynced unless the above pref is turned off. 

Bryan, in all the focus on the migration assistant, I wonder if we've lost track of the fact that the account settings ui should probably turn off autosync on that server, if it doesn't already.
(Reporter)

Comment 7

8 years ago
Ok, I've reinstalled 3.0.4 onto the original Mac with the issue.  I've manually set mail.server.default.autosync_offline_stores to false.  The problem seems resolved.

Of interest is the fact that I checked the other Mac and it had autosync_offline_stores set to true, but didn't exhibit the same behavior.   It seems you must select some folders for offline sync at some point to get the issue.

Also, there's still the question why sync would move 50GB out of a 3.3GB mailbox... I'm guessing some bad blood between TB and Courier =)  should I open a second report on this?   I'd actually be happy to provide IMAP accounts for testing against.

Comment 8

8 years ago
ah, ok, we have a separate issue on the mac with offline stores > 2GB. This was fixed in 3.0.3, iirc, to handle offline stores up to 4GB. In 3.1, we handle arbitrarily large offline stores. Essentially, if an offline store got over the limit, we would try to download the messages multiple times, because we thought the message was corrupted in the offline store.
(In reply to comment #6)
> Bryan, in all the focus on the migration assistant, I wonder if we've lost
> track of the fact that the account settings ui should probably turn off
> autosync on that server, if it doesn't already.

I'm pretty sure we do that now.  I've lost the bug but I believe we were changing the autosync default to be off such that the migration assistant could turn it on only when a person is looking at the page and choosing to sync all.

Of course this issue seems to be completely separate from that.
(Reporter)

Comment 10

8 years ago
Okay, I think I understand... it can't find a mail that should be in the store (because there wasn't room for it) and keeps trying to add it to the store.  Ouch.

Is the 3.1 release slated to be pushed at Mac users (i.e. actively prompt to
upgrade vs hoping they notice)?   

I sense this issue is going to rear its head repeatedly over the next couple of
months (I host mail for lots of companies and individuals) and I'm concerned
that the current solution boils down to putting out fires until 3.0 is supplanted on Mac.

Comment 11

8 years ago
unfortunately, in bugzilla, company pres ^=> critical

(In reply to comment #10)
> Okay, I think I understand... it can't find a mail that should be in the store
> (because there wasn't room for it) and keeps trying to add it to the store. 
> Ouch.

that would be key.  I can't speak to the privacy issues, but perhaps in future, before changing a user back to old behavior it will be useful to get a copy of the user's profile
Severity: critical → major
Keywords: perf
Summary: Thunderbird 3.0.3 on OSX uses too much bandwidth → Thunderbird 3.0.3 on OSX uses too much network bandwidth
(Reporter)

Comment 12

8 years ago
(In reply to comment #11)
> unfortunately, in bugzilla, company pres ^=> critical

Who said it did?    I stated I could not use the company president's laptop to test.   I marked this bug as "Critical" because it can cause serious financial loss to users.   Somehow you are conflating these two.

Had I not noticed the sudden jump in this company's bandwidth utilization within the first few hours, they could have easily racked up a bill for well over $3000 by the end of the month because of a single user.   Further, I would have undoubtedly lost them as a client.  I consider incurring serious financial consequences because some user checked a checkbox to be a "critical" bug.

> (In reply to comment #10)
> > Okay, I think I understand... it can't find a mail that should be in the store
> > (because there wasn't room for it) and keeps trying to add it to the store. 
> > Ouch.
> 
> that would be key.  I can't speak to the privacy issues, but perhaps in future,
> before changing a user back to old behavior it will be useful to get a copy of
> the user's profile

That would not be a good solution for anyone.   Profiles contain enough information to access a private mail account which in turn undoubtedly contains other private information.   I wouldn't do it with my personal email, let alone a corporate account.
(Reporter)

Updated

8 years ago
Severity: major → critical

Comment 13

8 years ago
(In reply to comment #10)
> Okay, I think I understand... it can't find a mail that should be in the store
> (because there wasn't room for it) and keeps trying to add it to the store. 
> Ouch.
> 
> Is the 3.1 release slated to be pushed at Mac users (i.e. actively prompt to
> upgrade vs hoping they notice)?   
> 
> I sense this issue is going to rear its head repeatedly over the next couple of
> months (I host mail for lots of companies and individuals) and I'm concerned
> that the current solution boils down to putting out fires until 3.0 is
> supplanted on Mac.

Cliff, 3.1 beta 2 is available at http://www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/

Also, what type of network connection/speed/isp was being used?
(Reporter)

Comment 14

8 years ago
Hi Wayne,

I'll have the clients I can control try out the beta.

The client under discussion has a Comcast business class connection (22Mbit down, 5Mbit up, static IP), the IMAP server is on a pair of DS3's (redundant 45Mbit) in a colocation facility.

Updated

8 years ago
Whiteboard: closeme 2010-12-15 [needs v3.1 or trunk test][needs protocol log]
RESOLVED INCOMPLETE due to lack of response to previous question. If you feel this change was made in error, please respond to this bug with your reasons why.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.