Closed Bug 762169 Opened 12 years ago Closed 12 years ago

Always asking to compact folders starting version 12, continuing version 13

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 710056

People

(Reporter: dannyfox, Unassigned)

References

Details

(Keywords: regression) 

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20100101 Firefox/13.0
Build ID: 20120601045813

Steps to reproduce:

Upgraded to TB 12  when released.  Then every time I move a message to another folder, TB asks if I want to compact folders.  But once I did a "compact" manually (ie- on demand from the menu), then I haven't been asked (except as normal usage would require).  This happened on THREE machines (all Windows 7).

Today, I upgraded to TB 13.  First time I moved some messages (three, over 50mb total), it asked (OK, reasonable in this case).  But then the very next message I moved (few kb), it asked again.  I've just run a manual compact -- not sure yet if it will clear the problem again.


Actual results:

It seems compacting when asked doesn't change the flag to say it's been done, so it keeps asking.  Only when a manual compact is done will it clear the flag.


Expected results:

TB should compact only when needed (based on cumulative moves), not every time a message is moved.
(In reply to Dan Pernokis from comment #0)
See bug 750569.
Depends on: 750569
Thanks Hashem, I agree that 750569 describes THIS problem.

I ran a manual compact (as stated above) but now can tell you that the request to compact may STILL occur regardless -- which seems to be different (and worse) than the original TB 12 issue in Bug 750569.  But it seems to depend where the cursor is when the manual compact happens:  On INBOX, the problem will recur (or not be cleared), but doing it on LOCAL FOLDERS so far has cleared the problem again.
Keywords: regression
Summary: Always asking to compact folders → Always asking to compact folders starting version 12, continuing version 13
(In reply to Dan Pernokis from comment #2)
> I ran a manual compact (as stated above) but now can tell you that the
> request to compact may STILL occur regardless -- which seems to be different
> (and worse) than the original TB 12 issue in Bug 750569.  But it seems to
> depend where the cursor is when the manual compact happens:  On INBOX, the
> problem will recur (or not be cleared), but doing it on LOCAL FOLDERS so far
> has cleared the problem again.

So if Inbox is selected and you do a manual compact, then you still get compact prompts?  

Can you test this again, and check profile before compact and after, to see if time stamp of Inbox and Inbox.msf change?
(In reply to Wayne Mery - comment #3)

I recorded file info for INBOX and INBOX.MSF files, then ran a manual compact with cursor on the Inbox.  Took a minute (I have several mailboxes), but the size of INBOX decreased from 304,412kb to 255,153kb, timestamped right now.  INBOX.MSF was unchanged, timestamp unchanged too from earlier today (when most recent message came in).  I then moved your email message to my Mozilla mailbox, and TB did NOT ask for compact.  Nor did it ask on a second message move either.

This is really tricky, because operating sequence is critical.

Following each of several TB upgrades, starting with TB 12.0, I found the problem occurs and never resolves itself unless & until I run a manual compact with the cursor on LOCAL FOLDERS.  (I display "All" in view/folders.)  Sometimes I've had to close TB and relaunch it to make the fix stick, but not always.  However, once fixed -- as it is now -- the problem doesn't seem to recur UNTIL the next TB upgrade.  This was true starting with TB 12.0 and into 12.0.1, 13.0, and now 13.0.1.  That is, I can't make it not-work (ie- start prompting all the time) once fixed.  And this has happened identically on all three of my machines, starting with the 12.0 upgrade.
Roland, I think of all the compact bugs, you won't do much better than comment 4.  (but not saying all compacts are the same)
Component: General → Backend
Product: Thunderbird → MailNews Core
(In reply to Dan Pernokis from comment #4)
> (In reply to Wayne Mery - comment #3)
> 
> I recorded file info for INBOX and INBOX.MSF files, then ran a manual
> compact with cursor on the Inbox.  Took a minute (I have several mailboxes),
> but the size of INBOX decreased from 304,412kb to 255,153kb, timestamped
> right now.  INBOX.MSF was unchanged, timestamp unchanged too from earlier
> today (when most recent message came in).  I then moved your email message
> to my Mozilla mailbox, and TB did NOT ask for compact.  Nor did it ask on a
> second message move either.
> 
> This is really tricky, because operating sequence is critical.
> 
> Following each of several TB upgrades, starting with TB 12.0, I found the
> problem occurs and never resolves itself unless & until I run a manual
> compact with the cursor on LOCAL FOLDERS.  (I display "All" in
> view/folders.)  Sometimes I've had to close TB and relaunch it to make the
> fix stick, but not always.  However, once fixed -- as it is now -- the
> problem doesn't seem to recur UNTIL the next TB upgrade.  This was true
> starting with TB 12.0 and into 12.0.1, 13.0, and now 13.0.1.  That is, I
> can't make it not-work (ie- start prompting all the time) once fixed.  And
> this has happened identically on all three of my machines, starting with the
> 12.0 upgrade.

Dan: this is helpful; Some followup questions:
0. What is the setting of Tools | Options | Advanced | Network & Disk Space | "Compact folders when it will save over ..."? Checked or unchecked? If it is checked, then what is the value ? e.g. 20MB? 1MB?
1. are all your accounts POP or IMAP or a mixture of both?
2. What operating system? Windows 7? XP? Mac OS ?
3. What anti-virus if any ?
4. it would be really helpful if you could copy and paste the contents of Help | Troubleshooting Information (please make sure "Include account names" is unchecked because I don't want to know your email address and other personal info :-) !) and email it to me (rtanglao AT mozilla.com)
(Reply to Roland - Comment #6)

Recall that I have three machines (Tower, Laptop, and Netbook).

> 0. What is the setting of Tools | Options | Advanced | Network & Disk Space |
> "Compact folders when it will save over ..."? Checked or unchecked? If it is
> checked, then what is the value ? e.g. 20MB? 1MB?
- On Tower, CHECKED and save if over 60mb.
- On Laptop, CHECKED and save if over 20mb.
- On Netbook, don't have access to it right now, but I think CHECKED (I never unchecked it, so it would be the default).

> 1. are all your accounts POP or IMAP or a mixture of both?
For all three machines:
ALL are POP / SMTP accounts.
All but one are on Port 110 / no security.
One is Port 995 / SSL/TLS.
One (on Port 110) is inbound-only (POP, no SMTP).

> 2. What operating system? Windows 7? XP? Mac OS ?
WINDOWS 7 Pro/32-bit (+SP1) on two Dell machines
WINDOWS 7 Cheapie/32 (+SP1) on Acer netbook

> 3. What anti-virus if any ?
NORTON Internet Security 2012 (presently "Version: 19.7.1.5"), always up-to-the-minute current.

> 4. it would be really helpful if you could copy and paste the contents of Help
> | Troubleshooting Information (please make sure "Include account names" is
> unchecked because I don't want to know your email address and other personal
> info  !) and email it to me (rtanglao AT mozilla.com)
- Tower and Laptop info sent by private email.  Laptop has fewer accounts but were set up as identically as I could to their counterparts on Tower.
- Netbook - don't have access to it right now.  Has only a few accounts, but all set up like their counterparts on other machines.
So my emails have run fine without compacting for a few weeks.  Then last night, I received a 24mb email (huge attachment) which I moved from INBOX to elsewhere.  Immediately it asked if I wanted to compact now.  Clicked OK (sitting in INBOX).  Today on *two* occasions, as I moved something trivial from INBOX to elsewhere, TB asked to compact.  TB had been opened and closed a few times along the way, but not sure if it was closed right after compacting.  In any case, I had two compacting events, said OK to both with cursor in INBOX.  Then a short time later I got a third and said "Cancel".  (I think I was in SENT at this time.)  I went to LOCAL FOLDERS, manually did a compact, then closed and relaunched TB.  Moved two trivial items again from INBOX but (as I expected) it never asked to compact this time.

This behaviour is consistent with what I have seen & reported all along.  I've just not had it happen without an upgrade first.  Somehow, once TB senses that compact is needed -- whether by upgrade or by move -- if you do it then and there, the compacting will keep recurring.  Go to LOCAL and do it manually, and the problem gets fixed.  Then doesn't recur until (i) the next upgrade, or (ii) some automatic legitimate trigger happens (such as huge file moved or deleted).

Note:  In my own initial complaint (Comment Zero), I did get a prompt to compact following an upgrade when moving 50mb at once.  Whether the upgrade or the move triggered the compact, I can't say and don't know.  But doing the prompted compact in place (INBOX in that case) didn't solve anything.  Only going to LOCAL FOLDERS and closing/relaunching seemed to work.
hi dan: so to summarize: 

0. moving an email with a 24MB attachment from the inbox to a Local Folder triggered the problem but it might have nothing to do with attachments, it might have something to do with moving large amounts of emails

1. Was this on your laptop where the limit is 20MB
or
Was this on your Tower where the limit is 60MB?

2. If the email isn't confidential, it would be great if you could forward it to me
Hi Roland...

The incident happened on Tower with the 60mb threshold.

This:  "...but it might have nothing to do with attachments..." is not necessarily true -- but how does an email get large except by attachment or embedding.  I think the largeness of this one email would have naturally triggered a compact request about now anyway -- hadn't been compacted for awhile and there had been lots of 1-2mb files through in the past while -- it was only a matter of time when.  However, ONCE TRIGGERED and ONCE COMPACTED IN OTHER THAN INBOX, then it kept repeating every time I moved something else.

In a sentence:  It's not the moving of large amounts (numbers) of emails -- it's the moving of enough emails that the threshold is eventually reached. (One email will eventually be enough, but one BIG email will be enough much sooner!)

The Laptop (20mb) also has been asking occasionally, also after a period of no compacts -- and it would have seen much of the same email traffic (but not the 24mb file which I deleted from the server before it needlessly went elsewhere).  It hadn't asked for awhile, but again I think once triggered naturally (by enough 1-2mb files), then it would start repeating.  It asked again tonight, so I did a manual compact from LOCAL FOLDERS.  I'll monitor for awhile and let you know if anything weird happens.

I haven't used the Netbook for awhile -- and it will have several emails come in but not a  lot of big ones.  It may or may not pass its threshold, but I likely will manually compact anyway.

The large email is not confidential -- in fact it's a promotion by someone I know -- contents are a few lines of explanatory text and one 17mb ZIP file.  Unfortunately -- and yet another cruel twist in the continuing saga -- I deleted the attachment after saving it, just to keep the saved version of the email small.  (Sorry, didn't know this request was going to happen.)  But I can say that the delete was done BEFORE the move, I'm sure (thinking why enlarge the destination).  Then request for compact happened on the MOVE and not the DELETE.  (And so on a few more times as documented in previous comments.)
One more thing...
I've said that the upgrading triggers the sequence leading to compact requests, and now in my Comment #10 I say that moving enough files (one large or many small) to exceed the threshold also sets off the problem.

Well, I think it's been longer between upgrades this time, or in any case it's the first time the natural flow of emails has triggered compacting before an upgrade came along and did it.  Either upgrade or move will do it.  (See my Comment #8:  "Once TB senses that compact is needed...")
Good News: I just upgraded to TB 15.0 and it DID NOT ask me to compact my files!  I had compacted manually a few times yesterday under 14.0 (in one case with cursor at a specific folder, another case at LOCAL FOLDERS) and did so again today, deliberately & immediately before the upgrade (cursor at LOCAL FOLDERS, did not close TB prior to upgrade).

Right after the upgrade -- and watching very carefully -- I moved several K-sized files to various folders.  Usually the first move or delete asks if I want to compact, regardless of size -- but there were no prompts this time.

FYI Roland: On Tower with threshold still set at 60mb.  Will try my other machines later.
That's good news for you. But I am leery it will translate to success for others. Myself for example, using trunk, my prompts continue unabated.

Do you have a copy of your profile from when it still failed?
Further to Comments 12 & 13:

I upgraded my Laptop to TB 15.0 yesterday without compacting beforehand.  Good news, it has NOT asked to compact on any DELETEs or MOVEs.  (Threshold is 20mb as before.)

I notice the new TB is much changed.  Could it be the old story that just how & where the code loads that makes it work for some and not others?  Recall that the problem did come on suddenly (for me in 12, others in 13), and now just as suddenly it went away for me.
I believe this defect persists in Tbird 14.0. Brand-new install on new Win7 desktop (early August '12). Threshold is 40MB (which AFAIK I did not alter from the install). First compacting prompt dialog appeared yesterday and I said, go ahead & compact. Second compacting prompt issued just now upon deleting a message in Inbox.

Same symptoms as with instance of Tbird on an old machine. Maybe Tbird 15 will fix but from reading comments in this thread I wonder (for sure, as an outsider) if that defect has been put to rest.

Thanks for all continuing efforts to improve Tbird.
You can try installing a debugging extension from here: https://bugzilla.mozilla.org/attachment.cgi?id=657563 and see the discussion starting at bug 750569 comment 62 and above and report what you see.
Thanks :aceman. Installed extension. Allowed one automatic compaction. After a couple restarts & message deletions have not had another nag for compacting. So will let this ride for a few days & see what happens. Assume it's unlikely that Heisenberg is involved here, as in, the debugging extension changes the compaction prompt behavior...
Let's hope not, this extension should not change the behaviour, only shows the state of folders.

So watch what the "Expunged" numbers do and if they disappear (meaning 0) after a compact.
Well, I'm both baffled and chagrined. I have not been presented with "wanna compact?" dialog since allowing that one compaction on 2012-09-13 10:23:05 PDT. I also need to state that I got automatically upgraded to Tbird 15.0.

Report on debug info:
1. First restart after installing "Extra Folder Columns" extension, Inbox Expunged has value of 48491951. Drafts shows 860841
2. After allowing the prompted, auto compaction, Inbox Expunged, Drafts Expunged = null.
3. After restarting Tbird, Inbox Expunged = null.
4. Permanently deleted one message in Inbox; Inbox Expunged = 1437, and no prompt to Compact.
Dan, is your problem resolved in version 16?
Update via Help | About | Check for updates
Flags: needinfo?(dannyfox)
(In reply to Wayne Mery (:wsmwk) from comment #20)
> Dan, is your problem resolved in version 16?
> Update via Help | About | Check for updates

Hi Wayne,
I haven't seen the problem since TB15 appeared to solve it (see my comment #12 & comment #14), and I've been through updates to 15.0, 15.0.1, 16.0, and now 16.0.1 as of October 14th.  I've had two or maybe three requests in the past two months, all legitimate.  Glad you guys finally got this problem "outfoxed" -- good work!  Thanks to all.
Flags: needinfo?(dannyfox)
Glad to hear that, thanks.
Status: UNCONFIRMED → RESOLVED
Closed: 12 years ago
Resolution: --- → DUPLICATE
I don't think this is properly a duplicate of bug 710056, because Dan's problem went away without having used the patch.  So inclined to change this to WFM - does anyone disagree?

F. Ehrhardt, is the problem still gone for you?
Bug 710056 (losing column header settings) isn't the least bit familiar to me, and I was confused to find my name suddenly added to their distribution list.  (Although I have had that problem ever since using TB...)

I would say this bug is not a duplicate, because it showed up independently at a specific time and went away again -- totally for me, only maybe for some.
Yeah, bug 710056s title is about a different problem, but the fix should fix your problem too, that's why the duping. See also bug 750569 comment 78.
Hi, to answer Wayne's question, __yes__ it appears that I no longer have the problem. Now I get compaction requests about once a week. Thanks for checking.
You need to log in before you can comment on or make changes to this bug.