Closed Bug 420115 Opened 16 years ago Closed 15 years ago

compact folder doesn't seem to take for imap folders marked for offline use

Categories

(Thunderbird :: Mail Window Front End, defect)

Product:
Thunderbird
Component:
Mail Window Front End
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 3.0b1

People

(Reporter: mkmelin, Assigned: Bienvenu)

References

Details

Attachments

(1 file)

proposed fix
1.47 KB, patch
standard8
: review+
standard8
: superreview+
Details | Diff | Splinter Review 
Since about a week back on trunk there is something odd with Compact folder. I use IMAP and mark as deleted, so that's why its visible. 

When I right-click a folder and Compact, "Downloading message..." appears in the status bar, but the deleted messages aren't removed from folder. Or it might be that they are, but the view doesn't refresh properly. At least sometimes, going to another folder and coming back, the messages are gone as they should be.

Anyone else see this?
Yes, I've been noticing the same thing recently.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a2pre) Gecko/20080814031842 Shredder/3.0b1pre
Flags: blocking-thunderbird3?
Ah this reminds me, this isn't a regression after all - i saw it with 2.0 too. It only happens for folders marked for offline use. (And possibly only with more than one such set up.)
Keywords: regression
OS: Linux → All
Hardware: PC → All
Summary: compact folder doesn't seem to take → compact folder doesn't seem to take for imap folders marked for offline use
Tested with Tb trunk 2008/8/09 build.
(0) IDLE support off(because Gmail IMAP is used)
    Delete model : Just mark it as deleted
    Offline-use=yes for a folder 
(1) delete 10 mails, compact folder => Expunge was not issued
(2) delete 60 mails, compact folder => Expunge was issued.

It looks to be design since initial. (like "don't issue expunge till 20 deletes")

I've been experiencing problems with Shredder 3.0a3 too. I don't use offline support, so haven't explicitly set up and folders for that. Compact (either via right-clicking on the folder, or the new toolbar button (finally!)), sometimes works, but mostly doesn't. When it doesn't work, it appears that the EXPUNGE is being sent, but the messages don't disappear from the folder view. I just turned off IMAP IDLE to see if that makes a difference...
IMAP IDLE doesn't seem to make any difference...
Given that the default is now for imap folders to be marked for off-line use when created, then I think we should try and fix this bug. Initially targeting at b2 we can move it forward if necessary or someone finds the fix before then.
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Target Milestone: --- → Thunderbird 3.0b2
mark as deleted model and I can compact folder just fine.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081028 Shredder/3.0b1pre
(In reply to comment #8)
> mark as deleted model and I can compact folder just fine.

Do you have *offline* set for that inbox? Do you have *multiple* IMAP accounts?
I have enabled - Keep messages for this account on this computer. And I have multiply accounts. My problem is sometimes it doesn't compact Inbox, but do this anyway on exit because of checkbox.
FWIW, I found that in my case, the EXPUNGE was happening, but it seemed the local index was not getting updated. In order to make the deleted messages "go away", I had to "rebuild index" from the folder (right-click) properties. I got tired of this, and went back to Thunderbird 2.x
Nikolay, this is workaround, not solution.
Look at bug https://bugzilla.mozilla.org/show_bug.cgi?id=462054 - TB3 just ignores EXPUNGE replies from server sometimes.
Iain> FWIW, I found that in my case, the EXPUNGE was happening, but it seemed the
local index was not getting updated.

Right! In my case (with two different servers on different platforms) also.

Iain> I got tired of this, and went back to Thunderbird 2.x
Me too :(
While a see it a lot, for me it's not reproducible at will. 

I wonder if the "Downloading message..." message that appears is somehow a key here. When i'm compacting, why would a message have to get downloaded.
I also seen during EXPUNGE, but according to log, it just a FETCH command after the EXPUNGE command (replies to which was ignored).
download message, or download headers? Do you have auto download of imap messages turned on? It may be that doing the expunge causes us to issue a select, which triggers the header download, which triggers the message download. I don't know why that would break expunge, but I can look into it.
No. Expunge often causes you to issue a FETCH..flags (attemt to find new messages):
...
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * 29 EXPUNGE♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=14 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * 29 EXPUNGE♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=13 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * 28 EXISTS♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=12 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * 0 RECENT♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=25 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: 22 OK EXPUNGE completed♪
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:SendData: 23 getquotaroot "INBOX"♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=28 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" "ROOT"♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=16 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: * QUOTA "ROOT"♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=24 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: 23 OK GETQUOTAROOT Ok.♪
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:SendData: 24 UID fetch 12619:* (FLAGS)♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=24 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: 24 OK FETCH completed.♪
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:SendData: 25 IDLE♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=22 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: + entering idle mode♪

or sometimes NOOP:

5548[3e2e810]: ReadNextLine [stream=5dbc4f8 nb=12 needmore=0]
5548[3e2e810]: 6f02e80:imap.enet.ru:S-INBOX.Drafts:CreateNewLineFromSocket: * 3 EXISTS♪
5548[3e2e810]: ReadNextLine [stream=5dbc4f8 nb=12 needmore=0]
5548[3e2e810]: 6f02e80:imap.enet.ru:S-INBOX.Drafts:CreateNewLineFromSocket: * 0 RECENT♪
5548[3e2e810]: ReadNextLine [stream=5dbc4f8 nb=24 needmore=0]
5548[3e2e810]: 6f02e80:imap.enet.ru:S-INBOX.Drafts:CreateNewLineFromSocket: 7 OK EXPUNGE completed♪
5548[3e2e810]: 6f02e80:imap.enet.ru:S-INBOX.Drafts:SendData: 8 IDLE♪
5548[3e2e810]: ReadNextLine [stream=5dbc4f8 nb=22 needmore=0]
5548[3e2e810]: 6f02e80:imap.enet.ru:S-INBOX.Drafts:CreateNewLineFromSocket: + entering idle mode♪
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:SendData: DONE♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=22 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: 25 OK IDLE completed♪
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:ProcessCurrentURL: entering
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:ProcessCurrentURL:imap://cherezov@imap.enet.ru:144/select%3E.INBOX:  = currentUrl
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:SendData: 26 noop♪
5136[3efbed0]: ReadNextLine [stream=61c6a50 nb=22 needmore=0]
5136[3efbed0]: 3a7c780:imap.enet.ru:S-INBOX:CreateNewLineFromSocket: 26 OK NOOP completed♪

or sometimes FETCH of next message (when RECENT reply received also?). But expunged messages not deleted from local index not depending of exact next command.
(In reply to comment #16)
The status bar says message. I have two folders set up for offline use, both set to check for new messages (but no special auto-download).
sorry, configuring it for offline use means you get autodownload.
I'll look at this - it's a pretty painful regression for imap delete model users.
Component: Message Compose Window → Mail Window Front End
QA Contact: message-compose → front-end
Target Milestone: Thunderbird 3.0b2 → Thunderbird 3.0b1
Assignee: nobody → bienvenu
the issue here is that "compact this folder" for imap kicks off a compact of the offline store, and then immediately issues an expunge. The offline store compact locks the folder such that we can't deal with the expunge results - for the non imap delete model, this isn't an issue since we've already removed the headers from the local db.

I suspect that compacting the offline store every time the user does an expunge is not that desirable anyway, since offline store compact gets more and more expensive as the folder gets bigger. This is better handled with the background compact stuff that Emre will be working on, so I think I'm just going to remove the offline store compact.

If you do a file | compact all folders, we will compact the offline stores as well as expunge the online folder.
Attached patch proposed fix — — Splinter Review 
don't compact the offline store if the user picks the context menu item "compact this folder".

offline stores will still be compacted by compact all folders, and semi-automatic "compact folders when it will save more than XX kb" settings.
Attachment #349210 - Flags: superreview?(bugzilla)
Attachment #349210 - Flags: review?(bugzilla)
Status: NEW → ASSIGNED
Whiteboard: has patch for review
(In reply to comment #21)
> I suspect that compacting the offline store every time the user does an expunge
> is not that desirable anyway, since offline store compact gets more and more
> expensive as the folder gets bigger. This is better handled with the background
> compact stuff that Emre will be working on, so I think I'm just going to remove
> the offline store compact.

Does this mean that the "Compact" button will no longer work? If so, I would strongly disagree with that decision. If the user goes through the effort of adding that button and then decides to use tht button, it is because he *wants* to perform a manual compact *now*, whether the offline store compact is expensive or not. 

If it is necessary to do an offline store compact to make the marked-as-deleted messages go away, then that is what should happen. If the *user* decides that takes too long, then the *user* can decide to stop using the compact button.

Of course, there is a high chance that I'm not understanding your tech-speak. :-\
It means compact will just do an imap expunge on the server, which is what I believe people generally expect.
(In reply to comment #24)
> It means compact will just do an imap expunge on the server, which is what I
> believe people generally expect.

I don't really care about expunging at the server (that should be a given). I expect the marked-as-deleted messages to disappear from my messages pane (a be permanently deleted everywhere).

It's the main reason I use m-a-d: I mark-as-deleted messages as I read them or that are uninteresting. Then I visually scan all m-a-d message to see if I've deleted one accidentally (if so, I "delete" again to un-delete). Then I press "Compact" to make them disappear and free up my messages pane to focus on the remaining messages.
we will only remove them from the view if we've expunged them from the server, so you do care :-)
OK. So will the marked-as-deleted message disappear from the message pane after we press the Compact button for IMAP accounts marked for offline use after your fix from comment #22, even though you will not be compacting the "offline store"?
yes - the offline store does not affect what gets displayed at all - it's like the memory cache; it's just used to speed up message display.
Attachment #349210 - Flags: superreview?(bugzilla)
Attachment #349210 - Flags: superreview+
Attachment #349210 - Flags: review?(bugzilla)
Attachment #349210 - Flags: review+
Comment on attachment 349210 [details] [diff] [review]
proposed fix

This looks reasonable. I think we may want to add a comment as to why we don't compact the offline store here, in case someone decides to try and put it back in later.
fix checked in, with comment added.
Status: ASSIGNED → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Whiteboard: has patch for review
You need to log in before you can comment on or make changes to this bug.