495666 - If mark Read/Unread, Add/Remove Tag etc. is executed while "Compact Folder" is running, marking/tagging is lost after end of compaction

Status: NEW

(MailNews Core :: Database, defect)

Description

[WADA:World Anti-bad-Duping Agency]

[Build ID]
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1pre) Gecko/20090527 Shredder/3.0b3pre

If "Mark Read/Unread"(update of X-Mozilla-Status:) is executed while "Compact Folder" is running, marking is lost after end of compaction.
I think it's same for tag, because "Add/Remove Tag" is update of X-Mozilla-Keys: header.
(Note-1:) I didn't check "Mark As Junk/Non-Junk" case.
(Note-2:) I don't know Tb 2's behaviour.

[Steps to produce]
1. Create big mail box which takes long to compact.
   Mark all mails as "Read". Shift+Delete for a mail.
2. Open 2 Tb windows. Open the mail folder at both Tb windows.
3. Window-1 : "Compact Folder" of the mail folder.
4. Window-2 : Select All, Mark As Unread => Unread mail count is shown.
5. After end of compaction, Unread status is lost.

(Note-3:)
If Tb trunk, error message of "other process is using folder" was displayed when "Shift+Delete" was executed while "compact folder" is running. So phenomenon of Bug 370409 couldn't be observed. I don't know Tb 2's behaviour.

Comment 1

[Kent James (:rkent)]

This clearly falls into my war on dataloss area, so I am assigning it to myself to keep it visible to me. But if someone feels inspired to fix it, you won't hurt my feelings.

Comment 2

[WADA:World Anti-bad-Duping Agency]

Bug 368768 seems "Mark As Junk/Non-Junk" case.

Comment 3

[WADA:World Anti-bad-Duping Agency]

"Add Tag" case was better to observe phenomenon. Tag of mails already copied to nstmp is lost.

Comment 4

[WADA:World Anti-bad-Duping Agency]

If "Add Tag" case, loss of tag data in X-Mozilla-Keys is also observed, even when tag is kept at thread pane.
  (1) mail_1 to mail_M:     (1-A) Tag at thread pane is lost.
                            (1-B) No added tag data in X-Mozilla-Keys: header  
  (2) mail_(M+1) to mail_N: (2-A) Tag at thread pane is kept.
                            (2-B) No added tag data in X-Mozilla-Keys: header
(1-B) & (2-B) is possibly cause of some status losses after ordinal compaction.

Comment 5

[Umang]

Can confirm the same happens on Linux. I share my profile between Windows and Ubuntu and this happens pretty often. It is pretty irritating, because the Unread messages tally gets horribly messed up (e.g I actually have 10 unread messages in Inbox, but the sidebar shows Inbox(3), etc) and I have to remember which messages were unread, and mark each of them read then remark them as unread.

If any other information is required, I will be glad to provide.

Thanks.

Umang

Comment 6

[Umang]

Also, if I move a file from Inbox to another folder in Local folders, the Inbox copy stays, and a copy is made to the other folder. If I don't realize this, then I move the mails again after the compacting is done and find two copies in the other folder, both unread.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5 and comment #6)

Umang, what is your auto-compact setting?  
> mail.prompt_purge_threshhold = true/false 
> mail.purge_threshhold = NNNN (threshold value in KB, default=100KB)
> mail.purge.ask = true/false
If you set prompt_purge_threshhold=true, set mail.purge.ask=true and restart Tb. You can cancel auto-compact by "Cancel" reply.
(I recommend you to put it in user.js, because it can be set to false at dialog.)
If small purge_threshhold value, dialog is displayed frequently and it's annoying. Increase the value.

(In reply to comment #6)

> if I move a file from Inbox to another folder in Local folders,
> the Inbox copy stays, and a copy is made to the other folder.

If compaction of move source folder is executed while bulk move between local mail folders is running, "delete from move source folder step of bulk move" fails. It's known phenomenon in some other bugs. But I don't know bug for this specific issue.
"bulk move" case is different from this bug (this bug is for status update case. bulk move contains "delete of mail".)
Umang, open separate bug for it, please.

Note:
If you set mail.purge.ask=true, you can always test with "Shift+Delete of some mails" and "manual compaction". If you create dummy POP3 account(no Global Inbox use), you can do "bulk move" easily by message filter of "Match all messages, Move to a local folder".

Comment 8

[Umang]

OK. Just a question before I post a new bug, so I'm sorry that I'm off-topic.

What is user.js? Where can I find it (not there in my profile folder) How do I edit it? Why can't I change from config editor? (haven't exactly fooled around with Tb before)

Anyways, so what you want me to do, is to try to move, copy, delete mails when compacting is going on and also check how often compacting happens by making Tb ask me before compacting? Then report a bug on errors while moving and deleting files when a folder is getting compacted. If yes, should I also explain about how read/unread doesn't change while compacting?

Also, here're my settings:

mail.prompt_purge_threshhold = true
mail.purge_threshhold = 200
mail.purge.ask = false

Thanks WADA,

Umang

Comment 9

[WADA:World Anti-bad-Duping Agency]

(Off-Topic)

> What is user.js? Where can I find it (snip) How do I edit it?
Google search for "thunderbird user.js" may help you.

> Why can't I change from config editor?
You can do it via Config Editor. However, if you check "Do this automatically ..." in dialog before auto-compact, mail.purge.ask is set to false. If mail.purge.ask=true is set by user.js, you can recover from mail.purge.ask=false by "Restart of Tb" only.

> so what you want me to do, (snip)

Simply (1) to reproduce problem, with { auto-compact=off } or { auto-compact=on + mail.purge.ask=true + "Cancel reply to dialog" }, with manual "Compact Folder", (2) to report separate bug.