550286 - Columns to display is lost if rebuild index/repair folder is used on an IMAP folder

Status: VERIFIED FIXED

(Thunderbird :: Folder and Message Lists, defect)

Description

[Kevin A. McGrail]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.18) Gecko/2010020220 Firefox/3.0.18 GTB6 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3

I have selected "Received" and "Recipient" as column headers to display.

They work as expected.

However, right clicking on a folder, going to properties and selecting rebuild index resets the display columns to default.

Reproducible: Always

Steps to Reproduce:
1.Click icon above vertical scroll
2.Change folder display columns
3.right click on properties for inbox and rebuild index 
4.watch columns revert to default.
Actual Results:  
Columns revert to default

Expected Results:  
I would have expected columns to remain as configured.

Comment 1

[Matt]

Confirmed. It also is a regression.

Shorter steps to reproduce:
1. "Inbox" columns are default
2. Using column picker, remove "Attachments"
3. Click "Rebuild Index" inside "Inbox" "Properties"

Result:
"Attachments" column returns

Builds tested below.

Works:
version 2.0.0.24pre (20091201)

Works:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1b3pre) Gecko/20090101 Shredder/3.0b2pre

Broken:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a3pre) Gecko/20100304 Shredder/3.2a1pre

Comment 2

[Matt]

Regression range:

Works:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a1pre) Gecko/20091103 Shredder/3.1a1pre
2009-11-03-03-comm-central-trunk

Broken:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a1pre) Gecko/20091104 Shredder/3.1a1pre
2009-11-04-03-comm-central-trunk

http://hg.mozilla.org/comm-central/pushloghtml?startdate=2009-11-03+02%3A00%3A00&enddate=2009-11-04+04%3A00%3A00

Comment 3

[Matt]

Candidate bug 510643.

Can you look at this please David Bienvenu?

Comment 4

[Matt]

Reproduced on linux. Setting "Platform" to "All".

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100216 Thunderbird/3.0.2

Comment 5

[Matt]

I thought all folders in Thunderbird 3 were created with the same default set of columns.

The default set of columns in a subfolder appears to be inherited from whatever the parent folder had at the time of the subfolders creation. And from the parent, not from the root parent folder (for deeply nested folders).

Comment 6

[Alan Batie]

Does tbird rebuild indexes when it starts up?  All my column settings disappear when I restart it (and I have dozens of active folders)...

Comment 10

[[e-mail]]

As mentioned for bug 682684, this problem is due to the transient nature of MSF files which are deleted/recreated when folders are compacted and on various other events which I have not been able to categorise. Message pane preferences should be saved in a more robust structure and the program defaults should be available for edit which would actually solve the issue for many. The Mnenhy addon would help but is unavailable for version 6

Comment 11

[Konstantin Stefanov]

I have the same on Earlybird 13.0a2, but only when repairing INBOX folder, other folders are not affected.

Comment 12

[David :Bienvenu]

Is it possible that the db is not getting committed after changing the thread pane columns? I'll look at the code that persists thread pane columns.

Comment 13

[David :Bienvenu]

Attachment: proposed fix [checked in in comment 26]

kSmallCommit doesn't do anything. This was the only caller of it. I'll file a bug to get rid of it.

Comment 14

[David :Bienvenu]

ugh, this is more broken than I thought. We're not getting the contents of the old imap db in the repair folder case, from what I can tell, so lots of things would be broken. I'll have to poke around a bit.

Comment 15

[Mike Dallwitz]

Bug 682624 (not bug 682684, as stated by keithnight above) has been marked as a duplicate of this bug. However, I doubt that it is, as for me the bug only appeared recently, I think with Thunderbird 6 (2011). However, I think that Bug 682624 may be a duplicate of Bug 710056.

I have had the problem in TB for several months - I think from Version 6. It always occurs when all the following conditions are satisfied:

(1) The folder has deleted items; (2) the folder is compacted; (3) the number of remaining items in the folder is 1 or 2; (4) Thunderbird is restarted.

The column headings in that folder after the restart are the ones from Inbox, or, if the folder itself was Inbox, some default settings.

This explains why the problem (for me) seemed to occur at random, and usually with Inbox. I compact Inbox every evening before backing up and shutting down, and the number of items it contains varies between 0 and about 10. I've never observed the problem when the number of remaining items is 0 or more than 4. It /sometimes/ occurs with 3 or 4 items.

I agree with keithknight that "Message pane preferences ... should be available for edit which would actually solve the issue for many". It would solve the problem for me, in that the bug would then set the columns to acceptable values. As it is, I have to manually change them every day.

Comment 16

[Wayne Mery (:wsmwk)]

(In reply to Mike Dallwitz from comment #15)
> Bug 682624 (not bug 682684, as stated by keithnight above) has been marked
> as a duplicate of this bug. However, I doubt that it is, as for me the bug
> only appeared recently, I think with Thunderbird 6 (2011). However, I think
> that Bug 682624 may be a duplicate of Bug 710056.

Mike, thanks for that research. bug 710056 is a relatively recent phenomenon - version 10. So I would think bug 682624 is a duplicate of bug 540857 - version 5. Do you agree? 

(note: some people skipped version 5, and would have first experienced the bug 540857 issue in version 6)

Comment 17

[Alexander Kogan]

same here...

Comment 18

[:aceman]

Bug 710056 is marked fixed now, can you all retry your tests to reproduce this?

Comment 19

[:aceman]

I mean with TB18 or TB17.

Comment 20

[:aceman]

Comment on attachment 616664 [details] [diff] [review]
proposed fix [checked in in comment 26]

What's up with this patch? It may have landed because bug 747102 didn't do this same change. But the is no kSmallCommit anymore in c-c. David, did you forgot the checkin link?

Comment 21

[Alexander Kogan]

sorry, my failure -> bug only exists in Seamonkey. Thunderbird works well. :)

Comment 22

[:aceman]

It would be strange if SM had this different from TB.

Comment 23

[Alexander Kogan]

I hoped this too, but it seems to be this different...

Comment 24

[rsx11m]

There are indeed subtle differences between Thunderbird and SeaMonkey. Common to both is that columns can be selected on a per-folder basis (this bug). However, resizing or moving the columns in one folder affects those in all other folders as well for SeaMonkey (except Drafts and Sent folders) in contrast to Thunderbird where column order and relative width are kept separately for each folder. Thus, the latter part wasn't ported to SeaMonkey as it seems.

Comment 25

[Wayne Mery (:wsmwk)]

(In reply to Wayne Mery (:wsmwk) from comment #16)
> (In reply to Mike Dallwitz from comment #15)
> > Bug 682624 (not bug 682684, as stated by keithnight above) has been marked
> > as a duplicate of this bug. However, I doubt that it is, as for me the bug
> > only appeared recently, I think with Thunderbird 6 (2011). However, I think
> > that Bug 682624 may be a duplicate of Bug 710056.
> 
> Mike, thanks for that research. bug 710056 is a relatively recent phenomenon
> - version 10. So I would think bug 682624 is a duplicate of bug 540857 -
> version 5. Do you agree? 
> 
> (note: some people skipped version 5, and would have first experienced the
> bug 540857 issue in version 6)

Comment 26

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #20)
> Comment on attachment 616664 [details] [diff] [review]
> proposed fix
> 
> What's up with this patch? It may have landed because bug 747102 didn't do
> this same change. But the is no kSmallCommit anymore in c-c. David, did you
> forgot the checkin link?

http://hg.mozilla.org/comm-central/rev/935d2a4dc62e

Comment 27

[:aceman]

Wayne, what is the question for me here?

Comment 28

[Mike Dallwitz]

(In reply to Wayne Mery (:wsmwk) from comment #16)
> (In reply to Mike Dallwitz from comment #15)
> Mike, thanks for that research. bug 710056 is a relatively recent phenomenon
> - version 10. So I would think bug 682624 is a duplicate of bug 540857 -
> version 5. Do you agree?

Wayne, sorry for the delay in replying. Most of the discussion of the bugs is way over my head. All I can say is that I no longer have the problem in Thunderbird. My last comment was #172 in Bug 710056 (4 Nov 2012), in which I stated that I didn't have the problem in V16.02, although some users had reported that it returned in that version. I haven't noticed the problem since. I just checked V17.0.4, using the procedure that reliably reproduced the problem before, and it's OK.

Comment 29

[Wayne Mery (:wsmwk)]

I just repaired my imap Inbox to fix a threading problem, and lost the column settings.

(In reply to :aceman from comment #27)
> Wayne, what is the question for me here?

I was thinking you had worked on this before. But maybe it was someone else? Maybe rkent remembers.

And I thought we had fixed this - folder settings being lost on compact or repair - but I'm not finding the fixed bug in https://bugzilla.mozilla.org/buglist.cgi?f1=longdesc&list_id=6728858&short_desc=repair%20folder&o1=anywordssubstr&resolution=FIXED&resolution=DUPLICATE&chfieldto=Now&query_format=advanced&chfieldfrom=2y&short_desc_type=anywordssubstr&longdesc=repair%20folder%20&v1=sort%20column&longdesc_type=allwordssubstr&product=MailNews%20Core&product=Thunderbird

maybe we only fixed the compact case??

Comment 30

[Wayne Mery (:wsmwk)]

I tested different imap account's Inbox folder and same results.
So perhaps my memory is faulty that this was all fixed at one point.
Local folder repair does not lose settings.
24.0a1 (2013-05-24)

Comment 31

[:aceman]

It was rkent who fixed stuff in this area.

Comment 32

[Kent James (:rkent)]

I believe that you are talking about Bug 710056. That bug was a subtle db cache issue with lots of symptoms, of which this column loss was only one, in fact my main target was Bug 750569 - Compact Folders window appearing very frequently starting in TB12. I can't really comment on whether that should have impacted the current bug without investigating further.

Comment 34

[WADA:World Anti-bad-Duping Agency]

I think location to save "Thread pane column choice" is better moved from pretty importnt "msgDatabase for msgDBHdr ==.msf file" to other place such as FolderCache(panace.dat), folderTree.json, sessions.json etc.
A known issue in "save Thread paane column choice in msf" is:
- When multiple messanger windows are opened, column choice is set at each messager window, and held in .XUL element, and saved in .msf.
- When Thunderbird is termnated with multiple messenger window opened, each messenger window saves his own column choice to .msf file.
   "Order of saving" is unpredictable.
- When Tb is restarted, because only one set of  column choice is held in .msf  file,
  "which messenger window's column choice is used after restart" is unpredictable.
This is never imap only issue.

Comment 35

[Wayne Mery (:wsmwk)]

Aceman, 
I can reproduce this. And Kevin can also**. If you are unable to reproduce, what can we do to help the cause?


** Kevin wrote me on 11/17/2015 "Yes, I am still using TB and still having the problem. For example, in 38.3.0, if I add recipient to the columns viewed, then repair the folder, recipient disappears."

Comment 36

[:aceman]

Does it only happen for IMAP? Have we ruled out extension interference here (I think there was a bug with that part)?

Comment 37

[Wayne Mery (:wsmwk)]

(In reply to :aceman from comment #36)
> Does it only happen for IMAP? Have we ruled out extension interference here
> (I think there was a bug with that part)?

I can reproduce with no addons with 24.6.0 and 31.3.0. 
Does not reproduce for local folder.
Does not reproduce by using compact folder

Comment 38

[Wayne Mery (:wsmwk)]

I can still reproduce. I am happy to test a try build if you have a theory for a patch. Or do you need more data?

Comment 39

[Ben Slade]

Bug 392510 *might* be related:  If you have an IMAP server which doesn't support "keywords", and you do rebuild index operation (new name is "Repair Folder"), it will cause .msf only "flags" (aka "tags", like "To Do") to be lost.   Sounds pretty similar to losing columns to display in some situations.   Not sure if the index rebuild/repair folder sometimes gets automatically run during upgrades.

Comment 40

[Kenyoni]

I can confirm this is still an issue on  Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0

Comment 41

[João Carlos Mendes Luís]

Still an issue on Thunderbird Linux 64 52.7.0

Not only columns disapear, also the sorting order is reset.

How to reproduce:

- Add a new column (example, Message Size)
- Change Sort order to Received, Threaded
- Call Repair Folder
- Changes above are now lost!

Comment 43

[linuxmc]

Please solve this problem !!!!!!!
We use this product in a company, and we have already had problems with this bug.

Comment 44

[Wayne Mery (:wsmwk)]

linuxmc, for what reason did you need to use repair?

Comment 45

[linuxmc]

(In reply to Wayne Mery (:wsmwk) from comment #44)

linuxmc, for what reason did you need to use repair?

Because Thunderbird sometimes behaves strangely:

1. When i click on an email, it opens another one
2. Some emails disappearead

These problems are solved by clicking on repair folder.
But.... in any case, i don't understand why the "Received" field it's not adopted by default (as in Outlook), given its greater reliability

Comment 46

[Jörg Zieren]

I'm a IMAP "power user" (50k messages in ~20 folders) and, sadly, repairing folders is frequently needed, e.g. because message threading doesn't work. (Though in Supernove repairing often doesn't help either.) Can I help with fixing this? From 10000 ft it seems straightforward, the column config is cleared when the folder is repaired and is never restored. I'd be happy to look at some code, or test a fix.

Comment 47

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 550286 - Try to preserve settings when repairing an IMAP folder. r=benc

Comment 48

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/efc7fb1fc847
Try to preserve settings when repairing an IMAP folder. r=benc,darktrojan,mkmelin

Comment 49

[Wayne Mery (:wsmwk)]

Comment on attachment 9408502 [details]
Bug 550286 - Try to preserve settings when repairing an IMAP folder. r=benc

[Approval Request Comment]
discussed risk/reward with Corey

Comment 50

[Corey Bryant]

Comment on attachment 9408502 [details]
Bug 550286 - Try to preserve settings when repairing an IMAP folder. r=benc

[Triage Comment]
Approved for beta

Comment 51

[Rob Lemley [:rjl]]

Thunderbird 128.0b6:
https://hg.mozilla.org/releases/comm-beta/rev/c8d3375a5cb3

Comment 52

[Wayne Mery (:wsmwk)]

Please test this. Thanks

Comment 53

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Hello,

Confirming this issue as verified fixed on 128.0b6(20240701153450) using Windows 11, Ubuntu 24 and macOS 14.

I have verified that my custom selection of columns for a folder remains the same after I have selected to "Repair Folder" from Properties.