356535 - retention policy "delete message more than N days old" changed by it self to "same as inbox"- after update to version 1.5.0.7 I lost most of my stored mails [pop]

Status: NEW

(Thunderbird :: Preferences, defect)

Description

[Rupert R. Foierl]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7

After restart I noticed that most of my stored mails in the sub files where gone. My inbox (Posteingang) was marked to delete the mails older than 30 days. In all folders and sub folders the mark was to keep all mails (Keine Nachrichten löschen). During my search for the reason of loss I found in random sub folders the mark changed to "same as inbox" (Einstellungen des Kontos verwenden)therefore all mails older than 30 days where deleted and I lost most of my importent mails with orders, codes etc. I used the Thunderbird for the last 6 month and found is is a good program but now this ! What should I say ?

Reproducible: Couldn't Reproduce




If you need pictures to show this damage just ask.

Comment 1

[Gervase Markham [:gerv]]

This bug is not security-sensitive; marking it as such just means that fewer people look at it.

Gerv

Comment 2

[Magnus Melin [:mkmelin]]

I have seen a few similar reports, not necessarily related to upgrade though. 

Comment 4

[voodoo]

Ugh. This just happened to me. See my whole story here: https://bugzilla.mozilla.org/show_bug.cgi?id=424927

Short story, lost half a year of emails. Tragic, actually. I believe it happened after the 2.0.0.12 update. Magnus is not convinced the update was the culprit. So maybe it was a coincidence, and it had to do more with recovering from a TB crash. In any case, I hope someone can improve failsafe mechanisms around this feature. It can be, and was, disastrous.

Comment 5

[Wayne Mery (:wsmwk)]

Voodoo, was this an imap folder?

Comment 6

[voodoo]

Nope. It was on several POP accounts. TB woke up one day and decided to set them all to "expire all mail older than 15 days". Took me weeks to figure out why my email was vanishing. Very unfortunate feature.

Comment 7

[Wayne Mery (:wsmwk)]

Joe, have you seen this mentioned?  I looked in forums but haven't found anything.

Bryan, related?
Bug 396302 – retention policy will spontaneously reset on some folders to "don't delete any messages" (edit)
Bug 231541 – "Leave messages on server" should be chosen by default (dataloss)

xref Bug 380136 – "Delete messages more than 30 days old" set on Local Folders after upgrade to 2.0 (SuSE)

confirming

Comment 8

[Joe Sabash [:JoeS1]]

I don't recall any reports like this in the forums.

Aside from the issue of what caused the pref change, I don't see the rationale of
defaulting the deletions to a shift+delete operation.
Maybe this was an unintended consequence of adding shft-del

There is always the possibility of a simple user error, and the mail is gone
irretrievably. (move to trash should at least be the default)

Comment 9

[Wayne Mery (:wsmwk)]

(In reply to comment #2)
> I have seen a few similar reports, not necessarily related to upgrade though. 

Magnus, were these reports in newsgroup?

Comment 10

[Magnus Melin [:mkmelin]]

I think in bugzilla (hard to recall), but "a few" might have been maybe one or two.

Comment 11

[Wayne Mery (:wsmwk)]

tanstaafl, have you seen reports of anything like this or Bug 396302?

I am investigating a couple reports at our site.

Comment 12

[Wayne Mery (:wsmwk)]

Rupert appears to be gone.  Too bad, it would be nice to know more.  But it has been a long time since his report

Scratch pad memory note: Stacey A had the problem here.

really confirming this time, but still don't know why this is happening

Comment 14

[Wayne Mery (:wsmwk)]

what needs to happen for this bug to more forward?

Matthias ... does this bug describe your issue?  And how is this issue different from Bug 231541?

Comment 15

[Darren Hill]

Something similar to this happened to me with thunderbird 3 beta 4 and I think I can give extra details or it may be something else completely.

Effects: Thunderbird 3 Beta 4 (I'm not on the computer this happened on so I don't have the exact useragent but it was an install from the release beta of beta 4 and a profile migration from thunderbird 2)



Step to reproduce:
1.) Make a sub folder
2.) Make a 2+ sub sub folder inside the first sub folder.
3.) Set the retention policy in one of those sub-sub folder
4.) All sub folders(that weren't previously set to something besides 'server settings policy') change to match the retention policy just changed which is unexpected and can easily cause a loss of archived mail.

Expected results:
1.) Each and every folder, no matter the hierarchy should have it own retention policy.  A folder should not control sub folders retention policies.

Details:

I have many sub folders that I used to organize incoming emails.  One sub-folder - let's call it sub1 - has a bunch of other sub-sub folders lets call them ssub2 and ssub3.  When I change the retention policy of ssub2, ssub3 changes to it as well.  
Essentially ssub2 received lots of redundant emails.  When I set it to only save emails for hte last 7 days, all the other sub-folders at that level(ssub3 in this case) also switch to it, and removed all the old emails I wanted to keep.

Comment 16

[Mark Banner (:standard8)]

(In reply to comment #15)
> Something similar to this happened to me with thunderbird 3 beta 4 and I think
> I can give extra details or it may be something else completely.

Nope, your issue is bug 521134 which is already fixed for Thunderbird 3 final.

Comment 17

[Darren Hill]

Ah thanks that's good to know.
I looked for the issue before posting but I didn't see that because
"pendingRemoval flag set by autosync constraints, but not cleared"
doesn't scream, or even hint at, the issues I had from a casual glance. :)

Comment 18

[:aceman]

I could imagine the folder setting could be reset to "use account settings" if the file containing the setting is lost. I assume it could be the corresponding .msf file. There are cases when this file is lost/corrupted and is rebuilt automatically. We could check if the rebuild preserves this setting.

But if the file is lost completely there is not much we can do, just apply the settings from the server. If those are stricter than those defined on the folder previously, then the data is toast :(
So possible workarounds:
1. use a different default for folders, e.g. "Don't delete any messages"
2. remove the "use account settings" option, or make it non-default
3. teach users to set the most lax settings on the account and make individual folder have more strict settings (less days to keep msgs). In case of this reset, nothing is then lost outside expectations.
4. programmatically enforce option 3. and do not allow user to set the hierarchy the other way round.

Also, when the automatic retention deletes messages, don't they go to Trash first? Can't the messages be recovered from there?

Comment 19

[[e-mail]]

(In reply to aceman from comment #18)
> I could imagine the folder setting could be reset to "use account settings"
> if the file containing the setting is lost. I assume it could be the
> corresponding .msf file. There are cases when this file is lost/corrupted
> and is rebuilt automatically. We could check if the rebuild preserves this
> setting.
Only the "use account settings" is cached, and this is done to avoid hitting the database when the account settings are used. Only if both the cache and the database agree can the database settings be used.

> 1. use a different default for folders, e.g. "Don't delete any messages"
> 2. remove the "use account settings" option, or make it non-default
That does make per-account settings somewhat inconvenient though. 

> 3. teach users to set the most lax settings on the account and make
> individual folder have more strict settings (less days to keep msgs). In
> case of this reset, nothing is then lost outside expectations.
Indeed, some sort of warning could readily be added to the folder properies UI.

> 4. programmatically enforce option 3. and do not allow user to set the
> hierarchy the other way round.
Problems ensue with existing users.

> Also, when the automatic retention deletes messages, don't they go to Trash
> first? Can't the messages be recovered from there?
I don't think they go to Trash; I don't know that that would be feasible. (Bear in mind that this has to cope with offline store maintenance too.)

Comment 20

[[e-mail]]

Actually there may be a third way. There are three possible values for the retention settings, "1" (use account settings), "0" (override settings), and "" (new folder, default to use account settings). If we see the retention settings have been overridden, but the overridden settings say to use the account settings, then we know that something is amiss and we can set them to keep all messages.