Open Bug 132121 Opened 22 years ago Updated 8 years ago

Can't undo shift delete of local messages

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

People

(Reporter: Bienvenu, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: dataloss) 

Shift Delete of local messages is not hooked up to undo. Not sure if this is a
regression or not, but there have been enough little changes here and there to
undo that I suspect it might be.
nominating - this is dataloss.
Keywords: dataloss, nsbeta1
QA Contact: esther → sheelar
Keywords: nsbeta1 → nsbeta1+
Priority: -- → P2
Whiteboard: [adt3]
Target Milestone: --- → mozilla1.0
this bug has been around for a while, I will look for the dup. I am working on a 
similar bug to make undo work for imap 'remove it immediately' mode. 
I can undo shift delete of imap messages, so I would think that undoing deletes
with the delete immediately model should be able to work, since they're equivalent.
but it doesn't - hmm, must be some code checking the delete model and disabling
undo when it shouldn't.
*** Bug 62497 has been marked as a duplicate of this bug. ***
I guess this never worked in 6.x because the dup bug was filed a long time ago.
The  tricky part here is that we'd need to keep track of the deleted msg hdr in
order to undo the delete - we need more than just the msg key - we need to
offset of the status line in order to undelete the msg, and we need to be able
to restore the whole msg hdr to the db. So, we'd need to extend the local undo
object to keep track of the src hdrs in this situation, like imap does.
Changing nsbeta1+ [adt3] bugs to nsbeta1- on behalf of the adt.  If you have any
questions about this, please email adt@netscape.com.  You can search for
"changing adt3 bugs" to quickly find and delete these bug mails.
Keywords: nsbeta1-
Changing nsbeta1+ [adt3] bugs to nsbeta1- on behalf of the adt.  If you have any
questions about this, please email adt@netscape.com.  You can search for
"changing adt3 bugs" to quickly find and delete these bug mails.
Keywords: nsbeta1+
Renominating this bug for the next upcoming release. Changing nsbeta1- to nsbeta1
Keywords: nsbeta1- → nsbeta1
Blocks: 162592
QA Contact: sheelar → stephend
No longer blocks: 162592
See also bug 69992, "Undo Delete Message doesn't show until selecting 
another folder or Get Msg for Remove it immediately mode".
Mail triage team: nsbeta1-
Keywords: nsbeta1-
Keywords: nsbeta1
By the definitions on <http://bugzilla.mozilla.org/bug_status.html#severity> and
<http://bugzilla.mozilla.org/enter_bug.cgi?format=guided>, crashing and dataloss
bugs are of critical or possibly higher severity.  Only changing open bugs to
minimize unnecessary spam.  Keywords to trigger this would be crash, topcrash,
topcrash+, zt4newcrash, dataloss.
Severity: normal → critical
I don't know if this is related, but undo for a single message deleted with the
"delete" key also doesn't always work. It seems you have to manually open the
Trash folder, and only messages deleted after that can be restored with UNDO,
see bug 198909.
Product: MailNews → Core
(In reply to comment #6)
> I guess this never worked in 6.x because the dup bug was filed a long time ago.
> The  tricky part here is that we'd need to keep track of the deleted msg hdr in
> order to undo the delete - we need more than just the msg key - we need to
> offset of the status line in order to undelete the msg, and we need to be able
> to restore the whole msg hdr to the db. So, we'd need to extend the local undo
> object to keep track of the src hdrs in this situation, like imap does.

How about not actually deleting the header, just setting its MSG_FLAG_EXPUNGED
flag. Then the Undo code will actually have something to hang on to. At shutdown
time we can go thru and delete all the headers that have the EXPUNGED flag set.
(Or we can use a timer...)
that approach is worrisome - we'd need to check if the header was expunged in
multiple places, most likely, and we'd need to fudge the folder and thread msg
counts. And cleaning up the expunged headers might be a pain (though in that
approach, the undo manager should clean them up when it gets cleaned up).

There might be a variation on that approach that will simplify things - in
addition to (or insead of) setting the expunged flag, we could remove the header
row from the two tables it's in (the thread table, and the all msg hdrs table).
The row will continue to exist in Mork, but the msg db code will never see it.
When undoing, we'd need to add the row back to those two tables. When the db is
closed, Mork will automatically delete the row because it's not in any tables.

The other alternative I mentioned before is just to have undo keep track of the
info it needs, which is just the message key and the status line offset.
However, we'd need to reparse the message headers to regen the msg hdr, so an
approach that keeps the msg hdr alive has advantages.
At least have a confirmation that selected records will be permanently deleted.
5 years on, and we have neither "undo" nor "confirmation message before permanent deletion" (that's related bug 308690). At least the proper tags are there: critical, dataloss. Just to keep this alive.
I'd believe tackling bug 308690 with a "don't show this again"-variant of the confirmation message before deletion should be far easier to implement than UNDO, so maybe we can try to get 308690 confirmed for a start...
Assignee: bienvenu → nobody
OS: Windows 2000 → All
Priority: P2 → --
QA Contact: stephend → backend
Hardware: PC → All
Whiteboard: [adt3]
Target Milestone: mozilla1.0 → ---
is this the savior for all undo cases, eg bug 308690?
Blocks: 308690
Flags: wanted-thunderbird3?
I do not consider undo for shift+del appropriate, as in windows, on a normal file shift+del does not offer undo, so would be an inconsistency in UI. After all, shift+del is inspired by os UI. And might undermine the point of "skip trash, permanent delete" that I see in shift+del

An undo in this case seams more related to a recover/restore of data

Alert is here on TB 2.0.0.12, and should prevent the accidental action, but missing in 3

I had similar comment on bug 308690
Since there seems to be so much controversy and very strong opinions about this wouldn't it be possible to make it configurable? That way everybody would be happy!
Product: Core → MailNews Core
(In reply to comment #20)
> Since there seems to be so much controversy and very strong opinions about this
> wouldn't it be possible to make it configurable? That way everybody would be
> happy!

I agree. Let's try and make the "everyone happy" with "configurable" scenario more explicit (THESES)

1.) As a default, show (optional) confirmation message before permanently deleting messages with shift+del, mainly for 3 reasons:
a) avoid accidental dataloss(!)
b) consistency with OS (and most apps, I reckon): Windows Explorer DOES show the confirmation msg for Shift+DEL, and even Total Commander does. Surely Total commander has not been invented to slow things down.
c) shift+del should not be undo-able (see below).
d) Besides, I imagine that integrating a simple confirmation dialogue, even with "don't show this again" checkbox, is surely less complex to program than undo...

2.) Make confirmation popup optional with "don't show this again checkbox".
a) This will grant peace of mind to all netscape4 power users who prefer that extra speed & thrill of *immediate silent* deletion
b) This will make accidental users aware of this very useful command, including the option to speed things up by checking "don't show..." once you get used to it.

3.) Do not make shift+del undoable, as this would be against the very concept of permanent deletion, as ovidiou rightly points out in comment #19. "Undo permanent deletion (bypassing trash)" is a contradiction in itself, but could well be provided by an extension, if need is. Points here:
a) consistency with OS - shift+del is NOT undoable for files.
b) complementary feature consistency: if I wanted deletion to be undoable, I'd have to use DEL to TRASH
c) privacy: when I shift+del (with or without confirm), I MEAN permanent deletion. Otherwise I'd just DEL them messages into trash for everyone to pick out again. No, I don't want that expunged secret love letter or the bloody porn spam to be easily undoable.
b) usability: having "undo permanent deletion" in your standard undo sequence is a nuisance, mostly because of 2. b) and c). Imagine having hand-picked your mails for permanent deletion, and they all come back when you want to undo a step before that. Confirmation dialogue ensures that even for the average user, shift+del will always be a conscious decision, so no reason for undo. And the netscape power user bypassing the confirmation (via once-only checkbox) will know how to recover lost data anyway, without undo in the UI (or install addon to do so).
I imagine undo will be far more useful for the msg which I have accidentally dropped into the wrong folder than for deliberate (and now fool-proof) permanent deletion.

4.) Make shift+del more easy to find and understand for the average user (until now, it's an undocumented shortcut, not in the menus, which increases the likeliness of accidental data loss!)
-> include shift+del in official shortcut documentation (bug 326869) and in help files
-> think about menu implementations, at least holding shift while deleting via menus or buttons should do the trick. Maybe informative tooltips while hovering over these.
- /default/ confirmation will be enough safety for the average user to safely use this excellent feature
- /configurable/ confirmation ("don't show this again") is there, ready for advanced users at the usual lightning speed as ever...




typo correction:
3) *d* (not b) usability: having "undo permanent deletion" in your standard undo sequence is a nuisance, mostly because of *3*b)+c) (not 2).
I like the break down of scenarios.  I'm mostly worried about people who press delete while accidentally holding shift down.  This, like the ctrl-enter to send an email is a hidden keyboard shortcut that can potentially have harmful effects.

I think what I'd propose is that the shift-del shortcut only sends messages to the trash.  We can use an info bar to ask people if they want to turn on the permanent shift-delete for messages.  This allows people to make a mistake on the keyboard and then decide they want to have the extra deleting power after that.  Of course if they choose to have shift-del as permanent delete then we'd permanently delete that message that just went to the trash. 

So, the user scenario should work like this.

- Person who wants shift delete -
Person presses shift-delete on message in focus.  Message is removed and an info bar appears above the message view pane.  Info bar explains that the message was sent to the trash, you have to turn on permanent delete.  Person clicks the button for [Shift-delete Permanent Delete].  The previous message that went to the trash is now permanently deleted and this person can continue using shift-delete as they intended.

- Person who clicks shift-delete by accident -
Person presses shift-delete on message in focus.  Message is removed and an info bar appears above the message view pane.  Person ignores the message bar and shift-delete continues to work as regular delete with the info bar showing when it's used.

A concern about the info bar is that we should avoid making the only button the item that enables shift delete as people might be clicking it to dismiss the info bar.  Instead we should probably use a checkbox with the button being clicked just so we're sure people are doing this on purpose.  For a person who knows what they are doing this is a one-time extra few clicks for good reason.

Also I'm not sure that undo is a bad concept for this.  I believe a users intention here is to never have to deal this the message in the trash folder, possibly for privacy or other reasons.  However people can often make mistakes and I don't see why we couldn't hold on to the message for a minute such that if you hit shift-delete by accident you could get it right back if you realized what you just did.  (there's likely a lack of feedback for this to work out well, but perhaps the new status bar could help here)  Assuming the message was set to expire permanently in a minute or on thunderbird exit I think we'd be doing what the person wants while still providing a slight escape for mistakes.  It would be a temporal undo, only available for a short period of time.

An alternate to a temp undo would be a confirm dialog, but people just get into a rhythm of hitting enter right away after their keyboard choice so it doesn't actually save the person from problems.
(In reply to comment #23)
> I like the break down of scenarios.  I'm mostly worried about people who press
> delete while accidentally holding shift down.  This, like the ctrl-enter to
> send an email is a hidden keyboard shortcut that can potentially have harmful
> effects.
That's what this is about and I would put it simply, the effects are related to alerting [awareness] about accidental actions.

> I think what I'd propose is that the shift-del shortcut only sends messages to
> the trash.  We can use an info bar to ask people if they want to turn on the
> permanent shift-delete for messages.  ...
Good scenario, IMHO. I take it as a smarter response and pace than the usual alert "are you sure.." with a check. I see it as basically same alert but getting a better interaction with user of any kind. 
Also I like that being OS shift del "standard" compliant but with a better msg.


> Also I'm not sure that undo is a bad concept for this.  I believe a users
> intention here is to never have to deal this the message in the trash folder,
> possibly for privacy or other reasons. 
I think undo is bad for these reasons:
-not behaving like OS del and shift del (what if drag +ctrl is not copy?also ..)
-adding extra steps for really del something, this beig solved in os with additional app, like norton? recycle etc that added extra undo or recovery. Thus, maybe better in an extension
-Maybe is more complicated in terms of usability (learn extra stuff)
-probably more complicated as coding (is it woth for just this function?)

And I think dealing with trash is not the main issue. But this is just my opinion with no info on it.

*In conclusion, I like the first proposal for dealing with a notification that is more involving that usual and results in satisfying both cases. 

[The other, undo/recovery like, I see as a wider issue for recovering data and backups, undo history or alike. Which is maybe another topic and broader than this. At least more complex and worth debate in appropriate larger context and implications.]



PS: there is this extension today that brings extra alerts for shift del, del or both and more. Probably started for the lack of shift del alert, but ending in a tool for those extra careful (normal del alert also). 

https://nic-nac-project.org/~kaosmos/index-en.html#cbd
"confirm before delete"
ah, not amo, as many other good ones here ..
While the info bar approach in comment 24 could be good for the users that accidentally hit shift-delete, it doesn't help if one *wants* a confirmation dialog for each shift-delete, which is fairly normal OS behaviour I'd say.

I don't think many users accidentally hit shift-delete, and as the mails aren't really physically deleted anyway (mbox must be compacted first), those could be saved by making undo work properly.
Ok, I think I'm feeling pretty against trying to create an undo system for permanent delete which intentionally by passes the current undo system of the trash folder.  The complexity of all that is just frustrating.

I think the approach of using the info bar to turn on the shift-delete "permanent delete" (which is off by default) which from then on always uses a dialog to confirm permanent deletes would work out well for all parties.

And I don't like the [x] "Don't show this again" for the confirm dialog of actions that aren't undo-able, it's just a bit too dangerous.  If you hit shift-delete by accident you should always have the chance to cancel.  If you hit shift-delete and then enter you shouldn't hit so many buttons after having turned on shift-delete. :-(
I'm not sure I understand - are you saying that everyone would lose the ability to shift delete w/o confirmation? Or would that be an option somehow?
Yes, shift delete would always require the confirmation.  Most systems react in this manner because there is no undo available.  

If people are using shift-delete a lot then it seems more likely that something is wrong with our trash settings than we need faster shortcuts for destroying mail.
Just my 2c - I doubt there are many users who uses shift delete on purpose that would want a confirmation - the main point is to make it as fast as possible. So I'd challenge your assertion that this will work out well for all parties. I think we're making a trade-off between safety for users who accidentally do shift delete, and users who want to quickly delete w/o trash. It's fine to side with safety, but we should just be aware of it. 

If there was some way to have zero-overhead delete to trash, then we could claim that shift delete wasn't needed, but given IMAP and our current berkeley db mailbox format, we're not there yet.
I don't get the problem with [x] "Don't show this again" - done right, accidentally hitting shift-delete + enter cancels, and to accidentally turn it on without confirmation, you would have to accidentally hit shift-delete + shift-tab + space + tab + tab + enter. Sure, kitty-typing might do that, but then it might rm -rf ~/, too.
(In reply to comment #30)
> I doubt there are many users who uses shift delete on purpose that
> would want a confirmation - the main point is to make it as fast as possible.

Exactly. In fact, having the mere "delete" be undoable while having "shift delete" for 'real' delete is the whole point! If we don't do real delete as fast as the undoable case, we don't need it at all. If I have to confirm each of my real deletes, I might as well not use it, but send stuff to the undoable trash until it explodes.

Making real delete require confirmation you can't even turn off is nonsens.
Just remove it, then, because it's useless.

> I think we're making a trade-off between safety for users who accidentally
> do shift delete, and users who want to quickly delete w/o trash. 

Exactly.

(In reply to comment #31)
> I don't get the problem with [x] "Don't show this again"

The point is not where to kill the confirmation, the point is to have the possibility at all. A linked "You can turn this behaviour on/off in the preferences." is probably better than a mere checkbox.
> I think we're making a trade-off between safety for users who accidentally do
> shift delete, and users who want to quickly delete w/o trash. It's fine to side
> with safety, but we should just be aware of it. 

Completely right.  As Phil points out we can put the "don't warn me checkbox" in there creating many hoops for people to jump through in order to get this delete-delete hooked up.  

I often like to error on the side of safety for these kinds of systems, even though it may slightly hurt the other cases.  But I can definitely be swayed and now that your arguments are clear and in the open we can forward any and all resulting support issues to these comments :)

> If there was some way to have zero-overhead delete to trash, then we could
> claim that shift delete wasn't needed, but given IMAP and our current berkeley
> db mailbox format, we're not there yet.

It would be nice to be there now, or soon...
This seems to have stagnated.  Just want to summarize my feelings on this.

Shift-Del is a hidden feature that can be dangerous when discovered, especially since there is no undo.  There is no undo for this feature by design so I believe we need to have a safety switch on by default that prevents people from hurting themselves by accident.  With "safety switch" reference I'm trying to not make some joke about shift-del doesn't kill messages, people do but I will stop myself...  Part of this feature is a quick, "I'll never need this message again", choice so it seems like a confirm dialog would detract from the feature itself.  I like to avoid all the "Don't ask me again" checkboxes on dialogs as a personal design preference.  

I still like the scenario from Comment #23 where shift-del is off by default and we allow people to turn it on and then have the current shift-del feature working.
But who really does a shift-delete by accident? I'd claim it's only people who know what it does who use it.

Note that this bug isn't even about someone finding accidental shift-delete to be the problem. It's more a "why did i do that" kind of bug, so a config option would do little good. (And I don't know of a bug where someone is complaining about it either.)
Blocks: 209189
For what it's worth, I just permanently lost 8,500 messages in my inbox because of this.

I had intended to select a bunch of spam and shift-delete it so as not to take up space in the trash.

What I actually did was accidently bumped the neighboring 'end' key before hitting delete, and since shift was already down, it selected everything in my inbox from the cursor downwards, and as delete came down, bang, everything gone. Undo didn't work, and even end-tasking thunderbird just seconds later wasn't fast enough.

Now I wish that I had backed up my gmail.
As long as you didn't compact, you can retrieve your deleted messages - http://kb.mozillazine.org/Undelete_a_message
(In reply to comment #37)
> As long as you didn't compact, you can retrieve your deleted messages -
> http://kb.mozillazine.org/Undelete_a_message

undo fails for me after shift+delete for both pop and local folder.

xref Bug 539730 - undo undelete does not undo (which might be entirely pop, not sure because the bug isn't well defined)
(In reply to comment #37)

Thanks, but that appears to only work for messages deleted without holding down shift.  I've been through all the deleted folders, viewed the mailbox via gmail.com, searched the mailbox etc. I do have copies of a lot of things that were quoted when I replied, but it would be nice to get the actual inbox back. If it makes any difference, this is talking to google via imap, not pop.
(In reply to comment #39)
> (In reply to comment #37)
> 
> Thanks, but that appears to only work for messages deleted without holding down
> shift.  I've been through all the deleted folders, viewed the mailbox via
> gmail.com, searched the mailbox etc. I do have copies of a lot of things that
> were quoted when I replied, but it would be nice to get the actual inbox back.
> If it makes any difference, this is talking to google via imap, not pop.

the only difference between delete and shift delete is that the former copies messages to the trash first. They both leave the deleted messages in the mail folder on disk, until the folder is compacted.
Oh, I see what you mean now. Thanks.. 8500 messages back again. Cheers! :)
Blocks: 644075
I just experienced the same problem as in #36 (shift + end instead of del and then del).

As a workaround would be the purgecontrol -> http://purgecontrol.mozdev.org/  Unfortunately, it does not seem to work with new Thunderbird versions.
Flags: wanted-thunderbird3?
You need to log in before you can comment on or make changes to this bug.