Last Comment Bug 250141 - Need way to purge/delete downloaded newsgroup messages
: Need way to purge/delete downloaded newsgroup messages
Status: VERIFIED FIXED
:
Product: MailNews Core
Classification: Components
Component: Backend (show other bugs)
: Trunk
: x86 All
: -- enhancement with 15 votes (vote)
: Thunderbird 3.3a2
Assigned To: Joshua Cranmer [:jcranmer]
:
Mentors:
: 281256 366753 589501 592083 638433 (view as bug list)
Depends on:
Blocks: TB2SM 237786
  Show dependency treegraph
 
Reported: 2004-07-07 01:43 PDT by Robert Harvey
Modified: 2012-08-27 10:42 PDT (History)
33 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Patch, version 1 (38.78 KB, patch)
2010-09-10 07:25 PDT, Joshua Cranmer [:jcranmer]
mozilla: review+
Details | Diff | Splinter Review
Patch, version 2 (38.70 KB, patch)
2010-11-24 07:31 PST, Joshua Cranmer [:jcranmer]
Pidgeot18: review+
neil: superreview+
Details | Diff | Splinter Review

Description Robert Harvey 2004-07-07 01:43:49 PDT
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040614 Firefox/0.9
Build Identifier: Thunderbird  version 0.7.1 (20040626)

At the moment I have my system set to retain newsgroup messages downloaded for a
day, but on high-traffic groups this is still something of a nuiscance.

I'd like to suggest that we need a way to discard threads, or all the messages,
once read.

Reproducible: Always
Steps to Reproduce:
Comment 1 Robert Harvey 2004-08-08 02:18:49 PDT
I am currently simulating this activity with view-threads-threads_with_unread
and account_settings-download-keep_messages_arrived_within_1_day

But it is clear that there is a lot of dross still on my hard disk, and a method
of purging downloaded messages still seems desirable.
Comment 2 Reuven Gonen 2005-04-05 22:52:22 PDT
we should really have a way to remove messages from the view list even when they
are on the server. this is done when receiving mail that is still on the server,
where the messages are not downloaded again.
Comment 3 Alexey Antipin 2005-04-20 09:16:11 PDT
It would be great to delete that spam messages:)
Comment 4 Adam Guthrie 2005-08-05 16:02:32 PDT
*** Bug 281256 has been marked as a duplicate of this bug. ***
Comment 5 Matthias Versen [:Matti] 2007-01-11 18:08:38 PST
*** Bug 366753 has been marked as a duplicate of this bug. ***
Comment 6 Wayne Sallee 2007-01-11 18:31:03 PST
This really needs to be fixed so that porn can be deleted from newsgroup posts without deleting good posts.
Comment 7 Melchert Fruitema 2009-04-17 06:58:28 PDT
What we really need is the ability to manually run a message filter for newsgroups, locally. I believe that enhancement has been requested but is not on the list for Tb version 3.
Comment 8 Alex 2010-01-06 20:34:44 PST
What we do need is to be able to delete messages from local storage. Right now it tries to cancel the message on server, which is not supported nowadays.

P.S. Five years and no fsking reaction?
Comment 9 Mattias 2010-01-11 11:25:22 PST
(In reply to comment #8)
> P.S. Five years and no fsking reaction?

Yes, we could definitely use this feature but please be civil.  You are dealing with programmers (people) volunteering their time.
Comment 10 Marc Chamberlin 2010-07-20 11:15:02 PDT
Mattias, and the Thunderbird team in general -

Alex's question regarding why this bug being now 5 years old is no longer a matter of civility but has become an extremely important question on a lot of levels. 

1. This particular bug should no longer be regarded as a mere enhancement request, but the importance of it raised to Major, Severe or even Critical! The inability of Thunderbird, to delete old, read, or unwanted newsgroup messages renders this subcomponent of Thunderbird to near unusable status since the database can become overwhelmed with postings, and it denies the user the ability to maintain control over what should be kept verses what will be removed via the automated methods used to expire old messages...

2. Since this bug has languished so long, the question of whether Thunderbird should continue to support Newsgroups should be raised. If volunteers cannot be found to address Newsgroup issues, in a timely manner, then perhaps the feature should be dropped entirely. This would prevent the waste of users time trying to figure out whether they should or should not be using Thunderbird as a newsgroup reader. Not having the feature, and declaring Newsgroups outside of the purview of Thunderbird's responsibilities allows users to quickly move on to other tools that will better meet their needs.

3. If a lot of bugfixes, feature requests, etc start taking an extraordinarily long time to resolve, as this one apparently has, users are going to become extremely discourage and will migrate away from Thunderbird, to look for better solutions elsewhere. And loosing user support is a critical issue, which is why all bugs and enhancement requests that are more than a year or 2 old should be raised to higher levels of importance, and addressed one way or the other asap. Being complacent about old bugs and not getting them resolved will kill this project eventually as more and more users get fed up, angry, frustrated and begin to badmouth the project openly and move away from it.  Do a Google search on this problem and you will find that it is already beginning to happen...  For example Google led me to this thread in your own forum! - http://forums.mozillazine.org/viewtopic.php?f=30&t=589746
Comment 11 Chris Ilias [:cilias] 2010-07-20 15:53:27 PDT
Guys, I'm a newsgroup user just like you, not a developer. But I don't think you're looking at this bug objectively. If you disagree, I (or anyone from the MozillaMessaging team) would love to discuss it via email or in one of the Mozilla newsgroups.
<http://www.mozilla.org/community/forums/>

But on bugzilla, bug comments are for constructive comments directly related to fixing the bug. See <https://bugzilla.mozilla.org/page.cgi?id=etiquette.html>

P.S. MozillaZine is not owned or operated by Mozilla. See <http://www.mozillazine.org/about/>
Comment 12 Robert Harvey 2010-07-21 03:24:06 PDT
Well, as the original poster, I must say that I stopped attempting to use Thunderbird for usenet a few months later.  I switched to Pan, and google groups.  I have not tried Thunderbird again in the ensuing half-decade.

This bug was only one of the reasons I found it unusable for usenet messages.

I offer these comments as an attempt to give scale and context to the significance of the the original report and it's impact.  I regard this as on-topic for bugzilla.

I fully accept that it is a collaborative project, and people much prefer re-designing the user interface over and over again to fixing underlying problems.  That's up to them.  Shoving off and using something else is up to me, so I did.
Comment 13 Wayne Sallee 2010-07-21 03:58:24 PDT
I agree with the original poster. It is completely irresponsible of Mozilla to ignore this bug.

This bug also prevents a person from deleting inappropriate content that some perverted person posted to the newsgroup.

Wayne Sallee
Wayne@WayneSallee.com
Comment 14 Mattias 2010-07-22 11:48:58 PDT
(In reply to comment #13)
> This bug also prevents a person from deleting inappropriate content that some
> perverted person posted to the newsgroup.

That's very important to me--it is why I voted for this bug

(In reply to comment #10)
> 2. Since this bug has languished so long, the question of whether Thunderbird
> should continue to support Newsgroups should be raised.

I am also not using Thunderbird for news groups anymore.  I agree with this point.  Focus resources on the essentials, since TB programmers are obviously spread too thin.
Comment 15 Jim Porter (:squib) 2010-07-22 12:32:38 PDT
(In reply to comment #13)
> This bug also prevents a person from deleting inappropriate content that some
> perverted person posted to the newsgroup.

It's not ideal, but you can kill the thread by pressing K (or the subthread by pressing Shift+K).
Comment 16 John H Meyers 2010-08-22 02:07:55 PDT
Although the original goal of this bug (250141) was stated as an enhancement
to purge messages automatically from one's profile, once read,
this has become confused with an essentially different bug, e.g.
Bug 366753 "No ability to delete messages in newsgroups" and
Bug 281256 - "I want to be able to delete (not cancel)
some (not all) newsgroup messages from view,"
which is that the GUI does not even allow one
to delete messages _manually_ from one's profile,
and thus prevents one from deleting a porn message, a spam, etc.

The latter have been marked as "duplicates" of this bug,
but they are not really, because the reason you can not even
_manually_ purge selected messages from your profile
(and two other consequences of the very same mistake)
is actually due to a very simple bug in the GUI itself,
which has an equally simple cure!

Because of the marking of the latter as "duplicates" of this bug,
this bug now carries the burden of representing the latter,
and is no longer an "enhancement" -- it's a plain and simple GUI bug.

To avoid confusion and to properly classify the new bug,
I have opened a new report, as Bug 589501,
and have included the straightforward solution.

I think you will find that solution self-evident,
and to give immediate relief to that long-standing,
but slightly different problem.

Best wishes to all.
Comment 17 Joshua Cranmer [:jcranmer] 2010-08-23 04:44:08 PDT
*** Bug 589501 has been marked as a duplicate of this bug. ***
Comment 18 Joshua Cranmer [:jcranmer] 2010-08-23 04:46:50 PDT
(In reply to comment #16)
> Although the original goal of this bug (250141) was stated as an enhancement
> to purge messages automatically from one's profile, once read,
> this has become confused with an essentially different bug, e.g.

The reporter of this bug states quite clearly in comment 2 that this bug is now mostly about what the subject says, no automatic bit.

> The latter have been marked as "duplicates" of this bug,
> but they are not really, because the reason you can not even
> _manually_ purge selected messages from your profile
> (and two other consequences of the very same mistake)
> is actually due to a very simple bug in the GUI itself,
> which has an equally simple cure!

Actually, nsMsgNewsFolder::DeleteMessages performs a cancel instead of a delete. It's not just a GUI bug, there's backend core that's needed to.

> Because of the marking of the latter as "duplicates" of this bug,
> this bug now carries the burden of representing the latter,
> and is no longer an "enhancement" -- it's a plain and simple GUI bug.

This requests to add a feature which doesn't exist--therefore it's an enhancement.
Comment 19 Timo Pietilä 2010-08-23 04:56:58 PDT
I just need to emphasize that what happens to files in HD is secondary in this design flaw. What really matters is that GUI should not show "deleted" messages. You can do that for entire threads using ignore thread-command, but not for individual messages. Deleting things from HD comes second (you can't see it, so why keep it in the HD).
Comment 20 John H Meyers 2010-08-23 16:28:53 PDT
(In reply to comment #18)

> This requests to add a feature which doesn't exist,
> therefore it's an enhancement.

No, the "feature" (to delete a post stored in your own profile)
obviously _does_ exist, not only because it's the normal
"delete" action that is offered as a tool button,
menu item, shortcut key, etc. for all messages in all accounts,
but also because it is an already _working_ function --
that is, you already _can_ delete your _own_ posts,
so obviously there is nothing _missing_ which needs to be _added_,
as the word "enhancement" means.

The only time it fails is when you try to purge some other post,
so it's only a bug in an existing feature,
as well as so easy to correct that it might well take less effort
to fix that bug than even continuing to argue that it's anything new,
rather than an easily fixed bug in an old function,
manifesting only under certain particular conditions.

The reason that it doesn't work on posts other than your own
is only that each separate request (cancel vs. delete)
invokes both functions, which is the mistake
that causes it to abort in that case.

In the GUI, both "cancel" and "delete" are clearly the same thing:

You can see this by inspecting the "Edit" menu when either
a mail message or any "Blogs+Feeds" message is selected,
where in the "Edit" menu we then see "Delete Message [Del]"

If we instead select a news message,
we see instead in the "Edit" menu "Cancel Message" [Del]"
(compare with the above)

Now, the "Del" [key], consistently
throughout the GUI, means "delete," so obviously we have
a mistake in the GUI here, where "cancel" should be invoking
only whatever internal function may confine itself
to sending a "cancel" to the server.

Similarly, "delete" should be invoking an internal function
which should confine itself to processing deletion of the stored message,
the same as it does for all other account types.

Various people assert they have been able to construct filters
(and/or use "delete more than NN days old," which is in the GUI)
which delete news messages without also attempting their cancellation,
which suggests that such an internal function already exists.

You say "nsMsgNewsFolder::DeleteMessages
performs a cancel instead of a delete. It's not just a GUI bug,
there's backend core that's needed too"

Do you mean that this function actually performs _both_
cancel and delete?  If so, all it needs is either to be
split into one function which only deletes and another function
which only cancels, or else perhaps it should serve as
a function which only cancels, which should be called
only from the GUI "cancel" element, while a different function
is called from the GUI "delete" element -- in any case,
the combination of adjusting the GUI to make separate calls,
one for "cancel" and a different one for "delete,"
to functions which likewise confine themselves
to performing only "cancel" or only "delete,"
would fix this bug.

> The reporter of this bug states quite clearly in comment 2 [1?]
> that this bug is now mostly about what the subject says.

Robert Harvey mentions a connection with "read/unread" status,
which has induced generations of subsequent readers (and their comments)
to consider the request to involve that status, and to propose solutions
involving that status, which _would_ make this an enhancement,
but if we eliminate that consideration, then what remains
is simply a bug, having its cause in at most two locations:

o The GUI itself equates "cancel" with "delete";
this has to be corrected, no matter what else happens,
and possibly this, by itself, might correct the "delete" issue.

o An internal function called by the GUI,
perhaps still the only function which "cancel" might call,
may perform both functions at the same time, in which case,
calls to that "two in one" function could be replaced by
calls to lower-level functions which do only one thing at a time,
or the given internal function could be "split" into two,
so that each split part would do only one thing.

If you would be willing for this to be elevated to "bug,"
if it can be fixed in GUI alone (which perhaps it can),
why not if the bug is even considered to involve two locations?

I'm sure that whoever steps up to make the apparently simple
correction(s) will become a hero to many users,
as well as elevate the product's otherwise well-deserved esteem
(see comment #10 by Marc Chamberlin, comment #12 by the OP,
and well, practically _all_ of the comments!)

Best wishes.
Comment 21 Joshua Cranmer [:jcranmer] 2010-08-31 13:29:52 PDT
*** Bug 592083 has been marked as a duplicate of this bug. ***
Comment 22 ipatrol 2010-09-03 05:05:05 PDT
Some objections raised here have been based on the fact that online mode has no real storage to delete from. What if this were implemented as a hashtable where each entry has a given lifespan and the newsreader hides anything that matches any entry in that table while removing it from memory and the cache?
Comment 23 David :Bienvenu 2010-09-03 07:42:03 PDT
Not to steal jcranmer's thunder, but I believe he's implementing the deletion of the local db hdr for newsgroup messages.
Comment 24 irneb 2010-09-07 22:47:53 PDT
This is NOT AN ENHANCEMENT!!!!!

It's a SECURITY RISK!!!!! Since you are unable to get rid of SPAM!!!!!

Please wake UP people! Due to this one single bug I will not use TB as a newsgroup client. Either drop that ability from TB in totality (seeing as no-one wants to fix this for more than 5 years now). Or fix the bug!

Explanation of how it MUST be fixed (preferred Option 2 below):

_Option 1_
When "Deleting" the message, check if sender is same as yourself (as it's doing already). If so do delete on local & cancel sent to server. If not ONLY do DELETE on LOCAL .... DON'T SEND CANCEL TO SERVER!

_Option 2_
Split Delete and Cancel as 2 separate functions (as they actually are 2 separate actions). Cancel only sends to the server, doesn't update local copy at all - only when the message is re-downloaded does it get handled in the same method as when someone else canceled their message. Delete ONLY affects the local copy, nothing is sent to the server AT ALL!
Comment 25 irneb 2010-09-07 22:50:48 PDT
And if the issue is Online-only folders, then have the Delete cause a "HIDE" in that header. Come one people, what's the big "PROBLEM" with doing something this SIMPLE? It shouldn't take even your lowliest programmer more than a day to fix this!
Comment 26 Ludovic Hirlimann [:Usul] 2010-09-08 03:54:46 PDT
Id it's this simple, please provide a patch.
Comment 27 Joshua Cranmer [:jcranmer] 2010-09-10 07:25:06 PDT
Created attachment 474036 [details] [diff] [review]
Patch, version 1

What this patch does is to move the code to cancel messages into a new IDL-accessible method on nsIMsgNewsFolder, and more or less copies the code for local folder deletion in the shift-delete case to DeleteMessage. And then I get the UI code to actually allow news to delete messages and move messages (there are so many layers of checks in there... o_O).

Requesting sr from Neil since this touches suite UI and the suite review requirements suggest that I need an sr for this as a result, as well as the addition to the IDL API.
Comment 28 Joshua Cranmer [:jcranmer] 2010-09-10 07:28:05 PDT
Oh yeah, I forgot to mention.

In the UI, I move Cancel Message to the Message menu from the Edit Menu, which means that localizers may need to change the access keys for specific locales. When I talked to sipaq, he suggested:
07:56 < sipaq> jcranmer: Here's what I would do:
07:56 < sipaq> Change the accesskey, but don't change the entity name
07:56 < sipaq> CC thunderbird@localization.bugs on your bug and ask for  localizer feedback there
07:57 < sipaq> Post to mdl about it and ask for feedback and post there again  when you commit your patch
07:57 < sipaq> You can tell people that I told you to not change the accesskey  entity name. Maybe I'm wrong ;)

So, if any localizers have feedback, please tell me now.
Comment 29 David :Bienvenu 2010-09-16 08:44:38 PDT
Comment on attachment 474036 [details] [diff] [review]
Patch, version 1

This will make a lot of people happy.

no need for braces here:

+      if (msgHdr)
+      {
+        rv = mDatabase->DeleteHeader(msgHdr, nsnull, PR_TRUE, PR_TRUE);
+      }
Comment 30 Rimas Kudelis 2010-09-19 02:38:55 PDT
(In reply to comment #28)
> So, if any localizers have feedback, please tell me now.

Here's some feedback: you can of course leave the entities as they are since this isn't really a semantic change. However, I think it's totally acceotable to rename them, because this is a context change which may require some more tweaking than just accesskey (e.g. grammatical cases/forms).

In any case, a message to mdl will be very welcome!
Comment 31 neil@parkwaycc.co.uk 2010-09-29 08:02:28 PDT
Comment on attachment 474036 [details] [diff] [review]
Patch, version 1

>-			     	if ( command == "cmd_delete" ) {
>-					      goSetMenuValue(command, 'valueNewsgroup');
>-				    	  goSetAccessKey(command, 'valueNewsgroupAccessKey');
>+            if ( command == "cmd_delete" ) {
>+              goSetMenuValue(command, 'valueNewsgroup');
>+              goSetAccessKey(command, 'valueNewsgroupAccessKey');
Is this just an indentation change?

>+        message.folder.QueryInterface(Components.interfaces.nsIMsgNewsFolder);
>+        message.folder.cancelMessage(message, msgWindow);
Nit: I don't mind calling QueryInterface on a JS variable to fake out xpconnect, but when it's on an xpconnect property I'm not so sure...
message.folder.QueryInterface(Components.interfaces.nsIMsgNewsFolder)
              .cancelMessage(message, msgWindow);
would be fine though.

> function MsgDeleteSelectedMessages(aCommandType)
> {
>-    // we don't delete news messages, we just return in that case
>-    if (isNewsURI(gSearchView.getURIForViewIndex(0))) 
>-        return;
>-
>     // if mail messages delete
Nit: don't need this comment line either.
Comment 32 Joshua Cranmer [:jcranmer] 2010-09-29 09:56:32 PDT
(In reply to comment #31)
> Comment on attachment 474036 [details] [diff] [review]
> Patch, version 1
> 
> >-			     	if ( command == "cmd_delete" ) {
> >-					      goSetMenuValue(command, 'valueNewsgroup');
> >-				    	  goSetAccessKey(command, 'valueNewsgroupAccessKey');
> >+            if ( command == "cmd_delete" ) {
> >+              goSetMenuValue(command, 'valueNewsgroup');
> >+              goSetAccessKey(command, 'valueNewsgroupAccessKey');
> Is this just an indentation change?

Yes, I couldn't get hg to forget about the whitespace change.
Comment 33 Joe Sabash [:JoeS1] 2010-10-20 16:46:58 PDT
With the latest proliferation of spam on news.mozilla.org (from googlegroups), this would certainly help to "clear the air"
Comment 34 neil@parkwaycc.co.uk 2010-11-21 07:19:04 PST
Comment on attachment 474036 [details] [diff] [review]
Patch, version 1

>+      case "cmd_cancel": {
>+        let selectedMessages = gFolderDisplay.selectedMessages;
>+        return selectedMessages.length == 1 && selectedMessages[0].folder &&
>+               selectedMessages[0].folder.server.type == "nntp";
return gFolderDisplay.selectedCount() == 1 &&
       gFolderDisplay.selectedMessageIsNews();

>+        let message = gFolderDisplay.selectedMessages[0];
selectedMessage suffices, no?

>+        message.folder.QueryInterface(Components.interfaces.nsIMsgNewsFolder);
>+        message.folder.cancelMessage(message, msgWindow);
I know XPConnect probably caches the fact that it's a news folder but I'd prefer it collapsed into a single statement please.

>+  <command id="cmd_cancel" oncommand="goDoCommand('cmd_cancel')"/>
This isn't in the edit menu so its command doesn't belong in the edit menu items commandset.

>+    <menuitem id="menu_cancel"
>+              label="&cancelNewsMsgCmd.label;"
>+              accesskey="&cancelNewsMsgCmd.accesskey;"
>+              command="cmd_cancel"/>
I think I'd prefer to see this next to the other news-only menuitems.
Comment 35 Joshua Cranmer [:jcranmer] 2010-11-24 07:31:26 PST
Created attachment 492998 [details] [diff] [review]
Patch, version 2

Updated with Neil's comments. Carrying r+ over from last patch.
Comment 36 Joshua Cranmer [:jcranmer] 2010-11-24 09:46:00 PST
Pushed as changeset 113880e9f043.
Comment 37 Joe Sabash [:JoeS1] 2010-11-24 17:34:11 PST
Nice Thanksgiving gift, Thanks Joshua. Have a great holiday.
Mozilla/5.0 (Windows NT 5.0; rv:2.0b8pre) Gecko/20101124 Thunderbird/3.3a2pre ID:20101124115027
Comment 38 Jens Hatlak (:InvisibleSmiley) 2011-01-04 08:31:08 PST
(In reply to comment #28)
> Oh yeah, I forgot to mention.
> 
> In the UI, I move Cancel Message to the Message menu from the Edit Menu

He, I already thought there was a bug in the context menu entry enabling code! I'm not at all opposed to the move, but maybe that change deserves a relnote? (Note: Both TB and SM are affected here.)
Comment 39 Todd 2011-01-23 18:52:22 PST
This says "fixed".  Can anyone tell me when to expect this patch?  I have a psychopath I need to kill file on a newsgroup.
Comment 40 Chris Ilias [:cilias] 2011-01-23 18:59:51 PST
You can test it in the latest Alpha release at <http://www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/>.
Remember to read the release notes at <http://www.mozillamessaging.com/en-US/thunderbird/3.3a2/releasenotes/>.


If you're not into testing pre-releases, the final release of version 3.3 is still yet to be determined. But there would presumably some Beta releases after the Alphas, so I would say "many months".
Comment 41 Philip Chee 2011-03-03 20:44:40 PST
*** Bug 638433 has been marked as a duplicate of this bug. ***
Comment 42 David :Bienvenu 2011-07-14 08:58:39 PDT
You need to toggle this hidden boolean pref news.allow_delete_with_no_undo to true in order to delete news messages. You can do this with the config editor (tools, options, advanced, general, config editor)
Comment 43 Jens Hatlak (:InvisibleSmiley) 2011-07-14 09:44:06 PDT
(In reply to comment #42)
> You need to toggle this hidden boolean pref news.allow_delete_with_no_undo
> to true in order to delete news messages.

Note that you will probably run into bug 660845 then (haven't check TB yet but I think it's a shared/back-end issue, see bug 660845 comment 6 about gCopyService.CopyMessages).
Comment 44 Lu Wei 2011-11-06 05:56:22 PST
Could it be considered "fixed"? Who knows that hidden pref "news.allow_delete_with_no_undo"?
Comment 45 Todd 2011-11-06 12:13:09 PST
(In reply to LU Wei from comment #44)
> Could it be considered "fixed"? Who knows that hidden pref
> "news.allow_delete_with_no_undo"?

I would not  consider it fixed.  "news.allow_delete_with_no_undo" is only less obscure then actually locating the configuration edition.  Needs to be something easily locatable for the user.  Googl'ing one's fingers off should not be a requirement to enable this feature

Note You need to log in before you can comment on or make changes to this bug.