1271189 - Not all Junk messages moved to Junk folder (When emptying Junk folder, training data seems to be reset)

Status: UNCONFIRMED

(MailNews Core :: Filters, defect)

Description

[Martin Dürst]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36 OPR/37.0.2178.32

Steps to reproduce:

Emptied the "Junk" folder using the "Empty Junk" function from the popup menu.


Actual results:

Junk filtering got a lot worse. Before, the percentage of junk mails that I had to select manually was maybe 5%. After, it started close to 100%.
Looking at the "training.dat" file in the profiles folder, that's 3GB, so it's not that all the training data is gone, but it very much feels like it's gone.
This has happened every time I have emptied the junk folder, which is about a couple times per year.


Expected results:

I would expect that the precision of the junk filter stays the same, because "Empty Junk" is a problem of space (I was told that my Junk folder is completely full and I have to empty it), and should not remove training data.

Comment 1

[Wayne Mery (:wsmwk)]

(In reply to Martin Dürst from comment #0)
> User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.87 Safari/537.36 OPR/37.0.2178.32
> 
> Steps to reproduce:
> 
> Emptied the "Junk" folder using the "Empty Junk" function from the popup
> menu.
> 
> 
> Actual results:
> 
> Junk filtering got a lot worse. Before, the percentage of junk mails that I
> had to select manually was maybe 5%. After, it started close to 100%.
> Looking at the "training.dat" file in the profiles folder, that's 3GB,

3GB? Or 3MB?

Comment 2

[Martin Dürst]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #1)
> 3GB? Or 3MB?

Very sorry, 3MB indeed. Or 3,646KB to be more exact. Thanks for catching this!

Comment 3

[Wayne Mery (:wsmwk)]

3MB is reasonable.

I don't believe we've ever had someone report that Junk processing gets altered by emptying Junk.
In your case, do you ever see the same problem if you always restart thunderbird after emptying Junk?

Comment 4

[Martin Dürst]

I cannot comment on "ever ... always" because it was only this comment that let me try to see what happens if I restart Thunderbird after emptying Junk. On the single occasion that I did it, it indeed helped.

I'm attaching a file with data that I took over several months. Each line shows a date (yyyy/mm/dd), a colon, and then three numbers separated by slashes.

The first number is the number of mails downloaded the first time I use "Get Messages" that day. The second number is the number of junk mails that get moved to the Junk folder automatically immediately after the download. The third is the number of mails that I have to move to the Junk folder manually in that set of mails. Sometimes, there are ?? instead of numbers. That's when I occasionally just forgot to take a note, or when I wasn't able to see the number because it was displayed only for an extremely short time. This doesn't cover all of my mail, it was just the easiest way to get some reasonably significant data.

The data starts on 2016/05/11, a few days after I did a Junk cleanout, and a couple days after I created this bug. It shows that false negatives are at that point just about the same magnitude as correct positives (if in line with medicine, we call junk "positive"). This is very similar to what I experienced after every other previous Junk cleanout. The false negative numbers get smaller and smaller. (In the original report, I said it started close to 100% of Junk that I had to select manually; the Junk filter indeed improved most quickly in the first few days after that.)

Then on 2016/11/05, I did another Junk cleanout, and immediately afterwards closed Thunderbird, as suggested in Comment 3. This time, the numbers are definitely much better. My subjective impression was also much better, although I noticed a few types of mails that I consistently classified as Junk and that didn't remain in the Inbox before this Junk cleanout turned up a couple of times in the Inbox after this Junk cleanout.

So this suggests that there may indeed be a correlation between whether Thunderbird is closed immediately after a Junk cleanout and whether (and to what extent) the Junk learning data is kept.

But I'd have to ask: Why am I supposed to have to close Tunderbird immediately after a Junk cleanout to keep my learned Junk data? I'd still say this is a bug.

Comment 5

[Martin Dürst]

Attachment: Spam filter count data as described in comment #4

Please tell me if you think it makes sense to continue this data sampling. I'd be willing to do it if it helps solve this bug, but otherwise, I'd be glad to no longer have to do it.

Comment 6

[Wayne Mery (:wsmwk)]

I don't think you need to continue sampling. I don't have a cause or solution for you.  

We could perhaps try an older version of Thunderbird.  How long have you  been running Thunderbird?  And how long have you been seeing the problem?

Similarly odd report at https://support.mozilla.org/en-US/questions/1147607 where not all the messages are being scored.
See http://mesquilla.com/2009/11/16/junquilla-version-1-0-0-released/

Comment 7

[Wayne Mery (:wsmwk)]

See also bug 324777. But I'm not sure it's totally relevant.

Comment 8

[Martin Dürst]

"I don't have a cause or solution for you.": Well, closing Thunderbird immediately after junk cleanout seems to have worked reasonably well, so it's at least half of a solution.

I have been using Thunderbird since Spring 2009, originally the Eudora/Penelope version.

I'm only cleaning out junk once or twice a year, and the problem has bugged me many times. But it's difficult to remember five or more years past, so I'm not exactly sure. It may have been there from when I started using Thunderbird.

I also occasionally get something like https://support.mozilla.org/en-US/questions/1147607, but that's usually associated to the download being cut off before finishing, or something similar. And it doesn't repeat, and isn't in any way related to junk cleanout. So I think it's a completely different problem.

I also don't remember having seen anything similar to bug 324777, although I'm not sure, because I make sure I hide the junk icon column to avoid clicking it erroneously.

Comment 9

[Wayne Mery (:wsmwk)]

Martin,

Have you seen the problem in 2017 or 2018?

Comment 10

[Martin Dürst]

(In reply to Wayne Mery (:wsmwk) from comment #9)

> Have you seen the problem in 2017 or 2018?

Not that I remember. But that's a very small sample size, I cleaned out my Junk folder maybe two times in 2017, and not yet in 2018. Also, I was careful to close Thunderbird immediately after that, because that was recommended. That seemed to have worked, at least to some extent.