375717 - New mail notification (mail alert/preview text) shows old items

Status: RESOLVED FIXED

(Thunderbird :: Mail Window Front End, defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: new mail notification

Look into the attachment. When I get a mail into the global inbox, the new mail notification popup shows four other items and not the new one. Furthermore, these other items are stored in a subsubfolder of the global (local) account.

Comment 1

[Mike Cowperthwaite]

Basically a dupe of bug 138631.

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

No, the number is correct, but the window shows rss items in subsubfolders of "Local Folders" (not of the inbox).

Comment 4

[Bill Dimm]

This seems to be related to bug 377301.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I believe this is still happen for Thunderbird 3 nightly builds. Anyone tested it yet?

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

version 3.0a1pre (2008040903)

Yes, still exists. Nominating for blocking-thunderbird3.

Comment 10

[Nikolay Shopik]

TB thinks its new mail until you open folder where new mail comes or change to other folder if you in folder where new mail comes.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.2pre) Gecko/2008072303 Shredder/3.0a2pre.

Comment 11

[Mark Banner (:standard8)]

Triaging according to policy for flags.
https://wiki.mozilla.org/Thunderbird:Release_Driving

Comment 13

[David Humphrey (:humph)]

Attachment: Untested patch based on fix for bug 459483

If someone with a win32 build env could try this for me, that would be great.

Comment 14

[Nikolay Shopik]

David, why do we need especially someone with win32 build env?

Comment 15

[David Humphrey (:humph)]

(In reply to comment #14)
> David, why do we need especially someone with win32 build env?

I've fixed this on Mac (see bug 459483), and wanted to do it on Windows too.  I thought this bug was Windows only, so my mistake.  However, this fix touches nsMessengerWinIntegration, so Windows ;)

The reason this bug occurs is that we currently cache the root folder which is getting the biff notification, and when the second/third/etc. one comes in, it uses mFoldersWithNewMail[0] (i.e., the first folder that got biff) every time vs. the folder reporting new mail.

Comment 16

[Kent James (:rkent)]

I tested the patch 348817, and it compiled fine, but neither from the bug nor
from IRC discussions with humph was I able to reproduce the problem this bug is
trying to solve without the patch applied. Some clear steps to reproduce would
be very helpful.

Comment 17

[Nikolay Shopik]

Kent,
Just get new mail - > new mail notification appears
get another new mail -> new mail notification appears showing alongside with first message.
You can clear counter by changing folders and come back.

Comment 18

[Kent James (:rkent)]

(In reply to comment #17)
> Kent,
> Just get new mail - > new mail notification appears
> get another new mail -> new mail notification appears showing alongside with
> first message.
> You can clear counter by changing folders and come back.

This is not the problem that humph described to be he was trying to solve. So, Nikoly, I send email "showme 1", receive it, biff shows "showme 1". I send another email, "showme 2", receive it, biff now appears with both "showme 1" and "showme 2". And your desired behavior is that the second notification only shows "showme 2" - is that correct?

Comment 19

[Nikolay Shopik]

Exactly. Just email that arrive right now.

Comment 20

[Kent James (:rkent)]

Nikolay, the behavior that you describe is one of the reasons for my proposals in https://wiki.mozilla.org/User:Rkentjames:NewCounts  Basically, we need a new flag, described there as 'A new flag MSG_FLAG_BIFF will be defined as "This newly received message has not completed a BIFF notification cycle"' Now that bienvenu and I seem to have resolved some issues associated with the backend for such processing flags, perhaps we need to reopen the questions in that proposal. Bienvenu seems unconvinced of the need for this flag (and I will admit I am able to do more under the current model than I previously thought possible) though I should let him speak for himself (hence my shameless misrepresentation which always elicts a reaction).

Comment 21

[Magnus Melin [:mkmelin]]

Note that we have a nsMessengerUnixIntegration.cpp too

Comment 22

[David Humphrey (:humph)]

Sure, the code is the same (see bug 458507).  I'll do that too, as soon as someone who reported or experiences this can tell me whether this fixes their issue on Windows.

Comment 23

[Bill McGonigle (not currently reading bugmail; please contact directly)]

(In reply to comment #21)
> Note that we have a nsMessengerUnixIntegration.cpp too

To clarify - did you mean there's a patch somewhere that fixes this for Unix or that we need a patch that also fixes it for Unix?  I'm seeing such errant behavior on linux (I've got certain 'special' IMAP folders that always show their old messages in the biff list despite no UI-visible differences).

Comment 24

[David Humphrey (:humph)]

fwiw, I've fixed this (and a bunch of related bugs) already for Mac, and all the nsMessenger*Integration implementations share the same set of bugs.  I'm not sure the best way to get the changes across all platforms (bug-by-bug or all at once in a new bug), but I'll look into it.

Comment 25

[Jed Brown]

I think I'm seeing the same thing.  I have a bunch of IMAP folders and there will be unread mail sitting in some of them.  I expect notifications when I get new mail in INBOX, bot not the others.  This seems to work fine except that the summary in the alert shows unread mail in other folders.  If I restart TB, it works fine until I visit one of these other folders at which point the summary will be wrong for the rest of the session.

Comment 26

[Magnus Melin [:mkmelin]]

humph: any progress? If not, i might take a look.

Comment 28

[David Humphrey (:humph)]

I might have time for this next week, actually.  Let me take a look today.  What's really involved here is moving Windows toward fixing bug 458507, which will fix this and a bunch of other things.  We've had this fixed on Mac for a while.

Comment 29

[Magnus Melin [:mkmelin]]

Assigning to you for now then:)

Comment 34

[[:Aureliano Buendía]]

Request wanted-thunderbird3.1.

Comment 36

[Nikolay Shopik]

dupe of Bug 226885 ?

Comment 38

[Nikolay Shopik]

(In reply to comment #18)
> This is not the problem that humph described to be he was trying to solve. So,
> Nikoly, I send email "showme 1", receive it, biff shows "showme 1". I send
> another email, "showme 2", receive it, biff now appears with both "showme 1"
> and "showme 2". And your desired behavior is that the second notification only
> shows "showme 2" - is that correct?

I just noticed that behavior was changed, you now only notified for first mail and not notification for second, etc. xref 627333
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b10pre) Gecko/20110117 Thunderbird/3.3a3pre

Comment 40

[Ken]

So, I am new to Thunderbird, and apparently this "bug" has been going on since 2007?  Does that mean that Mozilla doesn't follow up on bug reports from their customers ta all then?  

My new mail notification also shows more than the new email - sometimes messages that have been moved to the trash.  

I am using Gmail, IMAP with the latest version of Thunderbird Beta 10.  

It would be nice to see this fixed.

K

Comment 41

[Erdin Kacan]

Year 2013....
5 years ! Come on...It might not be so important but its a very visible and kind of frustrating one.And i don't think any bug takes 5 years to fix.I know you guys are busy with a lot of others things probably I'm a programmer too i understand a bit.But 5 years for a so small bug...

Comment 43

[Mark]

Is the patch still relevant? I'm still getting this in Mountain Lion, Thunderbird 24.3.0.

In the settings there is nothing for the notification window. When it pops up I get useless info. The subject line is for an unread email, but not the one that just arrived. It also tells me I have 2000 odd new messages, which is useless.

I want to see the subject/from for the email that just arrived and how many unread in my inbox/unified inbox.

Comment 44

[6e77a 70 6e 6e7a]

This has been happening to me and driving me crazy for years. Please fix!

Comment 45

[ultimitloozer]

Opened when TB3 was current? Here we are with TB38 and the damn thing is still there? 8.5 years without a resolution. Approaching the Microsoft level of efficiency.

Comment 46

[Mikhail T.]

To the developers trying to reproduce the problem: you must have multiple mailboxes configured.

Although Thunderbird properly notices e-mails arriving to all of them, what the new-mail popup often lists are already-reported (but still unread) messages, instead of the one(s) that just arrived and, actually, triggered the popup in the first place.

Instead it should show the very latest arrivals, obviously. Using 38.4.0 here and this is still a (very annoying) problem.

Comment 47

[Mark]

(In reply to Mikhail T. from comment #46)
> To the developers trying to reproduce the problem: you must have multiple
> mailboxes configured.
> 
> Although Thunderbird properly notices e-mails arriving to all of them, what
> the new-mail popup often lists are already-reported (but still unread)
> messages, instead of the one(s) that just arrived and, actually, triggered
> the popup in the first place.
> 
> Instead it should show the very latest arrivals, obviously. Using 38.4.0
> here and this is still a (very annoying) problem.

Me too. Multiple mailboxes configured, all are IMAP. The New Email popup occurs when a new delivery is received to any of my mailboxes but instead of showing the newest email received, it shows the first four 'unread' emails in any of my mailboxes (well, one is dominant).  Every. Single. Time. So aside from being aware i've received a new message, it serves no purpose (and i'm tempted to turn the whole feature off).  This has been ongoing for a long time.

Comment 48

[Jeff Spitler]

The last few comments are correct.  It should show the last four unread messages, not the first four.

Comment 49

[Mikhail T.]

> It should show the last four unread messages

Not quite what I said. It should show _all_ of the messages, that actually triggered the new-mail notification in the first place -- even if there are other unread messages in the mailbox.

For example, suppose there are 5 unread messages. Thunderbird performs its periodic check and detects 2 new ones. Only those two should be included in the notification. And it must properly handle the case, when new messages are detected in multiple accounts...

Comment 50

[Jeff Spitler]

OK - but I would be happy if it showed only the new ones OR the last four unread messages - and not the first four unread messages.

Comment 51

[ultimitloozer]

(In reply to Mikhail T. from comment #49)

> And it must properly handle the case, when new messages
> are detected in multiple accounts...

Each of my 7 accounts currently gets its own popup for notifications, so this should not be an issue.

Comment 53

[Thomas D. (:thomas8)]

OMG!!! ME TOO!!! Let me reserve my comment... $§%&*!!!!

Notifications have become totally useless because of this bug, and it's extremely irritating. Each time when it says "1 new message", I'm trying to identify that "new" message in the notification list which shows me 5, and more often than not it's not even there, and I have no way of identifying it between the other, unrelated notified messages. So it appears that the notification is never up to date and always showing stale messages.
Classic case of "useless-UI" and "ux-implementation-level" because this has been wrongly designed around the implementation in illogical ways.

Can we please fix this ASAP because this bug significantly impacts the overall usefulness of the application, more so for enterprise contexts where seeing new stuff at a glance may matter even more. I think it's just that it's a bit hard to nail down this bug in a volatile notification and to realize it's actually broken, otherwise we'd have way more duplicates. To be clear, even though this doesn't break things, it does make us look pretty bad and has the potential to create unconscious vibes of frustration with our product.

Comment 54

[Magnus Melin [:mkmelin]]

This is part of the the cluster of notification bugs. Not sure about the exact relationship between them all but many are more or less duplicates of each other. There's some back-end things to clear up first though.

Comment 56

[Ping Chen [:rnons] (away)]

Attachment: 375717-num-new.patch

Found that getNewlist returns all unread messages, while getNumNewMessages returns count of new messages since the last biff. Tested on Linux, will test on Windows later.

Comment 57

[Magnus Melin [:mkmelin]]

Comment on attachment 9204515 [details] [diff] [review]
375717-num-new.patch

Review of attachment 9204515 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/base/content/foldersummary.js
@@ +136,5 @@
>  
>          folder.msgDatabase = null;
>          let msgKeys = msgDatabase.getNewList();
>  
> +        // NOTE: getNewlist returns all unread messages, while getNumNewMessages

unread != NEW. 
NEW is imap UNSEEN. getNewList lists NEW.

Does getNumNewMessages also consider the things messages we got through IDLE?

Comment 58

[Ping Chen [:rnons] (away)]

Attachment: 375717-num-new.patch

Fixed comment.

Does getNumNewMessages also consider the things messages we got through IDLE?

Yes, tested new messages on start and on idle. Works on Windows as well.

Comment 59

[Magnus Melin [:mkmelin]]

Comment on attachment 9204732 [details] [diff] [review]
375717-num-new.patch

Review of attachment 9204732 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, r=mkmelin

Comment 60

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/9e48bb88fbd4
Only show newly received messages in notification. r=mkmelin

Comment 61

[Ping Chen [:rnons] (away)]

Comment on attachment 9204732 [details] [diff] [review]
375717-num-new.patch

[Approval Request Comment]
Regression caused by (bug #): Implemented like this years ago.
User impact if declined: When the notification window shows up on receiving a new mail, the notification actually shows old mails.
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): Impacts windows notification, and linux notification when system notification is not available or turned off.

Comment 62

[Wayne Mery (:wsmwk)]

Comment on attachment 9204732 [details] [diff] [review]
375717-num-new.patch

[Triage Comment]
approved for beta

Comment 63

[Rob Lemley [:rjl]]

Thunderbird 87.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/98ca31631af5

Comment 64

[Thomas D. (:thomas8)]

Thanks a lot Ping for fixing this perennial annoyance!

Comment 68

[Wayne Mery (:wsmwk)]

I'm thinking to not take this for 78.8.1, and just take bug 210148, so pick this up in the next point release.

Comment 69

[Thomas D. (:thomas8)]

(In reply to Wayne Mery (:wsmwk) from comment #68)

I'm thinking to not take this for 78.8.1, and just take bug 210148, so pick this up in the next point release.

Why? This one-line patch does not look risky at all. It's just reducing the set of messages shown in the notification, and reverting their order. Nothing else has changed, so the only "impact" on notfications is finally showing the correct set...
Even if it went wrong, it can't get worse than now. Right now, the notification is totally useless more often than not.
Imho we should roll it out now to fix this useless UI asap, and stop users' suffering.

msgKeys = msgKeys.slice(-folder.getNumNewMessages(false));

Kindly asking for reconsideration for 78.8.1.

Comment 70

[Wayne Mery (:wsmwk)]

Not about risk, but just giving users one bite at a time, so that we can more easily gauge feedback. And ASAP is pretty moot, given it's a 14 year old bug.

Comment 71

[Mikhail T.]

BTW, currently the new-mail notification does not have the close-button -- that little "X" in the screenshot found in ticket-description

is absent here (X11, using XFCE4). When the notice comes up, there is nothing to do but wait for it to go away on its own, which can get quite annoying, when it obscures the part of the screen I was just looking at.

Should I file a separate bug-report, or can that be fixed under this one too?

Comment 72

[Ping Chen [:rnons] (away)]

(In reply to Mikhail T. from comment #71)

BTW, currently the new-mail notification does not have the close-button -- that little "X" in the screenshot found in ticket-description

is absent here (X11, using XFCE4). When the notice comes up, there is nothing to do but wait for it to go away on its own, which can get quite annoying, when it obscures the part of the screen I was just looking at.

Should I file a separate bug-report, or can that be fixed under this one too?

Please file a new bug with a screenshot. I can see the "X" button on GNOME.

Comment 73

[Mikhail T.]

(In reply to Ping Chen (:rnons) from comment #72)

Please file a new bug with a screenshot. I can see the "X" button on GNOME.

Done: Bug 1696578.

Comment 74

[Wayne Mery (:wsmwk)]

Comment on attachment 9204732 [details] [diff] [review]
375717-num-new.patch

[Triage Comment]
Approved for esr78

Comment 75

[Rob Lemley [:rjl]]

Thunderbird 78.9.0:
https://hg.mozilla.org/releases/comm-esr78/rev/f54fc40959d0