Open Bug 885220 Opened 6 years ago Updated 2 years ago

no new mail notification (alert or icon) on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE)

Categories

(MailNews Core :: Networking: IMAP, defect)

x86
Windows XP
defect
Not set

Tracking

(thunderbird_esr2426+)

Tracking Status
thunderbird_esr24 26+ ---

People

(Reporter: al_9x, Unassigned)

References

(Blocks 1 open bug, )

Details

(Keywords: regression, Whiteboard: [workaround: comment #73])

Attachments

(1 file)

This started in the main profile but I can repro in a new profile, default settings.

1. start with new profile
2. add a gmail imap account
3. after it downloads and indexes, restart
4. send an email to it
5. no tray icon, no alert, inbox shows the new message
tb 17.0.6, xp sp3 (also in a clean xp vm)

reproducible for a newly created and added to a new profile gmail account.

step #3 (restart after sync and indexing) is needed to repro, alerts and icon are shown before restarting
There have been a couple of forum reports on this happening recently, mostly on Windows 7 platforms. your report is similar to bug 727790 but seems to be more specific to Gmail accounts, and you also mention indexing being necessary to trigger the issue which is contrary to my experience given that I'm not running the global indexer (and in the case of SeaMonkey, it doesn't even exist). Thus, let's keep the two bug reports in parallel for now in case anything is different between those two instances.
See Also: → 727790
I am not suggesting that indexing triggers it, but that *restarting* tb after the initial sync and indexing after account creation is necessary to repro in a new profile, restarting is the key, indexing is parenthetical
Summary: no new mail alert or icon on gmail imap accounts (new profiles) → no new mail alert or icon on gmail imap accounts (new profile)
I see, I may have misunderstood then. The problems reported so far occurred sporadically with existing profiles, and there are no clear steps to reproduce, thus any factor possibly prompting the issue certainly is of interest in narrowing down the cause.
Here's the imap log of the receipt of an email that does not produce a new mail alert, who is at fault here, gmail or tb?
______________________________________________________________________
ReadNextLine [stream=8fff280 nb=12 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 3 EXISTS

86f0800:imap.googlemail.com:S-INBOX:SendData: DONE

ReadNextLine [stream=8fff280 nb=33 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 14 OK IDLE terminated (Success)

86f0800:imap.googlemail.com:S-INBOX:ProcessCurrentURL: entering
86f0800:imap.googlemail.com:S-INBOX:ProcessCurrentURL:imap://{{email}}@imap.googlemail.com:993/select%3E/INBOX:  = currentUrl
86f0800:imap.googlemail.com:S-INBOX:SendData: 15 noop

ReadNextLine [stream=8fff280 nb=15 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 15 OK Success

86f0800:imap.googlemail.com:S-INBOX:SendData: 16 getquotaroot "INBOX"

ReadNextLine [stream=8fff280 nb=24 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" ""

ReadNextLine [stream=8fff280 nb=33 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTA "" (STORAGE 5 15728640)

ReadNextLine [stream=8fff280 nb=15 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 16 OK Success

86f0800:imap.googlemail.com:S-INBOX:SendData: 17 UID fetch 19:* (FLAGS)

ReadNextLine [stream=8fff280 nb=43 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 3 FETCH (UID 19 MODSEQ (4744) FLAGS ())

ReadNextLine [stream=8fff280 nb=15 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 17 OK Success

86f0800:imap.googlemail.com:S-INBOX:SendData: 18 UID fetch 19 (UID X-GM-MSGID X-GM-THRID X-GM-LABELS RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

ReadNextLine [stream=8fff280 nb=283 needmore=0]
86f0800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 3 FETCH (X-GM-THRID {{19 digit id}} X-GM-MSGID {{19 digit id}} X-GM-LABELS ("\\Important") UID 19 RFC822.SIZE 1721 MODSEQ (4744) FLAGS () BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)] {277}

86f0800:imap.googlemail.com:S-INBOX:STREAM:OPEN Size: 1721: Begin Message Download Stream
_________________________________________________________________
Can a tb dev please comment on the above imap log?
WADA, do you see anything suspicious in this log, or any other section we'd want al_9x to provide from the log?
(In reply to rsx11m from comment #7)

As for new message in log pasted in comment #5, the new message of "UID=19" doesn't look notified via IDLE. Because "/select%3E/INBOX:  = currentUrl" and "15 noop" is seen before "17 UID fetch 19:* (FLAGS)", it looks done by "check new messages every NN minutes".
So, "No \Recent flag" may be reason.

Is "mail of no \Recent flag" alerted by Biff of Tb?

A reason why "no \Recent flag" may be;
  1. The mail is added to "[Gmail]/All Mail" and Gmail Label of
     "[Gmail]/Sent" is added by Gmail's SMTP.
  2. Same mail data is appended8uploaed9 to [Gmail]/Sent by Tb's "sent
     mail copy to sent after mail send".
  3. Gmail Label of "Inbox" is added to same mail, because To: or CC:
     or BCC: contains mail address assigned to this Gmail account.
But it's uncertain, because "what operation was done by '4. send an email to it' in comment #0" is not stated at anywhere of this bug.
Thus, the assumption is that the notification was prompted by an IDLE (which apparently didn't work) and later to "\Recent" is sent based on the assumption that the IDLE has been processed already, thus should have triggered the biff? In that case, disabling IDLE for the server might be a workaround.

BTW: Don't be confused by "[Gmail]/Sent" for own messages, those don't prompt a biff unless you are sending the message to yourself (and the issue also happens with messages received from others = unread mail count increases in folder list but no biff is issued - see bug 727790).
(In reply to WADA from comment #8)
> (In reply to rsx11m from comment #7)
> 
> As for new message in log pasted in comment #5, the new message of "UID=19"
> doesn't look notified via IDLE. Because "/select%3E/INBOX:  = currentUrl"
> and "15 noop" is seen before "17 UID fetch 19:* (FLAGS)", it looks done by
> "check new messages every NN minutes".
> So, "No \Recent flag" may be reason.
> 
> Is "mail of no \Recent flag" alerted by Biff of Tb?
> 
> A reason why "no \Recent flag" may be;
>   1. The mail is added to "[Gmail]/All Mail" and Gmail Label of
>      "[Gmail]/Sent" is added by Gmail's SMTP.
>   2. Same mail data is appended8uploaed9 to [Gmail]/Sent by Tb's "sent
>      mail copy to sent after mail send".
>   3. Gmail Label of "Inbox" is added to same mail, because To: or CC:
>      or BCC: contains mail address assigned to this Gmail account.
> But it's uncertain, because "what operation was done by '4. send an email to
> it' in comment #0" is not stated at anywhere of this bug.

For this specific experiment the mail was being sent from a separate non gmail account, however it does not matter how it's sent or from whom, all my gmail accounts including a new test one have this problem.

So if I understood you correctly the problem is with gmail (missing recent flag on new messages).  The possible reasons you mentioned don't seem to apply.

Is this a gmail bug?  Can you repro with a new account you create?  Would it help if I gave you access to my test account that repros the bug?  Can mozilla's google contact be asked about this?
as for idle, in all my experiments, the log is written to almost immediately after mail is sent, this does not look like polling.  I suppose there could have been a coincidental poll at the moment I sent, I can try this again with polling off.
FYI.
When new mail is notified via IDLE, Tb doesn't show Biff alert, because, if Biff alert is shown when "check for new message every NN miutes" is disabled, it produces user's confusion, and such confusion produces unwanted bug open by general users.
However, if new mail check occurs at same time when "notification via IDLE" occurs, and if new mail is detected by the periodical new mail check, alert by Biff is shown and "mails detected via IDLE" is included in the new mail alert.

Please test "new mail detection by periodical new mail check" and "new mail detection via IDLE" separately.
here's another log with polling off
______________________________________________________________
ReadNextLine [stream=683c5e0 nb=10 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: + idling

ReadNextLine [stream=683c5e0 nb=0 needmore=1]
ReadNextLine [stream=683c5e0 nb=12 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 5 EXISTS

86f2800:imap.googlemail.com:S-INBOX:SendData: DONE

ReadNextLine [stream=683c5e0 nb=33 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 14 OK IDLE terminated (Success)

86f2800:imap.googlemail.com:S-INBOX:ProcessCurrentURL: entering
86f2800:imap.googlemail.com:S-INBOX:ProcessCurrentURL:imap://{{email}}@imap.googlemail.com:993/select%3E/INBOX:  = currentUrl
86f2800:imap.googlemail.com:S-INBOX:SendData: 15 noop

ReadNextLine [stream=683c5e0 nb=15 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 15 OK Success

86f2800:imap.googlemail.com:S-INBOX:SendData: 16 getquotaroot "INBOX"

ReadNextLine [stream=683c5e0 nb=24 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" ""

ReadNextLine [stream=683c5e0 nb=34 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTA "" (STORAGE 12 15728640)

ReadNextLine [stream=683c5e0 nb=15 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 16 OK Success

86f2800:imap.googlemail.com:S-INBOX:SendData: 17 UID fetch 21:* (FLAGS)

ReadNextLine [stream=683c5e0 nb=43 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 5 FETCH (UID 21 MODSEQ (5787) FLAGS ())

ReadNextLine [stream=683c5e0 nb=15 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 17 OK Success

86f2800:imap.googlemail.com:S-INBOX:SendData: 18 UID fetch 21 (UID X-GM-MSGID X-GM-THRID X-GM-LABELS RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

ReadNextLine [stream=683c5e0 nb=283 needmore=0]
86f2800:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 5 FETCH (X-GM-THRID {{19 digit id}} X-GM-MSGID {{19 digit id}} X-GM-LABELS ("\\Important") UID 21 RFC822.SIZE 1718 MODSEQ (5787) FLAGS () BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)] {277}

86f2800:imap.googlemail.com:S-INBOX:STREAM:OPEN Size: 1718: Begin Message Download Stream
______________________________________________________________________________
(In reply to WADA from comment #12)
> However, if new mail check occurs at same time when "notification via IDLE"
> occurs, and if new mail is detected by the periodical new mail check, alert
> by Biff is shown and "mails detected via IDLE" is included in the new mail
> alert.

I want to be sure I understand the intended behavior that you described, alerts for idle detected messages are show immediately on detection but only if polling is turned on?

If so, then the previous scenario (I turned off polling) would not have produced an alert anyway, however it still shows a new message with no recent flag, right?

That can't happen unless there's

a. a gmail bug
b. something else is retrieving my mail first

so is there a gmail bug?  Can you repro this issue with a new gmail account?
for this test I turned off IDLE, so the following is detected by polling, there was no alert, it appears to show no recent flag (where in the log would it appear?) as with idle detection:
________________________________________________________________________

86f2400:imap.googlemail.com:S-INBOX:ProcessCurrentURL: entering
86f2400:imap.googlemail.com:S-INBOX:ProcessCurrentURL:imap://{{email}}@imap.googlemail.com:993/select%3E/INBOX:  = currentUrl
86f2400:imap.googlemail.com:S-INBOX:SendData: 14 noop

ReadNextLine [stream=90212e0 nb=12 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 7 EXISTS

ReadNextLine [stream=90212e0 nb=15 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 14 OK Success

86f2400:imap.googlemail.com:S-INBOX:SendData: 15 getquotaroot "INBOX"

ReadNextLine [stream=90212e0 nb=24 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" ""

ReadNextLine [stream=90212e0 nb=34 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTA "" (STORAGE 17 15728640)

ReadNextLine [stream=90212e0 nb=15 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 15 OK Success

86f2400:imap.googlemail.com:S-INBOX:SendData: 16 UID fetch 23:* (FLAGS)

ReadNextLine [stream=90212e0 nb=43 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 7 FETCH (UID 23 MODSEQ (6387) FLAGS ())

ReadNextLine [stream=90212e0 nb=15 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 16 OK Success

86f2400:imap.googlemail.com:S-INBOX:SendData: 17 UID fetch 23 (UID X-GM-MSGID X-GM-THRID X-GM-LABELS RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

ReadNextLine [stream=90212e0 nb=283 needmore=0]
86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 7 FETCH (X-GM-THRID {{19 digit id}} X-GM-MSGID {{19 digit id}} X-GM-LABELS ("\\Important") UID 23 RFC822.SIZE 1722 MODSEQ (6387) FLAGS () BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)] {278}

86f2400:imap.googlemail.com:S-INBOX:STREAM:OPEN Size: 1722: Begin Message Download Stream
_______________________________________________________________________
I can reproduce with an existing Gmail account on an aurora build:

 1. Created a new profile
 2. Sent myself an e-mail, biff rings and icon shows up
 3. Restart
 4. Sent myself an e-mail, neither biff nor icon.

No "\Recent" in the IMAP log.

Let's see if I get a biff when posting this comment now...
I'll try disabling IDLE next.
for this test I turned IDLE on and set polling to 600 min, so the following must be IDLE detection, once again no recent, no alert

> Please test "new mail detection by periodical new mail check" and
> "new mail detection via IDLE" separately.

Does this satisfy your request?
_____________________________________________________________

2013-07-02 22:36:35.718000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: + idling

2013-07-02 22:38:36.453000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=0 needmore=1]
2013-07-02 22:38:36.453000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=12 needmore=0]
2013-07-02 22:38:36.453000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 EXISTS

2013-07-02 22:38:36.453000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:SendData: DONE

2013-07-02 22:38:36.593000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=33 needmore=0]
2013-07-02 22:38:36.593000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 14 OK IDLE terminated (Success)

2013-07-02 22:38:36.593000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:ProcessCurrentURL: entering
2013-07-02 22:38:36.593000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:ProcessCurrentURL:imap://{{email}}@imap.googlemail.com:993/select%3E/INBOX:  = currentUrl
2013-07-02 22:38:36.593000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:SendData: 15 noop

2013-07-02 22:38:36.718000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=15 needmore=0]
2013-07-02 22:38:36.718000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 15 OK Success

2013-07-02 22:38:36.718000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:SendData: 16 getquotaroot "INBOX"

2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=24 needmore=0]
2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" ""

2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=34 needmore=0]
2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTA "" (STORAGE 19 15728640)

2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=15 needmore=0]
2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 16 OK Success

2013-07-02 22:38:36.875000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:SendData: 17 UID fetch 24:* (FLAGS)

2013-07-02 22:38:37.093000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=43 needmore=0]
2013-07-02 22:38:37.093000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 FETCH (UID 24 MODSEQ (6692) FLAGS ())

2013-07-02 22:38:37.093000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=15 needmore=0]
2013-07-02 22:38:37.093000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 17 OK Success

2013-07-02 22:38:37.093000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:SendData: 18 UID fetch 24 (UID X-GM-MSGID X-GM-THRID X-GM-LABELS RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

2013-07-02 22:38:37.234000 UTC - 1316[746beb0]: ReadNextLine [stream=8f642e0 nb=283 needmore=0]
2013-07-02 22:38:37.234000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 FETCH (X-GM-THRID {{19 ditit id}} X-GM-MSGID {{19 ditit id}} X-GM-LABELS ("\\Important") UID 24 RFC822.SIZE 1727 MODSEQ (6692) FLAGS () BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)] {279}

2013-07-02 22:38:37.234000 UTC - 1316[746beb0]: 86f2400:imap.googlemail.com:S-INBOX:STREAM:OPEN Size: 1727: Begin Message Download Stream
Disabled IDLE and waited 10 minutes, same effect = no biff, no \Recent flags.
(In reply to rsx11m from comment #18)
> Disabled IDLE and waited 10 minutes, same effect = no biff, no \Recent flags.

Thanks for reproducing this.  So is it looking like gmail is misbehaving?  What can be done about this?  Certainly individuals reporting anything to google is a lost cause.
I've double-checked the log against one from another account where I get the alerts, which supports IDLE *and* gets a \Recent returned from the server:

S-INBOX:SendData: xx IDLE
S-INBOX:CreateNewLineFromSocket: xx OK IDLE completed
S-INBOX:CreateNewLineFromSocket: * xxxx FETCH (UID xxxxxx FLAGS (\Recent))

Thus, the lack of a \Recent returned from the Gmail server clearly is a problem. But then, with my original profile I do get biff alerts, and that's what others experiences as well (on and off) as reported in the other bug.
Interesting, after switching back to my usual profile, I got the biff alerts again with the same Gmail account. Restarted with IMAP logging switched on, and then didn't get an alert any more, thus no record of when it worked.  :-\
Ok, so after comment #21 I've got an alert and had the log running. It looks the same as without the biff, specifically no \Recent in the response either:

* 40593 FETCH (UID 41159 MODSEQ (1152046) FLAGS ())

Now I'm confused, how would it be known that a new message arrived without the \Recent flag? Just by the IDLE mechanism alone? If yes, then why would it work sometimes but not in other instances?
As a last test, I've sent myself some messages before starting up, thus allowed them to accumulate on the server as new messages once I'm logging in again.

After restart, the messages showed up as new and the alert sounded/icon shown.

However, still no "\Recent" flags either for any of those messages.
(In reply to rsx11m from comment #21)
> Interesting, after switching back to my usual profile, I got the biff alerts
> again with the same Gmail account. Restarted with IMAP logging switched on,
> and then didn't get an alert any more, thus no record of when it worked.  :-\

There are certain action that will "fix" it temporarily, for example when I completely empty the account, alerts come back until a restart, and then it stays broken.  Another action is restoring (by copy) a backed up profile, again, it work until a restart.  So perhaps that make it seem like on and off, however in my experience it does not spontaneously repair.

But these anecdotes about our existing profiles and accounts are far less useful than being able to repro in a new profile with a new gmail account.  I've been able to do this and if others can as well, that becomes a clear bug that can be reviewed by Mozilla and shown to Google.
Quick observation of "Get new messages".

(1) With Gmail, Tb 17.0.7, with no automatic new mail check, no IDLE.
    When new mail is sent via non Gmail SMTP, with non Gmail From:,
    and when "Get new Messages" is requested at Inbox of Gmail account,
    Gmail IMAP didn't return \Recent flag
    (no flag is returnd from Gmail IMAP, i/e. \Seen is not set)
    However, UID is NextUID(Highest UID + 1),
    so Tb considered the "mail of no \Recent" "new mail",
    and nsMsgMessageFlags::New was set in msgDBHdr.flags of the mail.
    => "new icon" was shown before Subject text at Thread pane.
(2) nsIMsgFolder.hasNewMessages=true is set for Inbox, according to
    nsMsgMessageFlags::New of the new mail's msgDBHdr.flags.
    => "New icon" was shown at account, Inbox at Folder Pane.
(3) Click account at Folder Pane, and click Inbox again
    => "New icon" of account/Inbox at Folder Pane
       and "new icon" at Thread Pane was clared.

"New mail alert by Biff" sees nsIMsgFolder.hasNewMessages=true, so, if  someone accesses Inbox before Biff shows new mail alert, "New mail alert by Biff" may not be shown.

Invocation of "New mail alert" is done by;
> http://mxr.mozilla.org/comm-central/source/mailnews/base/content/newmailalert.js#94
> 94 function onAlertLoad()
> http://mxr.mozilla.org/comm-central/source/mailnews/base/content/newmailalert.js#115
> 115 function showAlert()

gPendingPreviewFetchRequests is an item which Biff checks.
"new mail alert" requests fetch of previewText(top 2048 bytes by "uid xx fetch Body.peek[].<2048>") if previewText is requested, and waits for completion of previewText fetch in above logic. (see bug 823838 for previewText relevant setting)

Do you enable "previewText"? (default=enabled)
If yes, is "uid xx fetch Body.peek[].<2048>" issued for biff alert and successful?
I couldn't observe this bug in Tb 17.0.7 on MS Win-XP, with Gmail IMAP, with following settings and procedure.
  (1) Terminate Tb with Gmail account selected. (no Mbox is selected)
  (2) Restart Tb with logging enabled.
      At folder pane, account(isServer=true folder) is selected.
  (3) Because no new mail, nothing new is returned to first "uid fetch
      flags" by login_at_startup.
  (4) Send a mail to this account(via non Gmail SMTP)
  (5) By new mail check, new mail was detected.
      "New icon" was shown for account and Inbox at Folder Pane.
      UID fetch 297 (UID BODY.PEEK[HEADER.FIELDS
          (Content-Type Content-Transfer-Encoding)]
          BODY.PEEK[TEXT]<0.2048>)
      was isued for previewText.
      New mail alert by Biff was shown at right/bottom of desktop.

(A) Server Settings
    Note:
    In used profile, MailDirStore is used for this Gmail IMAP account.
> mail.accountmanager.defaultaccount = account4 (== server1 in my test)
> mail.server.server1.hostname         = imap.gmail.com
> mail.server.server1.realhostname     = imap.gmail.com
> mail.server.server1.userName         = yatter.king@gmail.com
> mail.server.server1.realuserName     = yatter.king@gmail.com
> mail.server.server1.login_at_startup = true (at startup)
> mail.server.server1.check_new_mail   = true (check every minutes)
> mail.server.server1.check_time       = 1    (every 1 minute)
> mail.server.server1.use_idle         = false
> mail.server.server1.is_gmail         = true
> mail.server.server1.storeContractID  = @mozilla.org/msgstore/maildirstore;1
(B) "New mail check of all folders" is disabled. 
> mail.server.default.check_all_folders_for_new = false
> mail.server.server1.check_all_folders_for_new is not defined
(C) "New mail alert" related settings.
>  -----------------------------------------------         -----------------------------------------
>  New mail alert related settings in Tools/Option         Corresponding settings in prefa.js
>  -----------------------------------------------         -----------------------------------------
>  Tools/Options/General
>    When new messages arrive:
>      [ ? ] Show an alert          [Customize...]   <->   mail.biff.show_alert         = true/false
>
>  At "Customize New Mail Alert" sub panel,
>      [ ? ] Message Preview Text                    <->   mail.biff.alert.show_preview = true (default)
>      [ ? ] Subject                                 <->   mail.biff.alert.show_subject = true (default)
>      [ ? ] Sender                                  <->   mail.biff.alert.show_sender  = true (default)
>  -----------------------------------------------         -----------------------------------------
(D) Auto-Sync is enabled
> mail.server.default.autosync_offline_stores = true
> mail.server.server1.autosync_offline_stores = true
(E) Offline-Use setting of Folder Property/Sunchronization.
    INBOX : Offline-Use = Off
    Note: 
    Auto-sync is enabked and some Mboxes are Offline-Use=On,
    so "STATUS" was issued for these Mboxes periodically by auto-sync.
(F) NSPR logging
    SET NSPR_LOG_MODULES=timestamp,imap:5,msgbiff:5
> https://wiki.mozilla.org/MailNews:Logging#Other_Protocol_Logging_options_within_MailNews
If MS Win, DebugView of SysInternal is useful.
    SET NSPR_LOG_MODULES=imap:5,msgbiff:5
    SET NSPR_LOG_FILE=WinDebug
  No need of timestamp, because DebugView adds timestamp.
  DebugView : http://technet.microsoft.com/en-us/sysinternals/bb896647
(In addition to comment #26)
"Message filter upon download" is not defined/used, and Junk filter is disabled in my test.
Correction.
   mail.biff.show_alert = true (default) is used in test.
Status: UNCONFIRMED → NEW
Component: General → Networking: IMAP
Ever confirmed: true
Product: Thunderbird → MailNews Core
(In reply to WADA from comment #25)
> Do you enable "previewText"? (default=enabled)

For my default profile I'm using mail.biff.show_balloon rather than the usual mail.biff.show_alert setting, thus previewText won't apply (and is broken for Gmail anyway due to bug 861911).

For the new profile created for testing, the defaults were left on except with synchronization disabled and indexing not applicable (tested with SeaMonkey).
 
> If yes, is "uid xx fetch Body.peek[].<2048>" issued for biff alert and
> successful?

I see "BODY.PEEK[HEADER.FIELDS(...)]" sent in all logs but neither shows fetching content of the message itself for the alert.
(In reply to rsx11m from comment #29)
> I see "BODY.PEEK[HEADER.FIELDS(...)]" sent in all logs
> but neither shows fetching content of the message itself for the alert.

IIRC, "requesting previewText" part of Biff code sees mail.biff.alert.show_preview only, so fetch for previewText was always requested even when mail.biff.show_alert=false.
i.e. If mail.biff.alert.show_preview=true, and if "getting new mail" is done by Biff(check for new messages upon startup/every NN minutes), request for previewText("uid xx fetch BODY.PEEK[TEXT]<0.2048>") is always queued or immediately executed.

Actually "New mail check by Biff"? "New mail notification via IDLE", isn't it? Get IMAP log with msgbiff:5, please. (NSPR_LOG_MODULES=imap:5,msgbiff:5).
Same serious problem for me, thunderbird 17.0.7, windows 7 64 bit, many imap accounts of gmail, no alert and no sound for new messages. I hope that you can help me.

Thanks to all
Sounds like good old bug 531002 which is regression
If it's a duplicate of that bug (which was filed back in 2009), why do we see the sudden influx of reports related to Gmail as documented in the forum thread?

Evidently, anything contributing in Thunderbird itself must have existed at least since 3.1, given that bug 727790 comment #29 stated reproducibility in that build.
My believe this is same issue just bit morphed (probably), as regression in 531002 was never fixed (workaround exist)
(In reply to Nikolay Shopik from comment #34)
> My believe this is same issue just bit morphed (probably), as regression in
> 531002 was never fixed (workaround exist)

What is the workaround? I tried the 3.1.18 versione, but the problem is the same. So I think that there is a change made from google in gmail... But I can't find the workaround
If you refer to bug 531002 comment #37, installing "Growl for Windows" sounds rather like a crude workaround, also given that Growl support is on and off and eventually removed, unless I'm mistaken (I am not following that discussion as it's rather specific to the Mac OSX platform).
(In reply to rsx11m from comment #36)
> If you refer to bug 531002 comment #37, installing "Growl for Windows"
> sounds rather like a crude workaround, also given that Growl support is on
> and off and eventually removed, unless I'm mistaken (I am not following that
> discussion as it's rather specific to the Mac OSX platform).

Yes, this work! Thank you. Anyway, the function "summarize" doesn't work, I hope that this is only a temporary workaround and that the problem with gmail can be solved.
FYI.
If new mail in Offline-Use=On folder is detected by auto-sync,
  by "uid NextUiD:* fetch Flags", if Mbox is already selected
  at a cached connection (==Mbox is opened),
  or by "STATUS MboxName", if Mbox is not selected at any connection,
"New icon" of Mbox is shown and Uread mail count is updated and Mbox name is highlighted at Folder Pane, and "new icon" of mail before Subject is shown and mail is highlighted at Thread pane if folder is opened. And, "MboxName of AccountName is up to date" is show at Activity Manager window.
In this case, "New mail alert by Biff" is not shown, because this is not "new mail download invoked by Biff".

(In reply to rsx11m from comment #33)
> If it's a duplicate of that bug (which was filed back in 2009),
> why do we see the sudden influx of reports (snip)

My guess-1.
"New icon" of Mbox at folder pane is correctly shown by recent Tb releases, then users are aware of existence of new mails merely by looking folder pane, thus users are aware of "no new mail alert".
My guess-2.
"New mail alert upon new mail detection by auto-sync" is stopped by recent Tb release, because it's annoying for many users especially when mail.server.default.check_all_folders_for_new=true, or because "New mail alert upon new mail detection by auto-sync" was shown even when mail.server.serverX.login_at_startup=false and mail.server.serverX.check_new_mail=false.

> related to Gmail as documented in the forum thread?

If new issue, and If Gmail specific issue and Gmail only issue;
My guess-3.
"New mail alert is shown or not" is relevant to "\Recent flag" or "number of RECENT mails in STATUS response", and Gmail IMAP stopped to return "\Recent flag to fetch flags" or "RECENT nn for STATUS" lately.
My guess-4.
isGMailServer=true(is_gmail=true in prefs.js) is now correctly detected even when hostname != imap.gmail.com. So, X-GM-MSG-ID is now obtained at any environment of "Thunderbird + Gmail IMAP" (note: imap.goolemail.com was/is set by auto-config). Because auto-sync correctly uses nsImapMailFolder::HasMsgOffline() instead of direct access of nsMsgMessageFlag::Offline flag(see bug 854161), auto-sync doesn't invoke "uid xx fetch BODY.PEEK[]" for the Mbox which has new mails(usually, fetch BODY.PEEK[] is issued only for [Gmail]/All Mail or INBOX which is accessed first). Because "download of new mail" doesn't occur, "New mail alert" is not shown.
"new icon is not shown" may be;
When new mail of X-GM-MSGID=mmm in INBOX is detected, if message data  for X-GM-MSGID=mmm is already downloaded to Offline-store file of [Gmail]/All Mail, Tb sees msgDBHdr of mail in Gmail]/All Mail. So, "status of new message in msgHDR of the mail INBOX" and "status of hasNewMessaes of MsgFolder.flags" is not correctly updated.
Bug 861911 may be;
Biff checks nsImapMailFolder::HasMsgOffline(), so "uid xx fetch BODY.PEEK[TEXT]<0.2048>" is not issued for getting previewText. However, Biff uses msgDBHdr of current Mbox for getting previewText, instead of Mbox where mail data of the X-GM-MSGID is actually held.
To al_9x@yahoo.com(bug opener) and rsx11m :

New message is detected by;
(i)   Biff(check new messages at startup/every NN minutes)
(ii)  Unsol response to IDLE for new messages or expungs messages
And, if auto-sync is enabled and Mbox is Offline-Store=On,
(iii) Periodical check for new mail/deleted mail by auto-sync.
And, if Gmail IMAP and auto-sync=enabled and Offline-Use=On, following can occur.
(a) At [Gmail]/All Mail, mail of UID=A, X-GM-MSGID=mmm is held in
    offline-store file of [Gmail]/All Mail
(b) When same mail of X-GM-MSGID=mmm is detected as UID=B at INBOX
    by Biff, auto-sync doesn't issue "uid B fetch BODY.PEEK[]" at INBOX.
    Message data for "UID=A,X-GM-MSGID=mmm in [Gmail]/All Mail" is used
    for mail display of "UID=B,X-GM-MSGID=mmm in INBOX".
(c) (b) may cause "failure to update MsgFolder.hasNewMessages of INBOX".
    If so, "New icon" of INBOX is not shown at Folder Pane".
(d) (b) may cause "no new mail alert by Biff for Inbox".

Please observe phenomena carefully.
@WADA - I think you're introducing too many variables.  We need to have a common baseline (everything at defaults) or we wont be able to focus on anything and this will never get resolved. This bug is reproducible for me in a new profile with default account config (from the wizard).

You've already identified the cause, unless I misunderstood you, no recent flag on new messages.

So please clarify, given defaults:

1. Is the absence of recent on new mail, sufficient to manifest this bug?
2. Does the absence of recent on new mail constitute a bug/breaking of protocol on the part of gmail?
3. if the answer to #2 is yes, then someone from mozilla should tell Google, right?
4. in parallel, tb should be modified to work around this.  New mail alerts/icon should optionally be shown for appearance of any unread messages that tb has not seen, regardless of recent flag.
(In reply to al_9x from comment #40)
> 1. Is the absence of recent on new mail, sufficient to manifest this bug?

You doesn't look to understand test result of comment #25 and comment #26 which is with Gmail IMAP...
Answer is "No".
"Absence of \Recent flag" is never reason of "no New icon at folder pane" and "no new icon at Thread Pane" at least when new mail, whose UID is larger than HighestUID, is detected by manual click of "Get new messages".
"Absence of \Recent flag" is never reason of "no new mail alert" at least when new mail, whose UID is larger than HighestUID, is detected by "new mail check ivoked by Biff".

> @WADA - I think you're introducing too many variables.

You doesn't look to understand comment #39...
If newly created account with default setting, all of comment #39 can occur with Gmail IMAP, because;
- check new messaes at start up, every NN minutes, is enabled
- New mail check by Biff is enabled for INBOX only
- auto-sync is enabled
- Ofline-Use=On is set in all folders of the Gmail IMAP account
- is_gmail=true is correctly detected, even when account is defined
  with server name = imap.googlemail.com
- show_alert is enabled, show previewText, Subject, Sender, is enabled
Further, 
- Global Search and Indexer is enabled

I can't understand is;
In your case.
  Why no problem after account definition, before termiation of Tb.
  Why problem always occurs after restart.
rsx11m's case.
  Why no problem with his non-new profile before termiation of Tb.
  Why problem always occurs after restart of Tb.
Bug 861911 by rsx11m.
  Why "no preview text in new mail alert" with Gmail IMAP.
In all above three cases.
  Why Gmail IMAP specific issue.

I'm suspecting "my guess-4", which is also written in comment #39.
My asking is;
Please surely rule out "new mail alert is not shown when new mail is detected via IDLE, by 'Get new messages', by auto-sync's periodical check". (I already let you know "msgbiff:5")
Please check header fetch / Body fetch for same X-GM-MSGID in IMAP log, for which "new mail alert" was not shown by Tb when new mail, which is sent via non-Gmail SMTP, without saving sent mail copy to [Gmail]/Sent Mail, is arrived to INBOX of mail IMAP.
Additional important default setting.
- IDLE comand use is enabled, and Gmail IMAP supports IDLE
(In reply to WADA from comment #41)
> (In reply to al_9x from comment #40)
> > 1. Is the absence of recent on new mail, sufficient to manifest this bug?
> 
> You doesn't look to understand test result of comment #25 and comment #26
> which is with Gmail IMAP...
> Answer is "No".
> "Absence of \Recent flag" is never reason of "no New icon at folder pane"
> and "no new icon at Thread Pane" at least when new mail, whose UID is larger
> than HighestUID, is detected by manual click of "Get new messages".

I am not sure we are talking about the same thing.  This bug is about the new mail envelope icon in the windows icon tray (not anything in TB panes) and the new mail alert box that is shown above the icon tray.

I've created a new profile; added gmail account through the wizard; set message polling to 100 and disabled message syncing (to force new detection by idle); unsubscribed from all-mail folder, to be clear, none of that is necessary to repro the the bug, it just makes things more deterministic/simpler/logs smaller and cleaner; message sent from a different address (non google), tb started with local folders selected, NSPR_LOG_MODULES=imap:5,ImapAutoSync:5,imapoffline:5,msgbiff:5,timestamp,sync

The log shows the the start of idling (04:56) then the reaction to the new message (04:57) after the peek it goes back to idling, nothing else is logged.

Are you saying that despite the missing recent flag the following should still have produced the new mail tray icon and alert?
_____________________________________________________________________________
2013-07-04 04:56:08.625000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 14 IDLE

2013-07-04 04:56:08.875000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=10 needmore=0]
2013-07-04 04:56:08.875000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: + idling

2013-07-04 04:57:39.375000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=0 needmore=1]
2013-07-04 04:57:39.375000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=12 needmore=0]
2013-07-04 04:57:39.375000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 EXISTS

2013-07-04 04:57:39.375000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: DONE

2013-07-04 04:57:39.531000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=33 needmore=0]
2013-07-04 04:57:39.531000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 14 OK IDLE terminated (Success)

2013-07-04 04:57:39.531000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:ProcessCurrentURL: entering
2013-07-04 04:57:39.531000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:ProcessCurrentURL:imap://{{username}}@imap.googlemail.com:993/select%3E/INBOX:  = currentUrl
2013-07-04 04:57:39.531000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 15 noop

2013-07-04 04:57:39.734000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=15 needmore=0]
2013-07-04 04:57:39.734000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 15 OK Success

2013-07-04 04:57:39.734000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 16 getquotaroot "INBOX"

2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=24 needmore=0]
2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTAROOT "INBOX" ""

2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=34 needmore=0]
2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * QUOTA "" (STORAGE 13 15728640)

2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=15 needmore=0]
2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 16 OK Success

2013-07-04 04:57:39.906000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 17 UID fetch 32:* (FLAGS)

2013-07-04 04:57:41.328000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=43 needmore=0]
2013-07-04 04:57:41.328000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 FETCH (UID 32 MODSEQ (8611) FLAGS ())

2013-07-04 04:57:41.328000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=15 needmore=0]
2013-07-04 04:57:41.328000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 17 OK Success

2013-07-04 04:57:41.328000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 18 UID fetch 32 (UID X-GM-MSGID X-GM-THRID X-GM-LABELS RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=283 needmore=0]
2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: * 8 FETCH (X-GM-THRID {{id}} X-GM-MSGID {{id}} X-GM-LABELS ("\\Important") UID 32 RFC822.SIZE 1717 MODSEQ (8611) FLAGS () BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)] {278}

2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:STREAM:OPEN Size: 1717: Begin Message Download Stream

...

2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:STREAM:CLOSE: Normal Message End Download Stream
2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=15 needmore=0]
2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: 18 OK Success

2013-07-04 04:57:41.500000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:SendData: 19 IDLE

2013-07-04 04:57:41.781000 UTC - 3068[7568eb0]: ReadNextLine [stream=8eb54c0 nb=10 needmore=0]
2013-07-04 04:57:41.781000 UTC - 3068[7568eb0]: 8a37400:imap.googlemail.com:S-INBOX:CreateNewLineFromSocket: + idling
(In reply to al_9x from comment #40)
> I am not sure we are talking about the same thing.  This bug is about the new mail envelope icon in the windows icon tray (not anything in TB panes) and the new mail alert box that is shown above the icon tray.

I thought "no new mail alert or icon" in bug summary and "no alert" in comment #0 is "ones by Thunderbird". And the "new mail alert box that is shown above the icon tray" is ordinal "new mail alert by Thunderbird". And, I thought correctly, Tb's Biff and new mail alert should work well to show tray-icon at Task Bar.

"New icon at Folder Pane of Tb" and "new icon at Thread Pane of Tb" is not involved in your "icon"?

As for Task-Bar tray icon, IIUC, it's shown based on "new message count" reported to Win by Tb. If it's riht, "Absense of \Recent flag" may be a cause, if "no \Recent flag" is by recent change in Gmail IMAP.

As for the "new mail alert box that is shown above the icon tray", if it's ordinal "new mail alert by Tb", it may be design/implementation change.

My guess-4 on "new mail alert" seems wrong.
Quick check result, with Tb 17.0.7 on MS Win-XP, with Offline-Use=On, new mail notification via IDLE (it might be by auto-sync's periodical check).
(1) auto-sync=enabled, IDLE is enabled, max cached connectons=5,
    new mail check upon startup, every 1 minute, is enabled. 
    INBOX : Offlie-Use=On, all mails are sync'ed.
            mail-A, UID=A, X-GM-MSGID=mmA, held in Offline-store file
            mail-B, UID=B, X-GM-MSGID=mmB, held in Offline-store file
(2) FolderX : Offline-use=On
    Delete all mails. Because of Gmail's auto-expunge=On, auto-expunged.
    Click FolderX, click other FolderY.
    FolderX is not selected at Folder pane.
    But FoldrX is still selected at a cached connection.
(3) By other IMAP client, copy mails in INBOX to FolderX, with clearing
    mail-A, X-GM-MSGID=mmA => UID=XA in FolderX, Unread status
    mail-B, X-GM-MSGID=mmB => UID=XB in FolderX, Unread status
(4) At Thunderbird, new mails are notified via IDLE.
    Needless to say, no \Seen flag, no \Recent flag, no \Deleted flag.
    Message headers are fetched, but BODY is not fetched by auto-sync.
    Unread message count is updated at Folder Pane.
    "New icon" is shown for FolderX at Folder Pane.
    "Popup message for new mail list" was shown by mouse move on
    FolderX. PreviewText might be omittted when this
    "New mail alert upon Hoover".
(5) click FolderX, go Work Offline mode.
    "new icon" is shown for two mail at Thread Pane".
    mail-A, X-GM-MSGID=mmA, UID=XA in FolderX is normally displayed.
    mail-B, X-GM-MSGID=mmB, UID=XB in FolderX is normally displayed.
    However, View/Message Source was grayed out.
    (differen problem from "nw mail alert" and "New icon")

"New mail alert upon Hoover" in step 4 may be recent implementation change in Biff alert.
(In reply to al_9x from comment #43)
(previous comment is also response to #43. sorry for typo)

Message header fetch is normally executed by Tb after "new mail notification via IDLE". 
I think "New icon"(small red star in folder icon) is show n for INBOX and "new icon for mail" is shown at Thread pane. Is it right?
If so, it's phenomenon of "New mail alert by Biff(toaster popup) is not shown when new mail detection by notification via IDLE", which is phenomenon I wrote in comment #44.

Can you check with IDLE disabled?
(IDLE is issued after IMAP command execution. So, if a connection is IDLE state, "disabling IDLE" does do nothing. Please restart Tb after "disablng IDLE".)
(In reply to rsx11m from comment #36)
> If you refer to bug 531002 comment #37, installing "Growl for Windows"
> sounds rather like a crude workaround, also given that Growl support is on
> and off and eventually removed, unless I'm mistaken (I am not following that
> discussion as it's rather specific to the Mac OSX platform).

Growl support in Windows aren't based on any Thunderbird core code. To make it work you need addon.
(In reply to al_9x from comment #43)
Quick observaion on task-tray-icon and new mail alert by Tb's bif(toaster popup) on Win-XP, with new mail for new mail detected by "new mail check by Biff": task-tray-icon is shown after "new mail alert" by Biff of Tb.
(0) Needless to say, "no \Recent flag" for new mail, because Gmail IMAP.
(1) Toaster popup is shown with preViewText, at right/bottom of Desktop.
(2) After close of toaster ppup, task-tray-icon(envelope icon) is shown
    at task tray.
(3) When INBOX is clicked, task-tray-icon disappered.
@Ludovic Hirlimann - referring to the last log I posted (Comment 43), please explain/comment:

1. Is the absence of recent flag on new mail, sufficient to manifest this bug? (i.e. the absence of new mail tray icon and alert)
2. Does the absence of the recent flag on new mail constitute a bug/breaking of protocol on the part of gmail?
3. if yes, then someone from Mozilla should tell Google, right?
4. in parallel, tb should be modified to work around this.  New mail tray alert/icon should optionally be shown for any newly retrieved unread messages, regardless of recent flag.
WADA, al_9x: I'm having trouble following you despite having at least a little bit of understanding how the IMAP protocol works. For reference, here are my relevant settings for the Gmail account showing the problem:

> mail.server.server#.check_new_mail;true
> mail.server.server#.download_on_biff;true
> mail.server.server#.login_at_startup;true
> mail.server.server#.offline_download;false
> mail.server.default.use_idle;true (toggling this didn't make a difference)
> mailnews.database.global.indexer.enabled;false
> mail.biff.show_alert;false  (no change when switched to default)
> mail.biff.show_balloon;true (no change when switched to default)
> mail.biff.alert.show_preview;true

Anything else should be the installation default. I'll be happy to provide a new log with specific settings if you want me to do so, but there is not guarantee that the biff will work/won't work at that time based on prior experience (where starting from a new profile apparently creates the best reproducible conditions).

(In reply to WADA from comment #38)
> My guess-1.
> "New icon" of Mbox at folder pane is correctly shown by recent Tb releases,
> then users are aware of existence of new mails merely by looking folder
> pane, thus users are aware of "no new mail alert".

I don't think this applies, I'm rather confident that I would have noticed this issue earlier even if it had been better disguised. As stated in many of the incoming reports, it only started as recently as a couple of weeks ago.
(In reply to rsx11m from comment #49)
> WADA, al_9x: I'm having trouble following you

You're probably having trouble following WADA because until his last post he did not understand what this bug was about and may not still, and consequently was posting effectively off-topic about new mail indicators in the TB window.

We need someone who knows the code and understand the bug to answer definitively, questions in Comment 48.  We also don't need any more logs, note that my log from Comment 5 is basically the same as one from Comment 43.  The picture both paint is pretty straight forward: tb idles, new message is detected with no recent flag, tb peeks headers, goes back to idling, no tray alert or icon.

There should be enough info there.
Ok, thanks for the clarification.
(In reply to al_9x from comment #50)
> The picture both paint is pretty straight forward: tb idles,
> new message is detected with no recent flag, tb peeks headers,
> goes back to idling, no tray alert or icon.

Does your this comment implies that new mail alert(and following tray-icon) was always shown by Tb when new mail was detected by "new mail check by Biff" with Gmail IMAP?
If so, I'm sorry for my misundersanding that you didn't check "new mail check by Biff" case with Gmail IMAP.

(In reply to rsx11m from comment #49)
> WADA, al_9x: I'm having trouble following you

You're probably having trouble following al_9x because he still pastes(paste, instead of attach...) IMAP log for "new mail detection via IDLE" and repetedly reports phenomenon of "new mail alert was not shown when new mail was detected via IDLE", even though I wrote comment #12 ...

Reason why I asked him and you about "New icon" at Tb's folder pane is;
(1) "new mail alert by Tb's Biff" looks prerequite of task-tray-icon.
    task-tray-icon doesn't look shown without "new mail alert"(toaster
    popup) at right/bottom of Desktop.
(2) "new mail alert by Tb's Biff" is shown only when Tb detects "New
    mail". 
    - New mail = mail flagges as "New Mail" in msgDBHdr.
      flag of "New Mail" is set when mail has \Recent flag,
      but "all mail of New Mail flag had \Recent flag" is always false.
      "New mail what didn't have \Recent flag" surely exists in Tb.
    - "New mail" is always "mail of Unread status" is true in Tb.
      I don't know Tb's behaviou on "mail of \Recent \Seen" which
      shouldn't be returned from IMAP server.
    - I cant't see any description about "new" in "inbox shows the new
      message" in following in comment #0.
        5. no tray icon, no alert, inbox shows the new message
      Because new mail is always "no \Seen", the mail is shown in Bold.
      I can't know whether this mail was actually marked as "New Mail"
      by Tb or not from this statement,
      "whether this mail was actually marked as New Mail by Tb or not"
      can not be known by IMAP log, because IMAP log says nothing about
      "New Mail" status set by Tb of the fetched mail.
(3) New Mail status of a mail is reflected to MsgFolder.hasNewMessages.
    MsgFolder.hasNewMessage=true is reflected to "New icon" of folder
    icon at Folder Pane.
(4) To rule out case of "Tb doesn't consider mail of no \Recent flag,
    no \Seen flag as New Mail",
    statement about "New icon" status by bug reporter is needed.

Please note;
  message header is newly fetched != Tb considers the mail as "New Mail"
  even though these are "equal" in many cases.

Following is still not denied, because we don't know Tb's behaviour with non-Gmail IMAP.
- If \Recent flag is set by server, new mail alert of Tb's Biff is shown
  even when new mail is detected by IDLE, then tray-icon is shown.
- When new mail is detcted by IDLE, if \Recent flag is set in mail by
  IMAP server, new mail alert by Biff is surely shown, then tray-icon
  is shown.
However, following is absolutely invalid,
  if no \Recent flag in new mail, new mail alert by Biff is not shown.
because Tb's Biff actually shows new mail alert at least when "new mail of no \Recent" is detected by new mail check invoked by Biff.

What is essential difference of this bug from bug 531002?
Actually Gmail IMAP specific issue?

If this bug is for "no new mail alert when new mail notificatipn via IDLE" only, it's simply "a phenomenon of bug 531002 was observed when new mail was detected via IDLE", and "no \Recent flag was also observed because of Gmail IMAP", isn't it?
If "absense of \Recent" is reason of "no new mail alert", how following can be explaned?
  (A) No problem after new account creation before restart of Tb,
  (B) but problem always occurs after restart of Tb.
"no new mail alert when new mail notificatipn via IDLE" can explain (B) well, but it can not explain (A).

I never say current Tb's behaviour on new mail alert is absolutely correct.
However, I can say that simply/repeatedly posting report about phenomenon of "new mail alert was not shown when new mail was detected via IDLE" is useless.
"New message" is also detected by "periodical new mail check by auto-sync(using STATUS command if Mbox is not selected at any cached connection)" when Mbox is Offline-Use=On folder.
I don't know whether this is same as "new mail check by Biff" or "new mail detection via IDLE".
Further, "periodical new mail check by auto-sync" doesn't look to issue "uid NextUID:* fetch Flags" if a Offline-Use=On folder is selected at a cached connection(Mbox is opened). This indicates that new mail is never detected automatically unless IDLE is enabled or "new mail check by Biff" is enabled for this folder.
I believe these are a reason of "chaos in bug 531002".

I think what needed in this bug is;
- To know "absense of \Recent flag" is actually cause of this bug,
  or not.
- To find conditions when new mail alert is not shown by Tb
  even though user expects "new mail alert".
  ("new mail notification via IDLE" was reported repeatedly in this bug)
- To find conditions when new mail alert is shown by Tb
  even though user expects "no new mail alert".
  ("new mail detected via IDLE" is reported in New mail alert,)
  (even though new mail check is disabled for this Mbox.      )
- To find difference between Tb's behavior" and "user's expectation".
  ("new mail alert upon each new mail notification via IDLE" looks
  (expectation of bug opener.)
- To find a way to improve "new mail alert"

"One new mail alert per a new mail notification via IDLE" is merely annoyance for users in following case.
  "A new mail per each 10 seconds",
  and "100 mails arrives during 1000 seconds"(28 mimutes) 
  => 100 new mail alerts during 30 minutes...
  Even if server limits "notification interval" to 1 minute, 30 alerts.
A possible solution in such case.
  If tray-icon is already shown, update new mail count shown by tooltip
  of tray-icon, without showing "new mail alert".
Difference between (A) and (B) is perhaps "[Gmail]/All Mail" is selected at a cached connection, or not.
  (A) No problem after new account creation before restart of Tb,
  (B) but problem always occurs after restart of Tb.

(0) IDLE is enabled, auto-sync=On, 
    INBOX : Offline-Use=On, [Gmail]/All Mail : Offline-Use=On
    New mail check by Biff is enabled.
    [Gmail]/All Mail: [ ] "When getting new message, check this folder"
                      is Unchecked.
    So, new mail check by Biff is invoked for INBOX only.
    Because Gmail IMAP pretty quickly notifies new mail via IDLE,
    Tb knows new mail in an Mbox almost always by "new mail
    notification via IDLE" when Mbox is selected at cached connection.
(1) To make it simple, start from "account is selected", "no Mbox is
    selected" at folder pane upon restart of Tb.
(2) Restart Tb.
    Because of login_at_startup=true, INBOX is selected at a cached
    connection, anf "uid 1:* fetch Flags" is issued.
    INBOX is perhaps internally opened even if INBOX is not selected
    at folder pane explicitely.
    Because check_new_mail=true, "uid NextUID:* fetch flags" is issued
    periodically for INBOX by "new mail check by Biff".
(3) New mail(mail-1) is sent.
    New mail is put int [Gmail]/All Mail and INBOX by Gmail IMAP.
(4) Because INBOX is selected at a cached connection, new mail is
    notified via IDLE for INBOX.
    Because [Gmail]/All Mail is not selected at any cached connection,
    and "new mail check by biff" is not enabled for [Gmail]/All Mail,
    mail-1 in [Gmail]/All Mail is not detected by Tb.
    So, mail-1 is detected via IDLE and fetched at INBOX only.
    => New mail alert is not shown.
    (bug opener sees this in (B))
    Nothing occurs on [Gmail]/All Mail at this stage.
(5) Because [Gmail]/All Mail is Offline-Use=On and auto-sync=enabled,
    "STATUS is issued by auto-sync for [Gmail]/All Mail sooner or ater.
    When new mail(mail-1) in [Gmail]/All Mail is detected by STATUS,
    [Gmail]/All Mail is selected at a cached connection,
    and mail-1 is feched at the cached connection, and IDLE is issued.
(6) By other tests, new mail alert was shown and tray-icon was shown.
    So, I clicked INBOX(explicite open of INBOX), in order to clear
    tray-icon.
(7) New mail(mail-2) is sent.
    mail-2 is detected almost same time via IDLE at INBOX and
    [Gmail]/All Mail
    => new mail alert was shown.
    => Clicked account, then click INBOX again at folder pane,
       in order to clear tray-icon.
(8) New mail(mail-3) is sent.
    mail-3 is detected almost same time via IDLE at INBOX and
    [Gmail]/All Mail
    => new mail alert was shown.
       new mail alert showed mail-2 and mail-3.
    => Clicked account, then click INBOX again at folder pane,
       in order to clear tray-icon.
(9) New mail(mail-4) is sent.
    mail-4 is detected almost same time via IDLE at INBOX and
    [Gmail]/All Mail
    => new mail alert was shown.
       new mail alert showed mail-2 and mail-3 and mail-4.
    => Clicked account, then click INBOX again at folder pane,
       in order to clear tray-icon.
(7)/(8)/(9) is perhaps same phenomenon as "new mail detected via IDLE is shown by new mail alert, even though new mail check by Biff is disabled for the Mbox".
I guess (A) is phenomenon of (7), (8), (9).
My guess-5.
Reason why report of "new mail alert is not shown if new mail is detected via IDLE" was rare with Gmail IMAP :
  new mail in INBOX and [Gmail]/All Mail is usually detected at same
  time if Gmail IMAP.
My guess-6. 
Reason why report of "new mail alert is not shown" increased recently :
  "don't show new mail alert if new mail is detected via IDLE"
  works pretty well from recent Tb release.
  Because almost all new mail is detected via IDLE when Mbox is selected
  at a cached connection, and because INBOX is always selected at a
  cached connection, and because IDLE is enabled by default and IDLE
  is supported by many IMAP servers, many users started to see
  phenomenon of "new mail alert is not shown if new mail is detected
  via IDLE".
After several times of test, I saw a part (C) of phenomenon reported by rx11m...
  (C) With existent profile, no problem.
  (D) After restart, problem always occurs. 
i.e. I can't see phenomenon of "new mail alert is not shown when new mail is detected via IDLE" in both (C) and (D), with same profile, even with Offline-Use=Off of [Gmail]/All Mail.
(Note: IMAP log for new mail detection is always same as one reported by al_9x. I can't find any difference from log by al_9x).
Only way to see "new mail alert is not shown when new mail is detected  via IDLE" consistently is:
  After new mail alert and tray-icon is shown, don't close tray-icon.

byal_9x(bug opener) and rx11m:

Is there any possibility of "task-tray icon is hidden"?
I see known phenomenon/issue like next sometimes with SeaMonkey in daily use, and with Thunderbird for testing.
  Browser window of Sm, MailNews window of Sm, Main window of Tb,
  is hidden at Desktop because of perhaps minimized.
  However, window is not shown at Task Bar of Win.
  One of following is needed to show again: 
  Select window from window list by Tab key, Window/Browser or MailNews
  menu of Sm, Request strt of Tb.
Similar issue may exist in Task tray icon too.

Is there any possibility of "task-tray icon is hidden but not closed correctly by termination of Tb"?
If this kind of problm, "after restart, problem always occurs" can be explained.

If Tb wrongly considers "tray-icon is opened", Tb doesn't show new mail alert.
rx11m, do you know how to check task-tray-icon status in Tb and/or OS?

Even when multiple new mails are shown by new mail alert at step (8)/(9) in comment #53, Tooltip of tray-icon always shows "... has 1 new messages".
Do you think this kind of issue relevant to problem of "new mail alert and tray-icon is not shown"?

After you see your problem, do you still see problem with "new mail detection by Biff", with IDLE terminated and disabled, without restart of Tb?
- Server Settings/Advanced, disable IDLE use
- to surely terminate IDLE, change mail status(Read/Unread/Starrred,...)
  of a mail in INBOX. (store flag is issued, but IDLE is not issued)
How about following?
- Click other than Inbox at folder pane, then click INBOX again.
  (This is operation to close tray-icon.)
(In reply to al_9x from comment #50)
> The picture both paint is pretty straight forward: tb idles,
> new message is detected with no recent flag, tb peeks headers,
> goes back to idling, 

Problem is : With these conditions, "no tray alert or icon" occurs some times(after restart of Tb in your case), but "no tray alert or icon" doesn't occur sometimes(before restart of Tb in your case, in my recent tests).

If DebugView is used with Tb's IMAP logging, you can see IMAP log in DebugView Console while Tb is running, and logs can be cleared, saved while Tb is running. This is far convenient than "sync" of IMAP logging.
Check using DevugView, with minimum configuration(with IDLE, with accessing INBOX and [Gmail]/All Mail only), please.
For example,
(1) auto-sync disabled : 
      mail.server.serverX.autosync_offline_stores = false
    Unless folder is not opened at folder pane, no activity occurs.
    So, activity on INBOX only occurs and logged.
    Then check with shortest new mail check interval(1 minutes) is easy.
(2) auto-sync enabled, Offline-Use=On folder : Inbox only.
(3) auto-sync enabled, Offline-Use=On folder : Inbox, [Gmail]/All Mail
(4) auto-sync enabled, Offline-Use=On folder : Inbox only
    new mail check by Biff for [Gmail]/All Mail is enabled.
(5) auto-sync enabled, Offline-Use=On folder : Inbox, [Gmail]/All Mail
    new mail check by Biff for [Gmail]/All Mail is enabled.
My check in comment #53 = above (3).

An imporant partiqularity of Gmail IMAP is : [Gmail]/All Mail.
Do you see "no tray alert and no tray icon" when a new mail is detected via IDLE at INBOX and [Gmail]/All Mail at same time?
FYI.
Quick observation of tray-icon on Win-XP.
(A) Kill Tb without close tray-icon.
1. Restart Tb, new mail arrives, new mail alert, tray-icon-1 is shown,
   terminate Tb without close tray-icon-1 => tray-icon-1 is kept
2. Restart Tb, new mail arrives, new mail alert, tray-icon-2 is shown,
   terminate Tb without close tray-icon-2 => tray-icon-2 is kept
3. Restart Tb, new mail arrives, new mail alert, tray-icon-3 is shown,
   terminate Tb without close tray-icon-3 => tray-icon-3 is kept
4. Mouseover on tray-icon-1, tray-icon-2, tray-icon-3
   => tray-icon-1, tray-icon-2, tray-icon-3 disappered.
(B) tray-icon is not visible, because Task Tray is collapsed.
1. auto-sync=disabled, IDLE is disabled
   => Inbox only access, new mail is detected by Biff always.
   Restart Tb.
2. new mail is sent. new mail alert is shown.
   tray-icon is invisible, because Task Tray is collapsed
   and tray-icon is placed at hidden part of Task Tray.
3. new mail is sent. new mail alert is not shown.
   This is because tray-icon is not closed yet by click of Inbox.
4. Expand Task Tray by "left arrow" button at Task Tray
   => tray-icon is visible
   Collapse Task Tray by "right arrow" button at Task Tray
   => tray-icon is invisible
   Many of "no new mail alert and no tray-icon" may be this phenomenon
   in my tests.

byal_9x(bug opener) and rx11m:
Please surely rule out phenomenon like above (B).
Checked "new mail is detected only via IDLE" case on Win-XP.
- auto-sync is disabled.
  mail.server.default.autosync_offline_stores = false
  mail.server.server1.autosync_offline_stores = false
- Offline-use = Off
  Inbox : Offline-Use=Off, [Gmail]/All Mail : Offline-Use=Off
- New mail check by Biff is disabled.
  mail.server.server1.login_at_startup = false
  mail.server.server1.check_new_mail   = false
- [ ] When getting new messages, check this folder
  [Gmail]/All Mail : Unchecked
- New mail alert is enabled
  mail.biff.show_alert = true
  mail.biff.alert.show_preview = true
  mail.biff.alert.show_sender  = true
  mail.biff.alert.show_subject = true  <= Big difference?
  mail.biff.show_balloon       = false <= Big difference?
(1) Restart Tb, open two Tb main windows 
    Window-1 : Click Inbox            => opened, always IDLE is issued
    Window-2 : Click [Gmail]/All Mail => opened, always IDLE is issued
(2) Send a new mail(mail-1)
    => new mail is detected at both Inbox and [mail]/All Mail
    => new mail alert is shown
       for mail-1 in Inbox, and mail-1 in [Gmail]/All Mail
       and tray-icon is shown
    Click account, then click Inbox again
    => tray-icon is cleared
(3) Send a new mail(mail-2)
    => new mail is detected at both Inbox and [mail]/All Mail
    => new mail alert is shown, and tray-icon is shown
       mails in alert :
        mail-2 in Inbox <= new mail status in Inbox is correctly cleared
        mail-1, mail-2 in [Gmail]/All Mail
    Click account, then click Inbox again
    Click account, then click [Gmail]/All Mail again
(4) Send a new mail(mail-3)
    => new mail is detected at both Inbox and [mail]/All Mail
    => new mail alert is shown, and tray-icon is shown
       mails in alert :
         mail-3 in Inbox
         mail-3 in [Gmail]/All Mail
         => new mail status of mails is correctly reset by folder click
    Click account, then click Inbox again
    Click account, then click [Gmail]/All Mail again
Observed phenomenon was following only:
- New mail alert is shown always for "new mails detected by IDLE",
  even though "new mail check by Biff" is disabled.
  (already reported problem. different from normal user's expectation)
- If tray-icon is not cleared by click of Inbox,
  new mail alert is not shown, and new mail count in Tooltip of
  tray-icon is not changd(always "1 new messages")
I still can't see phenomenon of "new mail alert is not shown, if and only if new mail is detectd via IDLE, and if no \Recent flag".

(In reply to rsx11m from comment #49)
> WADA: I'm having trouble following you despite having at least a
> little bit of understanding how the IMAP protocol works.
> (snip)
> > mail.biff.show_alert;false  (no change when switched to default)
> > mail.biff.show_balloon;true (no change when switched to default)
>(snip)
> I'm rather confident that I would have noticed this issue earlier
> even if it had been better disguised.
> As stated in many of the incoming reports, it only started as recently as a couple of weeks ago.

rsx11m, is show_balloon=true relevant to your problem?
Was default of show_alert/show_balloon changed from true/false to false/true recently? If yes, is it Win 7/Win 8 only change?
(if so, "no problem with old profile, but problem occurs always after restart" may be explained)
Is your problem Gmail IMAP only problem?
Is your problem "new mail detected via IDLE" only problem?
@Ludovic Hirlimann - I've built and debugged TB and narrowed this problem down somewhat:

nsImapMailFolder::UpdateImapMailboxInfo has the following code:

  int32_t numUnreadFromServer;
  aSpec->GetNumUnseenMessages(&numUnreadFromServer);
  
  bool partialUIDFetch;
  flagState->GetPartialUIDFetch(&partialUIDFetch);
  
  // For partial UID fetches, we can only trust the numUnread from the server.
  if (partialUIDFetch)
    numNewUnread = numUnreadFromServer;
___________________________________________________

numNewUnread calculated locally is >0 but since partialUIDFetch==true, it's set to NumUnseenMessages from the server, which is 0

numNewUnread being >0 at that point starts the cascade that results in new alert/icon

the next step is figuring out where aSpec->GetNumUnseenMessages comes from

it's set in nsImapServerResponseParser::resp_text

    else if (!PL_strcasecmp(fNextToken,"UNSEEN"))
    {
      AdvanceToNextToken();
      if (ContinueParse())
      {
        fNumberOfUnseenMessages = atoi(fNextToken);
        AdvanceToNextToken();
      }
    }
________________________________________________

this code, in fact this entire method never runs before fNumberOfUnseenMessages is read.  I don't know the protocol enough to say who's at fault here so there are two possibilities:

a. either TB is not issuing the appropriate command (STATUS?) needed to produce the UNSEEN token
b. or gmail is not returning the UNSEEN token in response to whatever command TB expects to return it
to be clear, Thunderbird doesn't rely on the recent flag to show new mail, because it's not reliable. So new mail is just unread mail that hasn't been seen by Thunderbird on that machine before.
(In reply to David :Bienvenu from comment #60)
> to be clear, Thunderbird doesn't rely on the recent flag to show new mail,
> because it's not reliable. So new mail is just unread mail that hasn't been
> seen by Thunderbird on that machine before.

could you please address comment 59
FYI.
"When new mail is notified via IDLE, Tb doesn't show Biff alert" which I called in my comment #12 is following.
- New mail check by Biff is enabld, new mail alert is enabled,
  IDLE command use is enabled,
  [Gmail]/All Mail is included in new mail check,
  auto-sync is enabled, Inbox, Gmail]/All Mail : Offline-Use=On
  All mails in all Offline-Use=On folders are sync'ed.
- FolderX : Offline-Use=On, not included in New mail check by Biff.
  All mails in FolderX is deleted.
  FolderX is opened by Tb => IDLE command is issued.
- At different IMAP client,
  Change a mail in [Gmail]/All Mail to Unread.
  Copy the mail to FolderX from [Gmail]/All Mail.
=> At Tb, new mail in FolderX is detected via IDLE,
   and new mail alert is not shown always,
   because Biff is not scheduled for FolderX, and because "new
   mail detection at folder where Biff is scheduled" doesn't occur.

(In reply to al_9x from comment #43)
> added gmail account through the wizard; set message polling to 100
> and disabled message syncing (to force new detection by idle); (snip)

Following is prefs.js entry relevant to "New mail check".
> mail.server.server#.login_at_startup = true
> mail.server.server#.check_new_mail   = true
> mail.server.server#.check_time       = 100 (minutes, 6000 sec)
"First INBOX open by login_at_startup just after restart of Tb" may be different from "each Biff interval".
- by login_at_startup=true, INBOX is opened/selected,
  and "uid 1:* fetch flags" is issued, 
  and if check_new_mail=true, set timer for first Biff.
- upon each Biff interval, issue "uid NextUID:* fetch flags" if Mbox
  is selected at cached connection, and if check_new_mail=true,
  set timer for next biff interval.

Please confirm that "Biff is scheduled for INBOX" by log like following(serverX : server # of Gmail IMAP account) when your problem occurs.
> :imap.gmail.com:S-INBOX:CreateNewLineFromSocket: + idling 
>(snip)
> : performing biffs
> :imap.gmail.com:S-INBOX:SendData: DONE 
> : biffing server serverX rv = 0
> : inserting biff entry at 0
> : setting 6000123 timer
(In reply to al_9x from comment #61)
> (In reply to David :Bienvenu from comment #60)
> > to be clear, Thunderbird doesn't rely on the recent flag to show new mail,
> > because it's not reliable. So new mail is just unread mail that hasn't been
> > seen by Thunderbird on that machine before.
> 
> could you please address comment 59
Flags: needinfo?(neil)
Flags: needinfo?(mbanner)
since the SNR here is low, let me highlight Comment 43 (provides the log) and Comment 59 (what I found debugging TB)

one more detail, in Comment 24 I mentioned that deleting all mail temporarily restores the alert/icon (until a restart).  I did not focus on this but, from what I remember while debugging, when you delete everything, flagState->GetPartialUIDFetch starts to return false, and it continues to return false (as new messages come in) until TB is restarted (is that correct behavior?)

when false, numNewUnread, retains the >0 locally determined value, and so the alert and icon are shown
FYI.
Following is a description about full sync, partial sync, sync from the previous highwater mark.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#4006
> 4006     // Figure out if we need to do a full sync (UID Fetch Flags 1:*),
> 4007     // a partial sync using CHANGEDSINCE, or a sync from the previous
> 4008     // highwater mark.
And following is a place where PartialUIDFetch=false is set.
Note: PartialUIDFetch is initialized by SetPartialUIDFetch(true)
> 4014     if (needFullFolderSync || needFolderSync)
> 4015     {
(snip)
> 4018       if (!needFullFolderSync && !GetShowDeletedMessages() && UseCondStore())
> 4019         PR_snprintf(fetchModifier, sizeof(fetchModifier), " (CHANGEDSINCE %llu)",
> 4020                     mFolderLastModSeq);
> 4021       else
> 4022         m_flagState->SetPartialUIDFetch(false);

If Gmail IMAP specific, and if PartialUIDFetch=true is cause, CONDSTORE may be relevant to problem because Gmail IMAP supports CONDSTORE, and "changing to Just mark it as deleted for ease of test during my tests" might be a reason why I couldn't see problem suddenly...

(0) mail.server.server1.autosync_offline_stores = false
    IDLE command use is enabled.
(1) IMAP delete model : "Just mark it as deleted"
    mail.server.server1.use_condstore is not set
    => New mail alert is shown.
(2) IMAP delete model : "Just mark it as deleted" => "Move to trash"
      mail.server.server1.use_condstore;false
    mail.server.server1.use_condstore is not set
    => New mail alert is not shown.
(3) IMAP delete model : "Move to trash" is kept.
      mail.server.server1.use_condstore;false
    mail.server.server1.use_condstore = false
    => New mail alert is shown.
Note-1: Value of check_time was irrelevant. Problem was observed in tests of (2) even with mail.server.server1.check_time=1 and "uid NextUID:* fetch flags" was issued repeatedly by Biff.
Note-2: Problem is not observed with mail.server.server1.autosync_offline_stores = true.

Question is:
Why no problem "after account definition with default, before restart"
even though problem always occurred after restart?
needFullFolderSync==true before restart because synchronization of all mail data by auto-sync is not completed yet?
(In reply to al_9x from comment #64)
> one more detail, in Comment 24 I mentioned that deleting all mail
> temporarily restores the alert/icon (until a restart).
> I did not focus on this but, from what I remember while debugging,
> when you delete everything,
> flagState->GetPartialUIDFetch starts to return false,
> and it continues to return false (as new messages come in) until TB is restarted (is that correct behavior?)

If code I pointed in comment #65 is one of causes of "starting GetPartialUIDFetch==false" or "GetPartialUIDFetch==true after restart", 
(a) it's needFullFolderSync==true, because !GetShowDeletedMessages() and UseCondStore() is not changed.
(b) Similar problem to bug 831664 occurs by "deletion of all mails" or "other operations", then UseCondStore()=false is returned because incommingServer.useCondStore==undefined is returned for a while.
bug 831664 can be reproduced by account collapse/expand=>folder re-discovery, even after bypassing problem by all folder property access.
(c) CONDSTORE may be relevant.
Gmail IMAP supports CONDTORE, and Tb uses CHANGEDSINCE when first open after restart.
> 13 UID fetch 1:* (FLAGS) (CHANGEDSINCE 117803)
CHANGEDSINCE is not used when each new mail check by biff, 
> 17 UID fetch 394:* (FLAGS)
and Gmail IMAP returns MODSEQ correctly.
> * 2 FETCH (UID 393 MODSEQ (117709) FLAGS (\Seen)) 
Code around following may be relevant.
> 4059       // if we're using CONDSTORE, and the parser hasn't seen any UIDs, use
> 4060       // the highest UID we've seen from the folder.
> 4061       if (UseCondStore() && !highestRecordedUID)
> 4062         highestRecordedUID = mFolderHighestUID;
Irving or David would be better looking at this.
Flags: needinfo?(mozilla)
Flags: needinfo?(mbanner)
Flags: needinfo?(irving)
That time frame matches my experience. Does anybody know what Google changed in their Gmail service around that time, thus most likely triggering this issue?
(In reply to rsx11m from comment #69)
> That time frame matches my experience. Does anybody know what Google changed
> in their Gmail service around that time, thus most likely triggering this
> issue?

It's certainly not widespread. I dug further and it seems my two examples look to be darn rare - https://www.google.com/search?num=20&safe=off&tbs=cdr%3A1%2Ccd_min%3A5%2F1%2F2013%2Ccd_max%3A7%2F9%2F2013&q=gmail+visual+OR+sound+OR+alert+OR+notification+OR+notify+OR+growl+site%3Agetsatisfaction.com%2Fmozilla_messaging&oq=gmail+visual+OR+sound+OR+alert+OR+notification+OR+notify+OR+growl+site%3Agetsatisfaction.com%2Fmozilla_messaging&gs_l=serp.3...18658.23599.0.24368.9.9.0.0.0.0.44.44.1.1.0....0...1c.1.19.serp.7dB3XRz6F54

And I wonder whether it is related to me not seeing imap changes in thunderbird - I delete a message on PC 1 and it doesn't disappear from PC 2.
(In reply to Wayne Mery (:wsmwk) from comment #70)
> It's certainly not widespread. I dug further and it seems my two examples
> look to be darn rare -

Well, if you have a look at the MozillaZine thread linked to in the URL field, there are quite a few users reporting that problem and new ones keep showing up.

It probably depends how you define "widespread" - if you are an affected user, it is definitely an annoying problem and may lead to not lost but missed e-mails.
(and just as if Gmail wanted to confirm my point, the alerts didn't go off for the bugmail message sent for my previous comment. :-D )
(In reply to rsx11m from comment #69)
> That time frame matches my experience. Does anybody know what Google changed in their Gmail service around that time, (snip)

As you know, CONDSTORE support is not so old enhancement by Google, so CONDSTORE can be called another "partiqularity of Gmail IMAP"(This is a reason why I wrote comment #65.)
I couldn't see CONDSTORE in capability response after login, [HIGHESTMODSEQ 119777] in select responce, in IMAP log for Gmail IMAP incidentally taken on 2013/03/12. 
"MODSEQ (119671) in UID fetch 414:* (FLAGS) response" was also not seen in log taken on 2013/03/12 and 2013/03/31.

If you can trust comment #65 by me, check difference between followings, please.
- No mail.server.server#.use_condstore (identical to true if Gmail IMAP)
- mail.server.server#.use_condstore = false
FYI.
Because CONDSTORE is supported, IMAP code appends "(CONDSTORE)" to SELECT command.
  48 select "[Gmail]/All Mail" (CONDSTORE)
However, this "(CONDSTORE)" is rarely seen for first 'select "INBOX" ' which is issued just after first login to IMAP server at first connection. I could see 'select "INBOX" (CONDSTORE)' only several times during tests. A way to force 'select "INBOX" (CONDSTORE)' is;
- Disable new message check, max cached connection=1,
  select account before termination, terminate Tb.
- Restart Tb => nothing is selected.
  Click a folder other than Inbox(call Mbox)
  => select MBox (CONDSTORE)
  => "select Inbox" request is queued,
     or is postponed because of no free connection
- max cached connection=3 or 4 or 5
  Click INBOX => select "INBOX" (CONDSTORE) is issued
Because "(CONDSTORE)" parameter is merely for requesting to guarantee MODSEQ in uid fetch response, it doesn't affect on actual MODSEQ response from server.
However, "no (CONDSTORE) in SELECT INBOX" indicates that Tb doesn't detect "Condstore is used" upon first select INBOX just after first login to server. This may be relevant to "inconsistent or unexpected flagState->GetPartialUIDFetch.
MODSEQ in response depended on (CONDSTORE) parameter of "SELECT".

(1) Just after restart of Tb.
    select "Inbox"
    no MODSEQ in response to uid NextUID:* fetch flags
(2) after go Work Offline, go Work Online
    select "Inbox" (CONDSTORE)
    MODSEQ is returned in response to uid NextUID:* fetch flags
(3) If [Gmail]/All Mail, CHANGEDSINCE is used
    select "[Gmail]/All Mail" (CONDSTORE)
    UID fetch 1:* (FLAGS) (CHANGEDSINCE 120418)
    * 836 FETCH (UID 3292 MODSEQ (120420) FLAGS (\Seen))

If flagState->GetPartialUIDFetch is cause of problem, and if CONDSTORE is relevant to problem, Tb's behavior and Gmail's behavior is perhaps relevant to problem.
- No (CONDSTORE) parameter in SELECT if select after first login
- No (CHANGEDSINCE nnnnnn) if first SELECT of INBOX
- No MODSEQ in "uid xx:yy fetch flags" response if no (CONDSTORE) parameter
FYI.
If mail.server.server#.use_condstore=false, following is issued as usual.
  11 select "INBOX"
  13 UID fetch 1:* (FLAGS)
Difference from [GMAIL]/All Mail with CONDSTORE used
  - no (CONDSTORE) in select
  - no CHANGEDSINCE in UID fetch 1:* (FLAGS)
Difference from INBOX with CONDSTORE used
   with CONDSTORE used     : UID "HighestUID in folder"+1:* fetch (FLAGS)
   with CONDSTORE disabled : UID 1:* fetch (FLAGS)
FYI.
Another difference was found : ENABLE CONDSTORE

(A) When second login to server for [Gmail]/All Mail with Condstore enabled, ENABLE CONDSTORE is issued after capability response for login, and UID fetch 1:* (FLAGS) (CHANGEDSINCE 120956) is always used.
   4 COMPRESS DEFLATE
   5 ENABLE CONDSTORE
   7 select "[Gmail]/All Mail" (CONDSTORE)
   9 UID fetch 1:* (FLAGS) (CHANGEDSINCE 120956) 

(B) If first login for INBOX after restart, ENABLE CONDSTORE is not issued, but UID fetch 1:* (FLAGS) (CHANGEDSINCE 121287) is issued as done on [Gmail]/All Mail, then MODSEQ is returned from Gmail IMAP.
   5 COMPRESS DEFLATE
  11 select "INBOX" 
  13 UID fetch 1:* (FLAGS) (CHANGEDSINCE 121287)
  * OK [HIGHESTMODSEQ 121318]
  After it, by Biff,
  17 UID fetch 425:* (FLAGS)
  Gmail IMAP returns MODSEQ. 
  * 7 FETCH (UID 424 MODSEQ (121211) FLAGS (\Seen))

(C) When explicit open of INBOX was not done in previous Tb session, "1:* (FLAGS) (CHANGEDSINCE 121287)" was not issued by Tb, then MODSEQ was not returned from Gmail IMAP.
  1. Restart Tb, select account, terminate Tb
  2. Restart Tb, terminate Tb without explicit open of INBOX
  3. Restart Tb => Following is done by first login for INBOX.

   5 COMPRESS DEFLATE
  11 select "INBOX" 
  13 UID fetch 425:* (FLAGS)        <= HighestUID+1:*, no CHANGEDSINCE
  * 7 FETCH (UID 424 FLAGS (\Seen))
  After it, by Biff,
  17 UID fetch 425:* (FLAGS)
  Gmail IMAP doesn't return MODSEQ.
  * 7 FETCH (UID 424 FLAGS (\Seen))
This looks one of a causes of pretty inconsistent behavior in "new mail alert" which I saw, because I sometimes do, 1. terminate Tb with account selected, 2. restart Tb, and terminate Tb without explicit open of INBOX. This was cause of my confusions in understanding test results.

(D) Even if (C) occurs, if following is executed, status of INBOX is changed to (B).
  1. Click INBOX at folder pane(explicit folder open)
  2. go Work Offline
  3. IMAP flow on INBOX is same as (B)

Diference among (A), (B), (C), (D) may be a cause of difference between "before restart" and "after restart" which is reported by bug opener and some other peoples. Funny flagState->GetPartialUIDFetch which bug opener saw may be difference among (A), (B), (C), (D).

To bug opener:
Please check IMAP log around Mbox selection too, instead of IMAP log around "new mail notification via IDLE" only.
(In reply to WADA from comment #65)
> (3) IMAP delete model : "Move to trash" is kept.
>     mail.server.server1.use_condstore = false
>     => New mail alert is shown.

(In reply to WADA from comment #73)
> If you can trust comment #65 by me, check difference between followings, please.
> - No mail.server.server#.use_condstore (identical to true if Gmail IMAP)
> - mail.server.server#.use_condstore = false

This seems to do the job, sorry for the delay in testing this. I've just had another instance where the alert didn't go off, disabled the condstore, and after restarting the alert showed up again for the next incoming message. Thus, let's see if that holds. I didn't change .max_cached_connections so far.
(In reply to rsx11m from comment #78)

CONDSTORE sounds relevant to problem.

FYI.
While watching IMAP log by DebugView with doing go Work Offline/go Work Online, with open [Gmail]/All Mail explicitly sometimes, but without open [Gmail]/All Mail sometims, funny phenomena was observed.

(a) no ENABLE CONDSTORE, no (CONDSTORE) for "select [Gmail]/All Mail".
    In this case, "STATUS INBOX" was seen at a newly opened anonymous
    cached connection. "no ENABLE CONDSTORE" looks phenomenon on
    first "SELECT" just after first login.

(b) When [Gmail]/All Mail is selected by "auto-sync",
    (by Updating folder: imap:// ... @imap.gmail.com/[Gmail]/All Mail)
(b-1) "UID fetch 1:* (FLAGS)" was issued (I saw a few times)
      just after "UID fetch 1:* (FLAGS) (CHANGEDSINCE ...)
   ENABLE CONDSTORE
   select "[Gmail]/All Mail" (CONDSTORE)
   UID fetch 1:* (FLAGS) (CHANGEDSINCE ...)
   UID fetch 1:* (FLAGS)
   => All UID is returned because of no CHANGEDSINCE.
      MODSEQ was normally returned.
(b-2) "UID fetch 1:* (FLAGS) (CHANGEDSINCE ...) only in many caes.
In both cases, upon new mail check by biff(by "performing biffs"), "UID fetch HighestUID+1:* (FLAGS)" was not seen in log. "DONE", "noop", "IDLE" only was repeated.
It looks phenomenon like next;
  because [Gmail]/All Mail is selected at a cached connection,
  STATUS for auto-sync or new mail check can not be issued.
These perhaps depend on Mbox status before go Work Offline/Online. And there also may be relevant to unstable or unexpected flagState->GetPartialUIDFetch what bug opener saw during test.

Note:
Because MODSEQ is seen in IDLE log for INBOX which was pasted to this bug by bug opener, and because bug opener carefully excluded affect by "new mail check by Biff", his case is perhaps following.
  No "ENABLE CONDSTORE"
  select "INBOX" 
  * OK [HIGHESTMODSEQ nnnnn]
  UID fetch 1:* (FLAGS) (CHANGEDSINCE ...)
  * mm FETCH (UID xxx MODSEQ (mmmmmm) FLAGS ( ... )
  IDLE
  * nn FETCH (UID yyy MODSEQ (nnnnnn) FLAGS () <= new mail notification
  DONE => header fetch
phenomenon like above (b) is not directly relevant to phenomenon of "when new mail is detected via IDLE, new mail alert is not shown".

Why flagState->GetPartialUIDFetch == true may be;
  When "UID fetch 1:* (FLAGS) (CHANGEDSINCE ...)" is issued,
  because of CONDSTORE is used, SetPartialUIDFetch(false) is not called.
Question.
Even if so, I think GetPartialUIDFetch == true is set when "UID fetch HighestUID+1:* (FLAGS)" is issued even when CONDSTORE is not used. Above "SetPartialUIDFetch(false) is not called" is applicable to "UID fetch 1:* (FLAGS) ..." only. 
If it's true, why alert is shown when CONDSTORE is not used?
needFullFolderSync is never reset if CONDSTORE is used?
Meaning of fPartialUIDFetch=true seems "CHANGEDSINCE is used because CONDSTORE is used".

fPartialUIDFetch = true is set by;
> (1) By nsImapFlagAndUidState object instanc creation
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapFlagAndUidState.cpp#79
> By calling ResetFlagInfo()
> (2) When "SELECT" is issued, ResetFlagInfo() is called.
> http://mxr.mozilla.org/comm-central/search?string=resetflaginfo%28%29&find=imap&findi=&filter=^[^\0]*%24&hitlimit=&tree=comm-central
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapFlagAndUidState.cpp#122
> (3) By m_flagState->Reset();, followed by  m_flagState->SetPartialUIDFetch(false); for "fall back to full flag sync" in nsImapProtocol.cpp.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#4036

In nsImapProtocol.cpp, m_flagState->SetPartialUIDFetch(false) is called if CONDSTORE is not used, and m_flagState->SetPartialUIDFetch(false) is NOT called if CONDSTORE is used and some other conditions exist("Just mark it as deleted" is an other condition).

Following comment and code looks to indicate that "m_flagState->GetPartialUIDFetch()=true" == "we did a CHANGEDSINCE fetch".
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#4028
> 4028         // if we did a CHANGEDSINCE fetch, do a sanity check on the msg counts
> 4029         // to see if some other client may have done an expunge.
> 4030         if (m_flagState->GetPartialUIDFetch())
> 4031         {
> 4032           if (m_flagState->NumberOfDeletedMessages() +
> 4033               mFolderTotalMsgCount != > GetServerStateParser().NumberOfMessages())
> 4034           {
And, following is perhaps "uid fetch 1:* FLAGS without CHANGEDSINCE on [Gmail]/All Mail just after uid fetch 1:* FLAGS (CHANGEDSINCE ...)" which I saw.
> 4035             // sanity check failed - fall back to full flag sync
> 4036             m_flagState->Reset();
> 4037             m_flagState->SetPartialUIDFetch(false);
> 4038             FetchMessage(NS_LITERAL_CSTRING("1:*"), kFlags);
> 4039           }
> 4040         }

"Number os unseen messages" is obtained by STATUS".
> 11 STATUS "[Gmail]/All Mail" (UIDNEXT MESSAGES UNSEEN RECENT)
>  * STATUS "[Gmail]/All Mail" (MESSAGES 853 RECENT 0 UIDNEXT 3310 UNSEEN 4) 

If STATUS is issued before selection of MBOX at a cached connection, "Number of unseen messages" can be obtained by STATUS command.
If Mbox is opened and selected at a cached connection before STATUS is issued for the Mbox,
- If no CONDSTORE, all flags are obtained by "uid fetch 1:*",
  and "number of unseen messages" can be obtained.
- If CONDSTORE is used, "uid fetch 1:* (CHANGEDSINCE ...)" is issued.
  Because (CHANGEDSINCE ...) is used, GetPartialUIDFetch() == true.

This is perhaps a reason of "chaos in GetPartialUIDFetch() and Unseen message count value with CONDSTORE".

If INBOX, it's selected at first cached connection which is opened just after first login after restart. So, there is no chance of "STATUS for INBOX", except when "go Work Offline, then go Work Online", and "max cached connections=1".

(In reply to al_9x from comment #59)
> numNewUnread calculated locally is >0 but since partialUIDFetch==true,
> it's set to NumUnseenMessages from the server, which is 0

"Unseen message of larger than Highest UID" is detected via IDLE by following unsol response.
  * 6 FETCH (UID 434 MODSEQ (122154) FLAGS ())
"Unseen message count" is not incremented by this response, and "not incremented" is partialUIDFetch==true case only?
I have advised participants in the MozillaZine thread to set use_condstore=false and one of them reported back that this resolved the issue and the notifications are working again after that change.
Whiteboard: [regression?] → [regression?][workaround: comment #73]
gmail recently added support for condstore. You could try telling thunderbird not to use condstore (it's a hidden pref, setting mail.server.default.use_condstore to false will turn it off for all servers). The partialuidfetch is done when we use condstore, so if turning off the use of condstore fixes the issue, that would be evidence that the partialuidfetch is involved.
Flags: needinfo?(mozilla)
Summary: no new mail alert or icon on gmail imap accounts (new profile) → no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, new mail alert by Biff is not shown when new mail is detected via IDLE)
(In reply to David :Bienvenu from comment #82)
> gmail recently added support for condstore. You could try telling
> thunderbird not to use condstore (it's a hidden pref, setting
> mail.server.default.use_condstore to false will turn it off for all
> servers). The partialuidfetch is done when we use condstore, so if turning
> off the use of condstore fixes the issue, that would be evidence that the
> partialuidfetch is involved.

partialuidfetch is absolutely involved (Comment 59 and Comment 64).  When true it overwrites the locally calculated numNewUnread with numUnreadFromServer which is wrong (0).  use_condstore=false does indeed disable partialuidfetch (confirmed in the debugger) and restore the alert and icon
https://getsatisfaction.com/mozilla_messaging/topics/no_new_email_notification_sound_or_visual also confirms condstore involved.

Is the regression criteria bug 436151 + something else?  Or just bug 436151?
(In reply to Wayne Mery from comment #84)
> Is the regression criteria bug 436151 + something else?  Or just bug 436151?

Well, the code mentioned in comment #59 was added by bug 517461, ostensibly to fix an incorrect unread count...
Flags: needinfo?(neil)
A simplest solusion:
(a) Don't set fPartialUIDFetch = true.
If CONDSTORE is used,
(a-1) upon select of Mbox:
        "uid fetch 1:*", fPartialUIDFetch = false.
(a-2) upon each "New mail check by Biff" :
        "uid fetch 1:* (CHANGEDSINCE ...)",  fPartialUIDFetch = false.
i.e. No one sets fPartialUIDFetch = true upon select of Mbox.
(a-2) is a solution of Bug 693204.

Performance gain by "uid fetch 1:* (CHANGEDSINCE ...) upon select + fPartialUIDFetch = true" should be re-considered after keeping consistency between "CONDSTORE support" and "Biff Alert" on "Unread count" again, and after keeping consistency among "STATUS by Biff and auto-sync when Mbox is not selected", "uid fetch x:* Flags by Biff when Mbox is selected", and "IDLE when Mbox is selected".
Blocks: 517461
Keywords: regression
Whiteboard: [regression?][workaround: comment #73] → [workaround: comment #73]
Duplicate of this bug: 903463
I have just tried backing out bug 517461 on a special try server build (as Neil mentioned in comment 85). Please could folks give the build a try:

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bugzilla@standard8.plus.com-32889705188d/

It is based on TB 24 beta.

I'm curious if this fixes the gmail issue, even though we might regress bug 517461...
biff is(In reply to Mark Banner (:standard8) from comment #88)
> I have just tried backing out bug 517461 on a special try server build (as
> Neil mentioned in comment 85). Please could folks give the build a try:
> 
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/
> bugzilla@standard8.plus.com-32889705188d/
> 
> It is based on TB 24 beta.
> 
> I'm curious if this fixes the gmail issue, even though we might regress bug
> 517461...

biff is working i.e. i get audio confirmations when i get gmail with this try build, anything else i should check?
We should probably try and confirm if some servers such as dovecot as mentioned on bug 517461 are definitely regressed again as well.
Flags: needinfo?(irving)
Duplicate of this bug: 921367
Issue seen here as well, against a Dovecot-2.2 server, which was also worked around by disabling CONDSTORE support.  If there's any chance this can also be fixed with TB 25 Beta (and go into 25 Final) that would be good..
There won't be a 25 final (the next major release won't be until 31.0), but a fix can come with a minor 24.0.x "dot" release. Given the tracking-thunderbird-esr24: 26+ flag, it is aimed for 24.0.3 in about 7 weeks from now (assuming that a fix is found until then).
I've been affected by this one for around a year... in my case, the current involved product versions are:

1) Dovecot 2.2.7-2.fc19.x86_64 (Fedora 19)
2) Thunderbird 24.1.0-1.fc19.x86_64 (Fedora 19)
3) Samsung Galaxy S3 with vendor supplied Android 4.1.2 and Samsung Email app

Scenario:

I have a small number of mailboxes for a particular account that I monitor via both desktop (Thunderbird) and phone (Samsung Email app). This seems like a very typical scenario.

Behaviour:

If I am looking at both the desktop and the phone, and I delete using Thunderbird, the phone picks up the update when it next syncs (set to every 15 minutes to conserve battery).

If I am looking at both the desktop and the phone, and I delete from the phone, Thunderbird doesn't seem to *ever* notice (without either a restart or a "Repair Folder") that the message has been deleted.

After reading this and a few other posts, I configured Thunderbird to no longer use condrestore. Now when I delete from the phone, Thunderbird shows the message deleted in seconds or less.

Oddly... I remember testing this and finding it working in around October, 2012. It broke at some point a few months later and has been broken for the last year. Unfortunately, I can't identify whether it was a software upgrade or a configuration change that broke Thunderbird. For example, one thing I remember doing at the time was adding more accounts to my Thunderbird configuration. Did that break it? Or was it the upgrade to Fedora 17 or Fedora 18? I don't know...

In any case, it really seems like the Thunderbird support of CONDSTORE is broken, or the Dovecot implementation of CONDSTORE is broken.

If you have any tests you'd like me to do please let me know... I think I can easily reproduce this and I don't mind turning on logging if it will help you...
I'm running dovecot 2.2.13/thunderbird 24.6 and having anomalous behavior (constant redownloading of directories as in https://bugzilla.mozilla.org/show_bug.cgi?id=806760).  Not to redirect this thread, but there are some hints that it could be related to CONDSTORE.  Has the CONDSTORE bug reported here been fixed?  The last update shows a tentative target of 24.3.
Flags: needinfo?(nobody)
I also get this behaviour in Thunderbird 24.6.0, with a Zimbra 8 IMAP mailserver, on Windows 7 x64.

Thunderbird identifies new mail and bolds it, but does not trigger the tray icon, popup overlay, or sound. The "New Mail Attention" extension, however, works.

If I disable "mail.server.default.use_condstore" and restart, the notifications all work. Three tests with and without the option continue to demonstrate this.
Flags: needinfo?(nobody)
I can confirm that for Thunderbird 31.3.0, disabling the condstore helps.
Even with CONDSTORE disabled it works far from useable solution (I have just switched from POP3 to IMAP protocol)

- I use hosted email (not Google) with dovecot, it supports IDLE
- I have about 15 subfolders (use server message filters)
- Maximum number of connections is default 5 value

Symptoms:

- notification of new emails in root INBOX folder is reliable and fast (taking advantage of IDLE) regardless of 'Check for new message every N minutes' is enabled or not.

- notification of new emails in INBOX subfolders is unreliable and works (delayed) only if 'Check for new message every N minutes' is enabled. Sometimes it is immediate, sometimes it takes the check period. It is completely random without any pattern.

I have log of my session I can provide but it is over 40 MB so I would need to know what to focus on. I would like to have it resolved because compared to absolutely reliable POP3 this is just mess.
(In reply to Petr Vones from comment #98)
> - notification of new emails in INBOX subfolders is unreliable
> and works (delayed) only if 'Check for new message every N minutes' is enabled.

As written in bug summary, this bug is obviously for following problem, and is only for following problem.
   (a) If CONDSTORE is supported by IMAP server,
         and if mail.server.default.use_condstore=true(deffault) and mail.server.server#.use_condstore=false is not set,
   (b) when new mail is notified via IDLE, <= obviously, problem when IDLE worked as designed, as expected, on an Mbox
   (c) new mail alert is not shown.
This bug is never for such phenomenon like "IDLE doesn't work as you expect/want" or "IDLE's functionality is different form your expectation".
If you want to know IMAP level flow with IDLE, I recommend you to get IMAP log and check log content by yourself with:
   Max cached connection=1, IDLE is enabled or not. Do as you like. CONDSTORE support is disabled.
   Because only one connection is permitted, all communication with IMAP server is done via single connection only.
   So, each step of , for example, "move mail from Inbox to Trash", is executed at single connection only.
        All steps in "fetch from Inbox + copy to Trash + fetch from Trash + store \Delete at Inbox" is serialized.
   And, as only one connection is available, you can see and understand "IDLE is effective on single Mbox only at a connection".
No longer blocks: 517461
See Also: → 1123094
Summary: no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, new mail alert by Biff is not shown when new mail is detected via IDLE) → no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE)
Summary: no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE) → no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE)
See Also: condstore-default
(In reply to neil@parkwaycc.co.uk from comment #85)
> Well, the code mentioned in comment #59 was added by bug 517461, ostensibly to fix an incorrect unread count...

Phenomenon and cause of Bug 517461 looks Bug 524902. If so, patch by bug 517461(merely a workaround on Unread count display) is not needed any more.
Can some one backout change by bug 517461? If backed out, this bug will be resolved.
Because pref("mail.server.default.use_condstore", false); is set in mailnews.js by bug 912216, even if "Wrong Unread count if condstore is used" of bug 517461 occurs again, it won't occur in user's enviroment unless user intentionally set mail.server.default.use_condstore"=true.
So, backout of the change is never dangerous.
Summary: no new mail alert or icon on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE) → no new mail notification (alert or icon) on gmail imap accounts (new profile) (When CONDSTORE is used, because of change by Bug 517461, new mail alert by Biff is not shown when new mail is detected via IDLE)
I've been having this too. I use an addon that keeps TB systrayed when I don't need it. When I get a new email on my gmail, the tray icon displays the number of new mails, so it's rather obvious TB's subsystems are aware of the new email since the add-on catches the event. And yet, TB's own popup is never shown and the sound is never played for gmail. All my other accounts on other providers work just fine.

This smells very foul because the symptoms here suggest part of the UI's behavior is dependent on the mail provider for new mail events, instead of just using the subsystem in TB that IS detecting new mails reliably and is used by e.g. my tray addon.

Like so many issues on this bugtracker, this fairly simple, almost stupid bug has been open for years with no fix in sight. Tell me Mozilla, are you guys short on developers or something? Does your entire Thunderbird branch consist of 2 people, one of whom is only a manager and not a developer? How is it possible for Thunderbird to neither get even the most basic bugfixes nor see any real development changes (I've been using TB for years and don't recall ever seeing any differences after updates) for years?
Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.
See Also: → 334483
You need to log in before you can comment on or make changes to this bug.