544439 - gmail imap message not marked read, after being marked read using web interface (only Inbox is subscribed)

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[R.]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

if I mark a message as 'read' in the gmail interface, the read/unread status is often (not always) not updated in thunderbird (it works quite fine in mail.app, though).

Reproducible: Sometimes

Steps to Reproduce:
1.set up imap synchronization with gmail inbox only
2.mark message(s) as read using the web interface
3.
Actual Results:  
the message often fails to get marked as read in thunderbird --- even though thunderbird is obviously getting new mail and syncing deleted mails, etc.

Expected Results:  
message should be marked as read the next time thunderbird checks for new mail.

the read status can be synced by a restart (for thunderbird) or by 'rebuilding' the inbox folder -- which unfortunately erases all custom column display settings.

Comment 1

[WADA:World Anti-bad-Duping Agency]

read/unread status of an IMAP mail is held as "\Seen flag of a mail"/"no \Seen flag of a mail" at IMAP server. There is no way to push status change(\Seen flag <-> no \Seen flag) of an IMAP mail to client from imap server. read/unread can be obtaind only by "uid xxx fetch" request for imap flags(Show "Order Received" column. It's uid of imap mail). In contrast to it, new mails/deleted mails can be notified to client from imap server via IDLE interface. It's the reason of difference you saw.

As "use of IDLE" is defaulted, connection for Inbox is always kept, thus "fetch flags" for all "mails of no \Deleted flag" is not executed unless special request is done by use(e.g. close of Inbox and re-open of Inbox, collaps account and re-expand account, ...).
Get IMAP log, and see IMAP level flow.
> https://wiki.mozilla.org/MailNews:Logging
> http://www.mozilla.org/projects/nspr/reference/html/prlog.html#25328
> SET NSPR_LOG_MODULES=timestamp,imap:5 (MS Win example)

Next may be a workaround.
1. Disable "use IDLE command"
2. Enable "check new message every NNN minutes"
   (never set too small interval) 
"fetch of flags of all non-deleted mails" may be executed upon each new mail check. Check IMAP log after change, please.

> 1.set up imap synchronization with gmail inbox only

Why do you do so, even though "copy or move to [Gmail]/Trash" is required to delete mail permanently via Gmail IMAP? (equivallent to "delete" or "move to Trash" at Web interface) 
New mail check is sufficient for you when you use Thunderbird and Gmail IMAP?
No need to view sent mails using Gmail's SMTP server or Gmail's Web interface? ([Gmail]/Sent Mail) 

By the way, "[Gmail]/All Mail" holds all mails of Gmail. Don't set "offline use=on" if the mail folder is big(Folder Properties/Synchronization).
See Bug 537498 and Bug 532323.

Comment 2

[R.]

Wada -- thanks for the extended comments.

I'm syncing my wife's inbox so that I can let her know if important mail arrives -- hence the inbox-only approach.

My IDLE was disabled and I was checking mail every one minute. How less frequently should I check?

I should mention here that Mail.app works fine with the same settings (no IDLE, check every minute) -- I use it for my GMail account.

Comment 3

[WADA:World Anti-bad-Duping Agency]

Try next settings, please.
  (a) Number of cached connections=0 (default : 5)
  (b) Disable IDLE command use 
  (c) Check every 5 to 10 minutes (1 is too small for many IMAP server)
By (a) connection is closed upon every mail check. So, I guess "fetch flags of all no-deleted mail" is executed upon every mail check by (c).
Get IMAP log with timestamp for both next settings with Inbox only.

> I should mention here that Mail.app works fine with the same settings (no IDLE, check every minute)

Get IMAP log with timestamp for both next settings.
  (case-1)
    - Number of cached connections=0 (default : 5)
    - Disable IDLE command use 
    - Check new messages every 10 minutes
  (case-2)
    - Number of cached connections=5 (default : 5)
    - Enable IDLE command use 
    - Disable new mail check(upon statup, every NNN minute)
When IMAP mail folder holds many mails([Gmail]/All Mail is an exmple), "fetch flags of all no-deleted mail" upon each access is very expensive for both IMAP client and IMAP server. So setting of (case-2) is initially set by Tb.
If connection for Inbox is kept and IDLE is used for the connection, new mail arrival in Inbox is notified from IMAP server, then there is no need to check for new message every NNN minutes.
I think default of Mail.app is setting of (case-1).

Comment 4

[R.]

Hi Wada -- thanks for the detailed instructions. The thing that worked was disbling IDLE usage -- problem SOLVED.

However, FYI, if I set the number of cached connections to 0, it automatically reset to 5 every time thunderbird performed a mail check! Also, I'm still checking mail every 1 minute -- the read status gets updated every 5mins or so (I expect this has to do with the 5 uncached connections). This is OK for my purpose.

Comment 5

[R.]

bug status update...

Comment 6

[R.]

I was too hasty in saying that the fix worked. The situation has improved, but random messages are still left marked read after they are marked as read on the gmail server. The only way to correct that is to quit and restart Thunderbird -- collapsing and re-expanding the account on the sidebar, etc does not work. My server settings are: IDLE disabled, poll every (tried 1-5 mins), maximum number of server connections to cache = (tried 1-5). Since I am just keeping an eye on the inbox, only that folder is synced to my machine.

Comment 7

[R.]

Typo: "random messages are still left marked UNread"

Comment 8

[R.]

I don't know if this is useful, but my settings are such that messages are not automatically marked as read when I open them inside Thunderbird.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Probably same phenomenon as Bug 518581(flag change at Inbox is not refleted to [Gmail]/All Mail). Setting Dependency to bug 518581 for ease of search/tracking.

Comment 10

[R.]

I am not syncing [Gmail]/All Mail, only the Inbox.

Comment 11

[R.]

One trend I observed which might be helpful in tracking the cause: if a bunch of new mails arrive and are synced to Thunderbird, if all are marked read from gmail, only the last of these mails is marked read in Thunderbird during the next polling cycle (determined by the poll interval and the number of cached connections settings).

Comment 12

[WADA:World Anti-bad-Duping Agency]

Have read and understand bug 518581 comment 6?
Have you checked IMAP log for the next polling?

Comment 13

[WADA:World Anti-bad-Duping Agency]

Your case can be said same phenomenon as Bug 518581 - UID fetch 1:* (FLAGS) is not issued then flag change can not be detected. However, your case is very very special - Inbox only, so there is no chance to issue the command, perhaps, after some changes.

R., can you check next?
  1. Max cached connections = 1
  2. Subscribe a Gmail IMAP folder additinally.
  3. Click Inbox, click other folder, click Inbox 
  4. After Read/Unread status change at Gmail Web,
     4-1. Click other folder, then click Inbox again.
          Read/Unread status is changed?
     4-2. If not reflected, click other folder, wait for a while,
          and try 4-1 several times.
If above doesn't work, get IMAP log, please.

Comment 14

[Wayne Mery (:wsmwk)]

R, can you confirm comment 12, and check comment 13?

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> if I set the number of cached connections to 0, it automatically reset to 5 very time.

You are right. If 0, Tb looks to use default=5. Sorry for my wrong instruction.

Comment 16

[WADA:World Anti-bad-Duping Agency]

FYI.
As found in bug 512745, Gmail IMAP doesn't return unsolicited response to IDLE for flag change. So, enabling IDLE can not be a solution of this bug's case.

Comment 17

[Joshua Cranmer [:jcranmer]]

RESOLVED INCOMPLETE due to lack of response to previous question. If you feel this change was made in error, please respond to this bug with your reasons why.