482472 - [BlackBerry] BES causes Exchange emails via IMAP to disappear and reappear

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[Julien BETI]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020409 Iceweasel/3.0.6 (Debian-3.0.6-1)
Build Identifier: version 2.0.0.19 (20090103)

Using Thunderbird to connect to an Exchange Server using the IMAP protocol, with an account linked to a BlackBerry using a BlackBerry Enterprise Server (BES) leads to very strange behaviours.

E-mail appear, then disappear few seconds after, then reappear sometimes minutes after, are duplicated, etc, etc. Turnig Thunderbird off and on again seems to "reset" everything for existing mails.

Using the same Exchange Server, people who do not have a BlackBerry do not experience the same issues, it just works nicely (as most as it can knowing the mail server involved...)

Reproducible: Sometimes

Steps to Reproduce:
1. Connect to an Exchange server using IMAP, with an account tied to Blackberry Enterprise Server
2. Receive e-mails
3. Manipulate received e-mails
Actual Results:  
E-mail appear, then disappear few seconds after, then reappear sometimes minutes after, are duplicated, have a blank body content, etc, etc. Turning Thunderbird off and on again seems to "reset" everything for existing mails.

Expected Results:  
Have the same standard behaviour of an IMAP client connecting to an IMAP server (or teach RIM architects what interoperability means, your choice ;-) )

I gave in the URL field a thread on the BlackBerry Forums web site showing you that:
-1- It is not an isolated case. The first post give technical information.
-2- Thunderbird itself seems to not be the problem
-3- The post from BlackBerry Enterprise Server "expert" is an illustration of... I let you judge, but basically it seems they will not move a finger.

If it is confirmed that the problem definitively not come from Thunderbird, may I ask you to please give us some hints, tips, or whatever can help us who cannot (and don't want!) to switch to another mail reading program. As stated in the post, impacted users are mostly Linux ones for whom OWA (Outlook Web Access) or Outlook is just not a solution:
- OWA is "defective by design" on everything else but IE which don't run on Linux
- Outlook does not run on Linux

I don't know if this should be classified as a bug or as an enhancement. If this is not a Thunderbird issue, maybe it can be reclassified as an enhancement and introduce a cooperation between Mozilla and BlackBerry ?

Thanks a lot.

Comment 1

[bugzilla.20.egoine]

I saw this on windows tbird, so I don't think it should be classified as a linux only issue.

Comment 2

[WADA:World Anti-bad-Duping Agency]

Get IMAP log and check flow first.
> Getting log : See Bug 402793 Comment #1
> IMAP command/response : http://www.faqs.org/rfcs/rfc3501.html
If and only if Tb's fault is seen in log, and if log analysis by developers is required, attach log file to this bug(never paste, please).
If time related issue is suspected, get log with timestamp on MS Win using DebugView. See Bug 402793 Comment #6 for NSPR log & DebugView.

Comment 3

[Ludovic Hirlimann [:Usul]]

Julien , Bugzilla.20.egoine could you provide impa logs as explained in https://wiki.mozilla.org/MailNews:Logging ? (would be vrey nice if logs would be produced with Thunderbird 3 beta2 that one can get from http://www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/)

Comment 4

[Julien BETI]

Attachment: Test case 01

Test Case 01:
> Thunderbird version: 2.0.0.19 (20090103) (Debian's Icedove flavour)
> Platform: Linux Debian SID/Experimental
-1- Waiting for a new mail (Subject: Test 02)
-2- New mail appears - No user action
-3- Mail disapears
-4- Mail reappears immediatly

This is the simplest case. I will try to generate more complex one (duplication, disapearing for several minutes, etc...) But that's not very easy: working environment with sensible data passing threw the log. Setting up a complete environment is not that easy as it requires a working BlackBerry account...)

We will also try to reproduce on Thunderbird 3 Beta 2

Comment 5

[Jérémie Grauer]

Attachment: Imap log on TB3 beta 2

here is the log for the same issue on TB 3 beta 2.

The mail was disappearing 3 times before settling down in my inbox.

Comment 6

[WADA:World Anti-bad-Duping Agency]

Test Case 01:
>(snip)
> -2- New mail appears - No user action

What part of IMAP flow in your log is evidence of Tb's fault?

Your IMAP server looks to return same mail data for UID=647419 & UID=647420.    
> 123 UID fetch 647419 => Subject: test 02, Message-ID: <[e-mail]> 
> 128 UID fetch 647420 => Subject: test 02, Message-ID: <[e-mail]>
Did you copy a mail twice to INBOX folder as "new mails arrived" in your test?

What was UID of mail(s) on which problem occurred? Display "Order Received" column during test. Value in the column is UID of mail when IMAP folder.

Comment 7

[Julien BETI]

(In reply to comment #6)
> What part of IMAP flow in your log is evidence of Tb's fault?

I isolated in the log the content written by Tb while the given scenario occured. The evidence is "there is an issue", but as I said in the bug description comment 0, Tb not seems to be at the origin of the problem. This is also the conclusion of the first post of the newsgroup thread linked to this issue (See URL field)

> Did you copy a mail twice to INBOX folder as "new mails arrived" in your test?

In this test case, there is no user action at all... I did not touched my keyboard nor my mouse ;)

There is more complex test case, more difficult to reproduce, where a mail is received, then being read/displayed by the user, then it disappears, and then reappears (seconds or sometimes minutes later...) as 2 separate distincs copies of the e-mail.

> 
> What was UID of mail(s) on which problem occurred? Display "Order Received"
> column during test. Value in the column is UID of mail when IMAP folder.

In the test case 01, only one mail was displayed. It appeared, disapeared, and then reappeared again.

Comment 8

[David :Bienvenu]

from this log, the imap server is telling us there is new mail, and then giving us the exact same message three times, but saying it's a different message each time. I'm not sure why a message would disappear in the UI, unless the message threading code is getting confused by having the exact same message pretend to be three different messages...

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> > What was UID of mail(s) on which problem occurred?
> In the test case 01, only one mail was displayed. It appeared, disapeared, and then reappeared again.

UID was one for first instance? (UID=647419 in your log for case 01)
If so, similar issue to Bug 414723 is possibly involved in your case.
What mail(s) of what UID is displayed at thread pane after "Re-build Index"?

Comment 10

[Jérémie Grauer]

(In reply to comment #9)
> UID was one for first instance? (UID=647419 in your log for case 01)
> If so, similar issue to Bug 414723 is possibly involved in your case.
> What mail(s) of what UID is displayed at thread pane after "Re-build Index"?

The same UID is displayed when I rebuild the index.

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #10)
> The same UID is displayed when I rebuild the index.

What happend is following, isn't it?
1. New mail(UID=647419) arrives.
2. Serve notifies it via IDLE, then Tb issues uid 647419 fetch, and issue IDLE
3. Server notifies illusional mail of UID=647420 via IDLE, then Tb issues uid
   647420 fetch. Server returns same mail data as UID=647419. Tb issues IDLE.
4. Tb knows UID=647419 & UID=647420, and displays them at thread pane.
5. After a while, LIST etc. occurs, and Server doesn't return information
   about UID=647420. It's same as "UID=647420 is deleted & expunged by other
   client" for Tb.
6. If Rebuild Index is executed, server doesn't return illusional UID=647420.

Similar situation can occur with Gmail IMAP, due to particular design and implementation of Gmail/Gmail IMAP.
 1. Copy a mail to a Gmail IMAP mail folder. (say UID=1)
 2. Copy the mail again to same Gmail IMAP folder, and Gmail IMAP reruns OK
    (Tb assumes UID=2)
 3. Because Gmail doesn't hold duplicated mail data, Gmail IMAP won't generate
    mail of UID=2 in the IMAP folder.
 4. When next access to the IMAP folder, Gmail IMAP reurns UID=1 only.
    So, it looks for Tb "UID=2 is deleted & expunged by other client".  

Anyway, illusional mail of UID=647420 is fault or design of your Server, or fault of external software such as mail proxy or mail filtering software.

Comment 13

[Julien BETI]

I personally solved the issue installing a DavMail server http://davmail.sourceforge.net/

Comment 14

[Paul miranda]

I see a similar problem and I don't have a BB.
Emails sent from outlook to me via a mailman list will appear in the t'bird inbox, but the next time I get mail (or t'bird automatically checks for mail) the email will disappear. The next time I manually get mail the email will reappear as if it was new.
If email is sent to me directly from Outlook or via SMTP and then mailman it behaves normally. I haven't gotten logs yet.

Comment 15

[dbrand10mozilla]

(In reply to Paul miranda from comment #14)
> I see a similar problem and I don't have a BB.
> Emails sent from outlook to me via a mailman list will appear in the t'bird
> inbox, but the next time I get mail (or t'bird automatically checks for
> mail) the email will disappear. The next time I manually get mail the email
> will reappear as if it was new.
> If email is sent to me directly from Outlook or via SMTP and then mailman it
> behaves normally. I haven't gotten logs yet.

I'm seeing exactly the same behavior.  I don't have a BB either.

Comment 16

[Jim McCormick]

I am seeing the same behavior, without a BB.  After using TB for a while, I click "Get Mail", and one of my messages disappears.  I click "Get Mail" again, the message re-appears.  Do it a 3rd time, and the message disappears again.  Etc.

If I restart TB, or change the IMAP server name (different name that resolves to the same address) then it clears the issue.

Comment 17

[Wayne Mery (:wsmwk)]

does https://bugzilla.mozilla.org/show_bug.cgi?id=513309#c7 and further comments describe your problem?  If so, we might kill this bug report incomplete, given comment 13

Comment 18

[Jim McCormick]

I don't think it is the same as 513309.  That one talks about a new message disappearing and reappearing within a couple of seconds, something to do with multiple version of the mail.  Here we are talking about a different email in the inbox that disappears and reappears which clicking "Get Mail".

Comment 19

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 20

[Wayne Mery (:wsmwk)]

Does anyone still see this?

Comment 21

[Phoenix]

Resolved per whiteboard