540554 - IMAP: In some folders a lot of mails is false marked as unread. (CONDSTORE is supportted by IMAP server)

Status: RESOLVED FIXED

(Thunderbird :: Folder and Message Lists, defect)

Description

[Joachim]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.9.2pre) Gecko/20100118 Namoroka/3.6pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de-DE; rv:1.9.1.7) Gecko/20100118 Lightning/1.0b1 Shredder/3.0.2pre

IMAP: In some folders a lot of mails is false marked as unread.

I have to select "Properties" ("Eigenschaften") and than "rebuild index" (Index wiederherstellen). Now all is o.k.
~1 hour later, exacly the same mails get marked unread again.
This is since start of this year. (3.0.1pre).

Reproducible: Always

Steps to Reproduce:
1.Connect to IMAP (cyrrus:
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS LOGINDISABLED] Dovecot ready.
2.Wait....
3.Several mail in some folders suddenly are marked unread
Actual Results:  
Readed mails are marked unread

Expected Results:  
Readed mails keep marked read

Comment 1

[Wayne Mery (:wsmwk)]

Joachim, can you provide an imap log of this activity
https://wiki.mozilla.org/MailNews:Logging

Comment 2

[Joachim]

Attachment: imap.log from a short thunderbird session

I started tb and waited till i saw 2 messages marked wrongly as unread.
These mails are in Genetik.Genetik, dated 30.11.2000 and 06.12.2000, 7 messages at all, these are the latest two

Comment 3

[Wayne Mery (:wsmwk)]

thanks Joachim

Comment 4

[David :Bienvenu]

This is really a dup of an other condstore bug, but it has an excellent log, so I'm going to do the work here.

Comment 5

[Joachim]

A test with TB 3.01 on WinXP gave the same bug.

Comment 6

[David :Bienvenu]

I've requested a try server build for a fix for this on the 3.0 branch - http://tinderbox.mozilla.org/showbuilds.cgi?tree=ThunderbirdTry

Joachim, Can you try this build when it comes out in a couple hours?

Comment 7

[David :Bienvenu]

Attachment: proposed 3.01 fix

Comment 9

[David :Bienvenu]

Windows build is here - http://s3.mozillamessaging.com/build/try-server/2010-01-20_15:[e-mail]-1264029848/[e-mail] - please let me know if this fixes the problem for you...

Comment 10

[Joachim]

O.K.

I tested for 5 houres on Linux and 3 on Windows.
The bug did not hit me any more.
Thanks

Comment 12

[David :Bienvenu]

Attachment: right patch

This has been reported to fix this for one user so far.

I'm going to write unit tests for nsImapFlagAndUidState, instead of worrying about adding condstore support to the fake imap server...

Comment 15

[Joachim Jablon]

I have the exact same problem. (it's a curse on the people named "Joachim" ^^).

I'll test the build and tell you if it works (or mainly "if I don't come back, then it works and thanks")

Comment 16

[Jeroen van der Ham]

Bug 535404 and 517461 appear to be a duplicate of this bug. These are reported to be fixed.

There was a comment that this fix was supposed to land in TB 3.0.1, but it doesn't seem like it did. The nightly Shredder appear to have the fix though.

Comment 18

[Pyther]

This patch seems to resolve the issue for me.

Comment 19

[David :Bienvenu]

Attachment: patch with c++ unit test

This adds a c++ unit test that fails w/o the fix, and succeeds with it. The unit test can be run from objdir/mozilla/dist/bin/TestImapFlagAndUidState.exe

Comment 21

[Joachim Jablon]

The build doesn't work for me, i still get old message that get unread on a few folders (always the same).

Comment 22

[David :Bienvenu]

Joachim Jablon, you might try turning off condstore support (config editor, toggle mail.server.default.use_condstore to false) , select those folders, see if it corrects the counts. Then turn condstore support back on, and see if the problem re-occurs.

Comment 23

[thunderbird]

Turning off condstore works for me on 3.0.1.

Comment 24

[Dennis]

Works for me too when turning off condstore.

Comment 25

[Joachim Jablon]

Yup, works fine. Sorry I didn't get that "condstore" reference on the first place, a little bit of research on my side would have easied your work. Lesson taken ^^'

Thank you for spending time on that one ! Great job ^_^.

Comment 26

[Luigi Rosa]

Confirmed that turning off condstore solved the problem (TB 3.0.1 on Ubuntu and Dovecot 2.0 beta 1)

Comment 27

[David Halik]

It might be more useful if those of you turning off condstore left it on and instead tested David's 3.0.2pre build to make sure that it is indeed patched. They are aware that condstore support is the issue and turning it off is a quick fix, but the patch needs to ensure that TB 3.0.2 is working properly with condstore on.

Comment 28

[Luigi Rosa]

David, I would be happy to, but right now I only have two linux boxes at hand. Maybe in the afternoon or on Monday I can try David's build

Comment 29

[Justin Dolske [:Dolske]]

(In reply to comment #9)
> - please let me know if this fixes the problem for you...

I've been running the OS X build from your try push for a couple of days now, and it seems to have fixed the problem for me... No instances of folders becoming marked unread, and it was happening to me daily before.

Comment 30

[thunderbird]

David, the pre build from #9 works with condstore for me.

Comment 31

[Ben Webb]

I can confirm that turning off condstore fixes bug. Running 3.0.1 on XP using omnis.com imap mail.

Comment 32

[Harald Dunkel]

I tried the patch on top of 3.0.1: Seems to work.

Comment 34

[Daniel]

I used the Mac OSX version of the pre build from #9 for 8 hours now, and can confirm it works for me as well.

Comment 35

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: nspr log with 3.0.2pre

I just tested 3.0.2pre (build id 2010012503) and it does not fix the issue for me. Disabling condstore works for me though.

I just before line 13682, the message count for Folder .projects.mozilla went up to ~1500.

Around line 14007, I selected "mark folder read". Then I selected other folders.

I went back to .projects.mozilla around line 14042, which caused the unread message count to reach ~1500 again. There is a total of ~3000 messages in that folder.

My IMAP server is dovecot, I believe version 1.2.9.

Comment 37

[David :Bienvenu]

fixed for 3.02 - nightly builds from tomorrow on will have the fix.

Comment 38

[David :Bienvenu]

marking fixed - the TFV is bogus, but we don't have TFV's for 3.0x; we're using status flags instead. This bug doesn't exist on the trunk.

Comment 39

[Jay Levitt]

This also fixes my problem on OS X (entire folders would generally become unread when condstore was enabled).

Comment 41

[David :Bienvenu]

I landed the c++ unit test on the 3.0x branch.

Comment 44

[javad]

my bug 543224 though marked as duplicate - if i see this whole communication - there could be some difference. 
1. I am using POP and not imap.
2. My problem is not wrong mails or already read mails marked unread - problem is say i have 25 unread mails - the folder name with bracket showing count of unread messages may say 5 or 10 or any number as unread mails but not 25.

anyway - still you might be correct as i do not know much in details, i just wanted to convey the difference.

Comment 49

[Matthias Fechner]

I can commit that version:
http://s3.mozillamessaging.com/build/try-server/2010-02-21_16:[e-mail]-1266797270/[e-mail]
Fixed the problem for me too.

Comment 50

[K Moser]

I'm using 31.1.2 (Windows) and have this problem. I'm using POP exclusively (never IMAP).

If it matters, I started by importing my old Eudora email. I had to go back and mark much of it as "read" even though it had been read ages ago. For a few days many of the old imported emails (often an entire folder) would sporadically revert to "unread". I thought I had it under control but once in a while it pops up. It even marked some as unread after I did a multi-folder search. I don't know how to reliably reproduce this.

Comment 51

[rsx11m]

This bug is closed and was addressing an issue with the IMAP implementation. Please open a new bug report with a detailed description of the issue as it manifests itself using POP now.