363008 - "Rebuild index" results in doubled message count

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect)

Description

[RickieES aka Ricardo Palomares]

Since I switched to SM 1.1b, I've noticed that, when I recreate the index using the [Folder] -> [Right click] -> [Properties] -> [General Info tab] -> [Rebuild Index] button, message number gets doubled. For instance, I've just compacted my movemail/Inbox folder and rebuilt the index, with these results:

- Message count: 78
- After selecting all msg. on msg. list pane, selected msg. count: 39
- Running 'grep -c -i "Message-ID: " Inbox': 39
- Inbox file size: 9 907 638 bytes

This happens with other folders in movemail and local folders, but not on newsgroups (I'm using leafnode, and the problem is other, but doesn't fit to this bug). I can't actually test it on IMAP or POP accounts right now (although if it was necessary, I can create accounts and test it).

Comment 1

[David :Bienvenu]

Thunderbird has this problem too, so it's a core issue. If you click away from the folder and back, the counts are correct. 

Comment 2

[David :Bienvenu]

this is going to be specific to local folders and pop3 account folders, which are also local. 

Comment 3

[Andrew Schultz]

This regressed between seamonkey builds 2006-09-19-09-trunk and 2006-09-20-09-trunk, bug 351685.  So this affects 1.8 branch (but not 1.8.0 because 1.8.0 doesn't have the rebuild index button).

Comment 4

[WADA:World Anti-bad-Duping Agency]

Are there any error messages in Error Console(Javascript console)?
See Bug 355194.

Comment 5

[Fernando Hartmann]

The error console shows the error reported in https://bugzilla.mozilla.org/show_bug.cgi?id=377239

Comment 6

[RickieES aka Ricardo Palomares]

(In reply to comment #5)
> The error console shows the error reported in
> https://bugzilla.mozilla.org/show_bug.cgi?id=377239
> 


As the original reporter, I confirm the console error to be the same of bug 355194 and 377239:

Error: uncaught exception: [Exception... "Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMsgFolder.updateFolder]"  nsresult: "0xc1f30001 (NS_ERROR_NOT_INITIALIZED)"  location: "JS frame :: chrome://messenger/content/widgetglue.js :: RebuildSummaryFile :: line 284"  data: no]

Comment 7

[WADA:World Anti-bad-Duping Agency]

A patch for issue of Bug 196732 Comment #89 (seems to be problem of my Bug 377239)    has been landed on trunk on 2007/4/12, and the patch has resolved problem of Bug 377239, although problem of Bug 355194(Error message when Rebuild Index) still remains.

To bug opener(and comment posters who expeirenced the problem) :
Does the problem of this bug(doubled message count) still remain on latest trunk?  

Comment 8

[RickieES aka Ricardo Palomares]

(In reply to comment #7)
> A patch for issue of Bug 196732 Comment #89 (seems to be problem of my Bug
> 377239)    has been landed on trunk on 2007/4/12, and the patch has resolved
> problem of Bug 377239, although problem of Bug 355194(Error message when
> Rebuild Index) still remains.
> 
> To bug opener(and comment posters who experienced the problem) :
> Does the problem of this bug(doubled message count) still remain on latest
> trunk?  
>


I've downloaded latest Linux es-ES trunk build from here:

ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk-l10n/

and the problem remains, but it must be noted that build date is 04/04/07 (why?). Er... Am I using the wrong directory, or should I just use the latest en-US trunk build?

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> should I just use the latest en-US trunk build?
Yes. 

Comment 10

[RickieES aka Ricardo Palomares]

(In reply to comment #9)
> (In reply to comment #8)
> > should I just use the latest en-US trunk build?
> Yes. 
> 

I'm sad to say that bug persists in the 18/Apr/07 en-US Linux build. :-(

BTW being a core bug, once it gets fixed, will we see both Thunderbird and SeaMonkey fixed, or should I file another bug specifically for SeaMonkey?

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #10)
> I'm sad to say that bug persists in the 18/Apr/07 en-US Linux build. :-(

Does problem occur when the mail folder file is copied?
 0. Shutdown SeaMonkey (All of browser, mail, composer, download manager, ...)
 1. Copy Inbox file(not Inbox.msf) to a file(say Inbox-2)
    under your mail directory, thru terminal or file manager.  
 2. Restart SeaMonkey
    2.1 Inbox-2.msf is recreated and mail folder of Inbox-2 appears.
    2.2 Message count may be ???. This is already known problem.
        Click the Inbox-2 folder. Message count will be correctly displayed.
 (Q1) Does your problem occurs even on this copied Inbox-2?
 3. "Compact folder" of Inbox-2
 (Q2) Will file size of Inbox-2(not Inbox-2.msf) be reduced?
 (Q3) Does your problem still occur on Inbox-2?  

By the way, since core bug, problem is usually in code which is used commonly by Tb & SM, then fix will be applied on both when problem will be FIXED.

Comment 12

[RickieES aka Ricardo Palomares]

(In reply to comment #11)
> Does problem occur when the mail folder file is copied?
>  0. Shutdown SeaMonkey (All of browser, mail, composer, download manager, ...)


I'm testing with Thunderbird trunk (en-US), not SeaMonkey.


>  1. Copy Inbox file(not Inbox.msf) to a file(say Inbox-2)
>     under your mail directory, thru terminal or file manager.

Does it matter too much if I choose Sent instead of Inbox? I had SeaMonkey open when I imported the mail configuration in TB and Inbox was not copied.


>  2. Restart SeaMonkey
>     2.1 Inbox-2.msf is recreated and mail folder of Inbox-2 appears.
>     2.2 Message count may be ???. This is already known problem.
>         Click the Inbox-2 folder. Message count will be correctly displayed.

Everything OK until here.

>  (Q1) Does your problem occurs even on this copied Inbox-2?

Yes, it happens (on Sent-2).

>  3. "Compact folder" of Inbox-2


Done, first with a folder not needed compacting, then deleting a bunch of e-mails so compacting actually did something.


>  (Q2) Will file size of Inbox-2(not Inbox-2.msf) be reduced?

Yes, it was compacted as intended (as always, I'm talking about Sent-2). The message in the error console persists:

Error: uncaught exception: [Exception... "Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMsgFolder.updateFolder]"  nsresult: "0xc1f30001 (NS_ERROR_NOT_INITIALIZED)"  location: "JS frame :: chrome://messenger/content/widgetglue.js :: RebuildSummaryFile :: line 249"  data: no]


>  (Q3) Does your problem still occur on Inbox-2?  

Yes, the message count gets added every time I hit the "Rebuild Index" button.

One thing I've noted is that the .msf file grows slightly after exiting the Folder Properties dialog:

- I right-click in Sent-2 folder (with 12 messages in it) and choose Properties.
- I press "Rebuild Index"; the .msf file is rebuilt and has 6,9 KBytes.
- I press "OK" to close the Folder Properties dialog; the .msf file grows
  to 7,1 KBytes.


> 
> By the way, since core bug, problem is usually in code which is used commonly
> by Tb & SM, then fix will be applied on both when problem will be FIXED.
> 

Thank you for the confirmation.

Comment 13

[WADA:World Anti-bad-Duping Agency]

Thanks for detailed description.

Problem is recreated with Tb trunk 4/13 build(Win-XP) on usual local mail folder.
 1. Say real messages number is M, and "Rebuild Index" is executed N times.
    If "re-open of mail folder" is not done after "Rebuild index", 
    message count display increases M on each "Rebuild Index".
 2. Even if increased message count is displayed, "re-open of mail folder"
    ( click other folder, then click the folder again ) corrects number.

I think similar issue to "??? after new file copy & restart".

Comment 14

[WADA:World Anti-bad-Duping Agency]

Similar phenomenon is observed when .msf recreation by "delete .msf $ restart". 
 1. Mail file has 2 active read mail and 2 deleted mail => mail count=2
 2. Shutdown Thunderbird
 3. Edit mail folder file and change deleted mail's status, and delete .msf 
    (Last 4 chars of X-Mozilla-Status: header, 0009 => 0001)
 4. Restart Thunderbird => mail count=2 is displayed (from data in panacea.dat).
 5. Click the mail folder => mail count is corrected to 4.

Comment 16

[Caio Tiago Oliveira (asrail)]

I've started to notice duplicate messages on SeaMonkey after April 09 without needing to follow these steps...


unread mail count is duplicated, but it's after the first time a new mail arrive and it increases each time the program checks automatically for new mail (it won't happen if I click the get messages button).
I do believe it's due to moving from nsFileSpec, so bug 380531 may not be a duplicate of this bug.

And:
"
###!!! ASSERTION: can't be your own parent: 'PR_FALSE', file /home/asrail/mozbuilds/mozilla/mailnews/db/msgdb/src/nsMsgHdr.cpp, line 716
"

Comment 17

[David :Bienvenu]

Attachment: proposed fix

we need to clear the counts before calling GetTransferInfo because setting properties on a transferInfo object doesn't work

Comment 18

[David :Bienvenu]

Comment on attachment 266799 [details] [diff] [review]
proposed fix

I'll let this bake on the trunk for a bit, but it should be a safe-enough fix for a confusing problem for users...

Comment 19

[David :Bienvenu]

fixed on trunk.

Comment 20

[Worcester12345]

(In reply to comment #18)
> (From update of attachment 266799 [details] [diff] [review])
> I'll let this bake on the trunk for a bit, but it should be a safe-enough fix
> for a confusing problem for users...
> 

(In reply to comment #19)
> fixed on trunk.
> 

Should this be left open until it gets fixed on branch? Is there a key word or Whiteboard entry to flag for this? 

Comment 21

[Magnus Melin [:mkmelin]]

No, the main status is for trunk. If/when it  get fixed on branch the fixed1.8.x keyword is added.

Comment 22

[WADA:World Anti-bad-Duping Agency]

"Property/Rebuild index" case is verified with 2007/6/03 trunk build(MS Win-XP).
Changing to VERIFIED.

FYI.
Count display issues when rebuild index during restart are still exist in next intentional test cases.
 1. Change "X-Mozilla-Status: 0001" of a mail to "X-Mozilla-Status: 0009"
    => Count is not reduced by 1 after restart. Click corrects mail count.
 2. Copy a mail folder file to file of completely new name (not copy ".msf")
    => Count is displayed as "???". Click corrects mail count.
No problem when usual manual ".msf" recovery(delete ".msf" and restart), because mail count never becomes different from one before restart, except when incorrect count case due to some malfunctions before restart.  

Comment 23

[Daniel Veditz [:dveditz]]

Comment on attachment 266799 [details] [diff] [review]
proposed fix

approved for 1.8.1.5, a=dveditz for release-drivers

Comment 24

[David :Bienvenu]

fixed for 1.8.1.5

Comment 25

[[:Aureliano Buendía]]

If I rebuild the index of a folder, in "Java Error Console", it is displayed this error:
Errore: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMsgFolder.updateFolder]
File sorgente: chrome://messenger/content/widgetglue.js
Riga: 249

Ciao
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.5pre) Gecko/20070628 Thunderbird/2.0.0.5pre ID:2007062808

Comment 26

[Magnus Melin [:mkmelin]]

ulysse: I filed bug 387585 for that.

Comment 27

[Carsten Book [:Tomcat]]

verified fixed 1.8.1.5 Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.5pre) Gecko/20070713 Thunderbird/2.0.0.5pre Mnenhy/0.7.5.666 ID:2007071303  - Rebuilding index is working as expected. 

For the js Error Message is Bug 355194 on file.