The default bug view has changed. See this FAQ.

Thunderbird Daily starting from 20111225 messed-up some messages (showing their source in the preview pane due to offset shift by 2 bytes, when pop3 download of multiple msgs with quarantinging turned off and leave on server turned on)

RESOLVED FIXED in Thunderbird 12.0

Status

MailNews Core
Database
--
major
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: Aureliano, Assigned: Bienvenu)

Tracking

({regression})

Thunderbird 12.0
regression
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 3 obsolete attachments)

(Reporter)

Description

5 years ago
Nightly builds starting from 20111224 messed-up some messages dysplaing message source in preview pane. Executing rebuilding index (repair folder resolve the issue).
It seems that appairs on threaded view when the message is the first message of a conversation.
I have tried also in safe-mode
Whiteboard: Regression?
(Reporter)

Comment 1

5 years ago
regression start from build 20111224
Keywords: regression
Whiteboard: Regression?
(Reporter)

Updated

5 years ago
Summary: Daily starting from 20111224 messed-up some messages → Daily starting from 20111225 messed-up some messages
(Reporter)

Comment 2

5 years ago
(In reply to [:Aureliano Buendía] from comment #1)
> regression start from build -20111224-

My mistake: build 20111224 works fine. Regression date is *20111225* --- push-log is http://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2011-12-25&enddate=2011-12-26

Comment 3

5 years ago
You probably want to use http://hg.mozilla.org/comm-central not http://hg.mozilla.org/mozilla-central

Fallout from bug 402392?
(Reporter)

Comment 4

5 years ago
(In reply to Magnus Melin from comment #3)
> You probably want to use http://hg.mozilla.org/comm-central not
> http://hg.mozilla.org/mozilla-central

Yes magnus, i'm sorry

> Fallout from bug 402392?
probably also for me, but I'm not sure
http://hg.mozilla.org/comm-central/pushloghtml?startdate=2011-12-24&enddate=2011-12-26

Comment 5

5 years ago
Seeing this too on linux. Don't know if it is related, but got this in error console : 

Timestamp: 28/12/2011 16:45:30
Warning: Use of getAttributeNodeNS() is deprecated. Use getAttributeNS() instead.
Source File: chrome://messenger/content/messenger.xul
Line: 0
(Reporter)

Comment 6

5 years ago
platforms\OS => All per comment #5
OS: Windows 7 → All
Hardware: x86 → All
tracking-thunderbird12: --- → ?
(Assignee)

Comment 7

5 years ago
Aureliano, are you using POP3 or IMAP? Are these message in your inbox, or messages moved to other folders?
(Reporter)

Comment 8

5 years ago
(In reply to David :Bienvenu from comment #7)
> Aureliano, are you using POP3 or IMAP? 
POP3

>Are these message in your inbox, or messages moved to other folders?
Both cases: my mozilla TB messages that are affected by this behavior are moved in a subfolder by a specific filter; but the problem occurs also for messages that stay in Inbox folder and don't are moved by filters
(Assignee)

Comment 9

5 years ago
Is today's build any better? I did fix an issue having to do with the msgOffset for moved messages (though I think that didn't affect messages moved by incoming mail filters)
Assignee: nobody → dbienvenu
(Reporter)

Comment 10

5 years ago
I have updated now from 20111224 to today build. I have enter a comment in bug #713814 and the received message is fine... based on this first observation it seems that the regression is gone
(Reporter)

Comment 11

5 years ago
(In reply to [:Aureliano Buendía] from comment #10)
> I have updated now from 20111224 to today build. I have enter a comment in
> bug #713814 and the received message is fine... based on this first
> observation it seems that the regression is gone

I'm sorry... this message in my folder look again bad. I revert to 20111224 build
(Reporter)

Comment 12

5 years ago
(In reply to [:Aureliano Buendía] from comment #11)
> I'm sorry... this message in my folder look again bad. I revert to 20111224
> build
Different case of comment #10, in comment #11 I have received 2 messages at the same time and one of this look messed-up: could be a concurrent receiving?
(Assignee)

Comment 13

5 years ago
Are you using the global inbox, with multiple pop3 servers storing messages in the same inbox?
(Reporter)

Comment 14

5 years ago
(In reply to David :Bienvenu from comment #13)
> Are you using the global inbox, with multiple pop3 servers storing messages
> in the same inbox?

Yes, it is
(Assignee)

Comment 15

5 years ago
When this happened, did you receive two messages from two different servers at the same time? Or two messages from the same server? And was it from an automatic check for new mail, or did you click get new mail? The latter is supposed to serialize requests; the former relies on folder locking to prevent contention.
(Reporter)

Comment 16

5 years ago
(In reply to David :Bienvenu from comment #15)
> When this happened, did you receive two messages from two different servers
> at the same time? 
Same server

> And was it from an automatic check for new mail, or did you click 
> get new mail? 
click get new mail
(Assignee)

Comment 17

5 years ago
(In reply to [:Aureliano Buendía] from comment #16)
> (In reply to David :Bienvenu from comment #15)
> > When this happened, did you receive two messages from two different servers
> > at the same time? 
> Same server
> 
> > And was it from an automatic check for new mail, or did you click 
> > get new mail? 
> click get new mail

That should all be fine, in that it works for me here - any chance you were reading one new message while the other one was being downloaded? I take it that some of your mail is filtered to other folders, and some stays in the global inbox. I'm trying to reproduce the problem here but not having any luck.

Comment 18

5 years ago
I just saw this on both the 27th nightly and again after an update to:
Mozilla/5.0 (Windows NT 5.1; rv:12.0a1) Gecko/20111229 Thunderbird/12.0a1
pop3 manual check for new mail (leave messages on server)
Only one server checked for mail.
No global inbox.

Just to be clear..what I see in message pane or standalone message window is:
Nothing in the message reader headers are, and everything displayed (headers and text) in the content area.
(Assignee)

Comment 19

5 years ago
Created attachment 584846 [details] [diff] [review]
fix the db's of pop3 filter moves

I don't know if this is related to any of the issues reported in this bug, but it's the one thing I can reproduce - we weren't marking the db valid after applying a move filter action for pop3 filters. In theory, an invalid db shouldn't cause any corruption, but it does prevent people from knowing that a message has been filtered into a folder.
Attachment #584846 - Flags: review?(mbanner)
(Assignee)

Comment 20

5 years ago
Created attachment 584853 [details] [diff] [review]
w/ whitespace changes.

prev patch was a -w patch to make the change clear, but I will clean up the whitespace indenting before landing.

Comment 21

5 years ago
I should have added that no filter moves were involved on the system where I observed the problem. Fortunately, I can stay on a current nightly on that machine to evaluate.
In every case "rebuild folder" has fixed things.
Comment on attachment 584846 [details] [diff] [review]
fix the db's of pop3 filter moves

Looks like the pluggable store changes mean that destMailDB now gets modified a few lines above this point:

http://hg.mozilla.org/comm-central/annotate/c25a8efb9585/mailnews/local/src/nsParseMailbox.cpp#l2572

So I agree, this change seems the right one to make.
Attachment #584846 - Flags: review?(mbanner) → review+
(Assignee)

Comment 23

5 years ago
first patch landed - http://hg.mozilla.org/comm-central/rev/068dfc71f91d

I agree that this doesn't explain the inbox corruptions - I'm still trying to reproduce that.
(Reporter)

Comment 24

5 years ago
Created attachment 584958 [details]
bug-in-action (build 20111230)

(In reply to David :Bienvenu from comment #23)
> first patch landed - http://hg.mozilla.org/comm-central/rev/068dfc71f91d

20111230 still don't work for me (see attached bug-in-action screenshot)
(Reporter)

Comment 25

5 years ago
(In reply to [:Aureliano Buendía] from comment #24)
> 20111230 still don't work for me (see attached bug-in-action screenshot)

for messages moved by filters
(Assignee)

Updated

5 years ago
Duplicate of this bug: 714182

Comment 27

5 years ago
Bumped severity and changed component.
I personally am not willing to test Daily 12.0 with database corruption involved.
(Even though "repair Folder" seems to fix it.)
Severity: normal → major
Component: Message Reader UI → Database
Product: Thunderbird → MailNews Core
QA Contact: message-reader → database
(Assignee)

Comment 28

5 years ago
If you turn on anti-virus quarantining, I don't think you'll see any corruptions, fwiw.

Comment 29

5 years ago
(In reply to David :Bienvenu from comment #28)
> If you turn on anti-virus quarantining, I don't think you'll see any
> corruptions, fwiw.

Yep,
Sent myself 3 consecutive messages and all viewed fine.
Back on the daily here. This has got to be one tough nut to crack.
(Assignee)

Comment 30

5 years ago
OK, I think this is because we're overzealous about not flushing the mailbox file when getting multiple messages. So when we're re-using a stream, we need to use the stream pos of the end of the stream instead of the mailbox file size as the msg offset of the new message. I think I have a fix that does this; testing it now.
(Assignee)

Updated

5 years ago
Blocks: 402392
(Assignee)

Comment 31

5 years ago
Created attachment 585415 [details] [diff] [review]
proposed fix

This fixes it for me. I'll try to figure out a unit test for this.

Comment 32

5 years ago
I can also see this, when I downloaded messages from only one server.
Status: NEW → ASSIGNED
Summary: Daily starting from 20111225 messed-up some messages → Thunderbird Daily starting from 20111225 messed-up some messages (showing their source in the preview pane)
(Assignee)

Comment 33

5 years ago
(In reply to :aceman from comment #32)
> I can also see this, when I downloaded messages from only one server.

Yes, this bug has nothing to do with multiple servers - it specifically has to do with downloading multiple mails in the same session to the same inbox.

Comment 34

5 years ago
OK, then I can try to run with the patch.

Comment 35

5 years ago
Today's hg checkout plus this patch applied makes new downloaded messages display fine (more of them fetched in single check).
(Assignee)

Comment 36

5 years ago
Created attachment 585571 [details] [diff] [review]
fix with unit test

This adds a unit test. Since reproducing this bug requires that quarantining be turned off, and pop3 leave on server be set to true, the test has to set the prefs accordingly. But since the pop3 fake server didn't implement UIDL, which is needed for leave on server, I had to fix that as well.
Attachment #585415 - Attachment is obsolete: true
Attachment #585571 - Flags: review?(mbanner)
(Assignee)

Comment 37

5 years ago
Created attachment 585575 [details] [diff] [review]
remove debugging code

oops, prev patch had some debugging code to verify the cause of the issue.
Attachment #585571 - Attachment is obsolete: true
Attachment #585571 - Flags: review?(mbanner)
Attachment #585575 - Flags: review?(mbanner)
Comment on attachment 585575 [details] [diff] [review]
remove debugging code

Review of attachment 585575 [details] [diff] [review]:
-----------------------------------------------------------------

I think the code looks fine. However adding the UIDL to the capabilities appears to have broken some of the unit tests that are now sending an extra XTND XLST Message-Id message to the fake server and that's showing up in the results. It probably just needs those tests adapting.

::: mailnews/local/test/unit/test_pop3Download.js
@@ +33,5 @@
> +{
> +  // get message headers for the inbox folder
> +  let enumerator = gLocalInboxFolder.msgDatabase.EnumerateMessages();
> +  var msgCount = 0;
> +  let hdr;

nit: unnecessary definition - duplicated in the while loop below.
Attachment #585575 - Flags: review?(mbanner) → review-
(Assignee)

Comment 39

5 years ago
Created attachment 585760 [details] [diff] [review]
fix tests
Attachment #585575 - Attachment is obsolete: true
Attachment #585760 - Flags: review?(mbanner)
Attachment #585760 - Flags: review?(mbanner) → review+
(Assignee)

Comment 40

5 years ago
http://hg.mozilla.org/comm-central/rev/34489081da88
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 12.0
(Assignee)

Comment 41

5 years ago
oops, forgot unit test http://hg.mozilla.org/comm-central/rev/bf6957ab2201
Flags: in-testsuite+

Updated

5 years ago
Duplicate of this bug: 715118
Summary: Thunderbird Daily starting from 20111225 messed-up some messages (showing their source in the preview pane) → Thunderbird Daily starting from 20111225 messed-up some messages (showing their source in the preview pane due to offset shift by 2 bytes, when pop3 download of multiple msgs with quarantinging turned off and leave on server turned on)
Hi,

Sorry for not reading the 42 (42 !) comments ;-) but is this going to fix the issue with the same symptoms on IMAP, or is it just for POP?
tracking-thunderbird12: ? → ---
You need to log in before you can comment on or make changes to this bug.