Closed Bug 1734847 Opened 3 years ago Closed 3 years ago

TB (beta regression) 94.0b1 occasionally mis-indexing mail "subject" with previous inbox msg (NOT VERSION 91.x)

Categories

(MailNews Core :: Database, defect, P1)

Product:
MailNews Core
Component:
Database
Version:
Thunderbird 94
Type:
defect

Tracking

(thunderbird_esr91 unaffected, thunderbird95+ fixed)

Status:
RESOLVED FIXED
Milestone:
96 Branch
Tracking Flags:
Tracking Status
thunderbird_esr91 --- unaffected
thunderbird95 + fixed

People

(Reporter: dan, Assigned: benc)

References

(Regression)

Details

(Keywords: dataloss, regression)

Attachments

(4 files)

multiple-emails-when-opening-1st-one.png
43.40 KB, image/png
Details
FOOK.tar
20.00 KB, application/x-tar
Details
Bug 1734847 - Fix message size calculations when copying multiple messages to local folder. r=mkmelin
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-beta+
Details | Review
TBerrors11152021whiledoingrepair.JPG
404.26 KB, image/jpeg
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36

Steps to reproduce:

Restarted TB (beta 94.0B1) after TB automatically updated, checked for new mail.

Actual results:

When retreving mail TB 94.0b1 now occasionally displays "subject" of email correctly, but when opening mail it shows occasionally shows content of next previous email in "inbox" (duplicates the the content of the previous email) and apparently discards (or at least does not display) the actual contents of the newer email. Restatrting Thunderbird and then restarting PC had no effect. Help!

Expected results:

Correct mail subject and content should be displayed

Now attempting this: after exiting TB, deleted global-messages-db.sqlite file, then restarted TB and it is rebuilding file.

Rebuild of global-messages-db.sqlite file completed... not sure problem was resolved. Exited Beta Channel, installed release channel v91.2.0 and allowed the "downgrade" of my profile. All seems to be OK. Living life in the slow lane now ;)

There have been rare cases of subject/body mismatch for IMAP messages. Strange that this only happens in v94 now but not v91.x.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: regression, regressionwindow-wanted

Dan, Please retest with 94.0b4 which is just released.

Flags: needinfo?(dan)

Would be happy to, but unable now... moved both my PCs back to the release channel and the current release version. Thanks for your help with this!

Flags: needinfo?(dan)
Status: NEW → RESOLVED
Closed: 3 years ago
Component: Untriaged → Database
Product: Thunderbird → MailNews Core
Resolution: --- → INCOMPLETE

Let's confirm this. I don't have steps to reproduce, but have seen it (even on 95 trunk) and there are some other reports as well.

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → NEW
Severity: -- → S1
tracking-firefox94: --- → ?
tracking-firefox94: ? → ---
tracking-firefox95: --- → ?
tracking-firefox95: ? → ---
Priority: -- → P1

This would be caused by recent changes in the backend wrt. to file streams, seeking, offset, etc. This report could also be related:
https://thunderbird.topicbox.com/groups/daily/T51927946941c9c68-Mc061a0a79f75977acd6f7dd2
Overall it looks like the mbox file is accessed at the wrong offset.

Time line - 94.0b1 (per comment 1) shipped 10/6 at rate 100 and was built on 10/4. This bug report (which is the earliest I can find) was created on 10/8 - so this starts at least at the very beginning of beta 94. But perhaps earlier?

Questions:
No one saw this when using 93?
Did anyone using daily 94 see this?
Has anyone seen this for a pop account?

(In reply to Wayne Mery (:wsmwk) from comment #12)

Has anyone seen this for a pop account?

Yes, all my mail accounts are POP3 and that's where I experienced this when I originally reported on 10/8. Sorry I can't help now... moved my two machines back to the release channel.

Thanks for all of everyone's efforts with this!

Dan

I think I saw it on daily just the other day (IMAP).

(In reply to Magnus Melin [:mkmelin] from comment #15)

I think I saw it on daily just the other day (IMAP).

Oops, what I mean is daily prior to 10/6, when daily was still 94

(In reply to Wayne Mery (:wsmwk) from comment #12)

Time line - 94.0b1 (per comment 1) shipped 10/6 at rate 100 and was built on 10/4. This bug report (which is the earliest I can find) was created on 10/8 - so this starts at least at the very beginning of beta 94. But perhaps earlier?

Questions:
No one saw this when using 93?
Did anyone using daily 94 see this?
Has anyone seen this for a pop account?

My observation: it started happening pretty recently. I've noticed it before but didn't report it because it only happens occasionally. I'm on 94.0b4 and use it daily, and yes it did happen on a POP account.

(In reply to Wayne Mery (:wsmwk) from comment #16)

(In reply to Magnus Melin [:mkmelin] from comment #15)

I think I saw it on daily just the other day (IMAP).

Oops, what I mean is daily prior to 10/6, when daily was still 94

I'm on the beta update channel and have been for quite some time.

looks like a good list. should be easy to narrow it down with one or two try builds backing out one at a time?

(In reply to Wayne Mery (:wsmwk) from comment #12)

Questions:
No one saw this when using 93?

I was using beta 93.xx prior to the "automatic" update to 94.0b1, and did not experience this issue prior to the update.

Probably not so easy until we have steps to reproduce. After that it would be way easier.

Keywords: dataloss

As I told Magnus in bug 1738105, if there's some kind of logging or whatnot that I can turn on to try and catch this thing in the act, let me know and I'll give it a try. I didn't see it start happening until 94.0b5 and I'm on IMAP.

My guess would be there is no logging that could catch it.

(In reply to Magnus Melin [:mkmelin] from comment #24)

My guess would be there is no logging that could catch it.

Bummer. And nothing in that Error Console output that I added in bug 1738105 is any hint?

Almost forgot and maybe it's related. I can't seem to run a repair operation on a folder anymore. I say seem because when I do run it on my Inbox, I don't see the usual Downloading xxxxxx messages in the Status Bar. I do see the progress bar animation kind of doing something but it stops shortly after I run the repair.

So it looks like I've been pushed a 94.b05. I received an email at 9:15 and can see the email body when I select it. For an email received at 9:27 all all thereafter, I can select it but cannot see the email body at all. This is happening for 2 POP accounts. I can no longer read incoming emails. :-(

Oops - abo(In reply to Josh from comment #27)

So it looks like I've been pushed a 94.b05. I received an email at 9:15 and can see the email body when I select it. For an email received at 9:27 all all thereafter, I can select it but cannot see the email body at all. This is happening for 2 POP accounts. I can no longer read incoming emails. :-(

Oops - times above are in EDT.

Don't know if this is what is being reported here but definitely similar. I was sent via email a saved eml as seen by user breezymozilla with 94.0b (see bug 1734843 comment 105) where he attempts to open an email and sees below it many other emails, but just the raw mbox content. The attached png shows the end of the 1st valid email followed by the first line of the raw mbox content. The whole message with raw mbox content came to 28Mbytes ! This is using IMAP.
I couldn't duplicate with 94.0b4 so asked him to just report a new bug but I suspect this is caused by the same thing.

(In reply to Magnus Melin [:mkmelin] from comment #19)

My main suspects in https://hg.mozilla.org/comm-central/pushloghtml?startdate=2021-09-10&enddate=2021-10-04 would be

For debugging,

(1) the first patch,
I would record the current position of the file in near where seek to the end was removed and then
re-insert the seek to end and then read the file position again compare the file position recorded earlier.
If they are the same, the removal of seek was OK, but if not we have a problem.
(I could find a similar file position problem when I tried to introduce the buffered write [still to be submitted/merged in the latest re-incarnation.]
When a file stream was converted to buffered stream, the stream was AUTOMATICALLY rewound to the initial 0 position which caused a serious bug and that was why it had been removed many years ago, several years before I tried it.
I could only find this automatic rewinding when I trace the file position very carefully in a similar approach. :-(

(2) the second patch.
I think only careful reading of the code, and dumping of various intermediate file, etc. can reveal any surprises or misunderstandings of the operation. (Well, in my attempt to introduce buffered write, I also touched this part and I was a bit surprised that
a buffered input stream could not be modified into buffered output stream or something like that. The conversion of various file types were not quite symmetrical and uniform. If BenC could eliminate such extra code to deal with the limit of low-level IO function, so much the better.)

I was assuming the above two are the only potential culprits, though.

Using the test case from bug 1736917 I was able to pin down the regression window to 2021-09-01 -> 2021-09-03 dailies.
https://hg.mozilla.org/comm-central/pushloghtml?startdate=2021-09-01+12%3A00&enddate=2021-09-03+14%3A01

By local backouts I found the offender is https://hg.mozilla.org/comm-central/rev/2c8857af0eb3112dac2250b7f7bdf777e8911771 (bug 1728924)

I'm going to back that out to fix this bug.
Something with m_envelope_pos or m_headerstartpos might be wrong but it will require some debugging to find out what and how to fix it.

The .msf files for affected folders will have been damaged. Repair folder is required and fixes it.

Assignee: nobody → mkmelin+mozilla
Status: NEW → ASSIGNED
status-thunderbird95: --- → affected
status-thunderbird_esr91: --- → unaffected
tracking-thunderbird95: --- → +
Keywords: regressionwindow-wanted
Target Milestone: --- → 96 Branch
Regressed by: 1728924

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/70c0061f9f84
Backed out changeset 2c8857af0eb3 (bug 1728924) for causing .msf corruption. rs=backout

Status: ASSIGNED → RESOLVED
Closed: 3 years ago → 3 years ago
Resolution: --- → FIXED

I think the backout is causing failures in mailnews/search/test/unit/test_quarantineFilterMove.js crashing at seekableStream->Tell(&filePos);

Probably safest to back out the backout for now
Backout: https://hg.mozilla.org/comm-central/rev/87f80cf1335e987a0ee19ff8671afb6617a807eb

Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Over to you Ben

Assignee: mkmelin+mozilla → benc

Definitely looks like my changeset 2c8857af0eb3 which screwed it up.

I can replicate it now too:

  1. have an IMAP account with a folder containing at least 3 messages
  2. copy those messages to a local folder
  3. observe one or more messages screwed up

It doesn't seem to happen when copying messages from other local folders.
"repair folder" fixes it.

No fix yet, but wanted to note down one anomaly I've spotted:

The .messageSize attribute on the new messages seem to be screwed up after the copy. It uses the messagesize from the source message. But during the copy, an extra 97 bytes is added when an "X-Mozilla-Keys" header (plus blank space to later write keywords) is inserted. Might also be an issue for X-Mozilla-Status & X-Mozilla-Status2, but I'm not sure (my test messages already have those set in the IMAP mbox).

I'd have thought this would cause all kinds of chaos and mayhem... but seems like it was already happening. The size of the copied messages is wrong even if the patch is backed out.

Attached file FOOK.tar —

Just out of interest, here's the more constrained, human-readable test case I'm using now.
Two folders containing the same three messages. One good, one borked.

The mbox files are identical (other than differing "From "line timestamps).

(In reply to Ben Campbell from comment #36)

The .messageSize attribute on the new messages seem to be screwed up after the copy. It uses the messagesize from the source message. But during the copy, an extra 97 bytes is added when an "X-Mozilla-Keys" header (plus blank space to later write keywords) is inserted.

My mistake - I was looking at the .offlineMessageSize attribute (not entirely sure what that is used for... and it still seems very wrong).

In any case, it does seem like the borked messages are getting the wrong .messageSize. Haven't figured out the exact cause yet (there are a lot of moving parts involved), but I've got some ideas for potential fixes.

I'm not sure if this is helpful at all but I have only seen this problem with combined and screwed up messages when I also have Message Filters enabled. I have a bunch of Message Filters set up for the sole purpose of organizing messages into sub-folders. I also have offline folders enabled for all the folders I've had trouble with. I haven't seen any new corruption since I disabled all message filtering. From what I've seen, it seems like the problem is related to messages being moved/copied, maybe by multiple threads (?), maybe looking at the wrong message size or the corrupt message size in the process. I haven't specifically tried to reproduce or really even analyze what's going wrong - just my observations and gut feel based on what I've seen.

Same here, I have quite a few message filters filtering into Local Folders. And I'm now moved back to TB 91.3.0 and the problem isn't occurring, so it was introduced somewhere after that.

Brian and Josh: Yep, the issues you describe there do sound like the breakage I caused.

I think I've figured it out. It's not the filters which are the issue, but the copying of multiple messages from non-local folders to local ones.

For copying from non-local to local, the copy routines pass the message through a message parser (nsParseMailMessageState).
The changes I made in Bug 1728924 assumed that nsParseMailMessageState is only used for a single message at a time. This is usually the case... but here the local folder copy code just streams all the messages through the parser. And it gets confused where it's up to and sets the wrong sizes on the messages.
(there is a derived class, nsMsgMailboxParser, which does handle parsing multiple messages, but that's not being used here, for whatever reason).

I'm not sure why it uses the single parser for multiple messages. Probably it should be creating a new parser for each message. But the copy code is so brittle (Bug 1731177) that I think it's safer to just hack it for now and fiddle the required fields in the parser when each new message is started, so the size calculation is correct.

I'm just testing a patch now to do just that.

The real solution is that the parser shouldn't care about the size of the incoming message anyway - the size should be set by the code handling the writing (the mail store). Much more accurate, and better able to handle "From " escaping and all those icky details.
I'm working on it (Bug 1719121, Bug 1733849 and friends) but it's a major job. Assumptions about mbox are all over the place :-(

(In reply to Ben Campbell from comment #41)

Brian and Josh: Yep, the issues you describe there do sound like the breakage I caused.

I think I've figured it out. It's not the filters which are the issue, but the copying of multiple messages from non-local folders to local ones.

For copying from non-local to local, the copy routines pass the message through a message parser (nsParseMailMessageState).
The changes I made in Bug 1728924 assumed that nsParseMailMessageState is only used for a single message at a time. This is usually the case... but here the local folder copy code just streams all the messages through the parser. And it gets confused where it's up to and sets the wrong sizes on the messages.
(there is a derived class, nsMsgMailboxParser, which does handle parsing multiple messages, but that's not being used here, for whatever reason).

I'm not sure why it uses the single parser for multiple messages. Probably it should be creating a new parser for each message. But the copy code is so brittle (Bug 1731177) that I think it's safer to just hack it for now and fiddle the required fields in the parser when each new message is started, so the size calculation is correct.

I'm just testing a patch now to do just that.

The real solution is that the parser shouldn't care about the size of the incoming message anyway - the size should be set by the code handling the writing (the mail store). Much more accurate, and better able to handle "From " escaping and all those icky details.
I'm working on it (Bug 1719121, Bug 1733849 and friends) but it's a major job. Assumptions about mbox are all over the place :-(

This sounds interesting.
Many moons ago, I asked in one of the bugzilla, how TB reports back an error during the copying of multiple messages.
Imagine that you want to copy 100 messages, but the transfer of 50th message failed due to filled up file system, temporary network outage of the remote file system, etc. Irrespective of what one should do afterward, the error ought to be returned to the caller and eventually to the
UI.
Back when I asked that someone answered he will check. But I heard nothing afterward.

You may want to make sure the error is returned properly to the caller in your local hack now. (Back when I asked the question above, it seems to me that TB kickstarted the copying process and waited for the completion via listener or whatever it is called, but there was no clear path for error information propagation. Some functions involved in the copying process were declared void and obviously there was no way to return an error value from such a function. I looked at the potential pathways including output parameter, but the listener only seemed to care for the termination (error or success).
I thought that WAS BAD.

(In reply to ISHIKAWA, Chiaki from comment #43)

This sounds interesting.
Many moons ago, I asked in one of the bugzilla, how TB reports back an error during the copying of multiple messages.
Imagine that you want to copy 100 messages, but the transfer of 50th message failed due to filled up file system, temporary network outage of the remote file system, etc. Irrespective of what one should do afterward, the error ought to be returned to the caller and eventually to the
UI.
Back when I asked that someone answered he will check. But I heard nothing afterward.

You may want to make sure the error is returned properly to the caller in your local hack now. (Back when I asked the question above, it seems to me that TB kickstarted the copying process and waited for the completion via listener or whatever it is called, but there was no clear path for error information propagation. Some functions involved in the copying process were declared void and obviously there was no way to return an error value from such a function. I looked at the potential pathways including output parameter, but the listener only seemed to care for the termination (error or success).
I thought that WAS BAD.

This sounds a lot like the "copy lots of messages to another server bug" which I tried to summarize here: bug 538375 comment 194. This is something that has been a problem for many years as you can see by the long list of comments.

See Also: → 538375

I don't know if it's related to the attempts to fix this corruption issue but wanted to note bug 1740486 that I opened today.

(In reply to ISHIKAWA, Chiaki from comment #43)

You may want to make sure the error is returned properly to the caller in your local hack now. (Back when I asked the question above, it seems to me that TB kickstarted the copying process and waited for the completion via listener or whatever it is called, but there was no clear path for error information propagation. Some functions involved in the copying process were declared void and obviously there was no way to return an error value from such a function. I looked at the potential pathways including output parameter, but the listener only seemed to care for the termination (error or success).
I thought that WAS BAD.

There are a few listeners involved in message copying, but they seem to be used rather inconsistently. Some paths don't seem to invoke all the callbacks you'd expect, and there are lots of corner cases where stuff gets initialised in multiple different places, depending on the exact details of the messages being copied...
Message copying has just evolved over time into a brittle lump of concerns, made worse by the way message streaming, mbox handling and nsIMsDBHdr population is all tangled up together (not to mention move/copy/move-to-trash complications and undo support).
It needs some real effort to refactor and simplify all these moving parts, and that's more or less what my focus is at the moment, starting with mbox (with the side effect outcome of getting maildir support solid!).

But it's really hard not to break stuff along the way - in fact this whole bug was a result of a relatively modest refactoring attempt by me! So it's obvious I need to be waaay more careful as I continue.

All of which is to say: yes, I totally agree with you that the error reporting needs to be much more robust! It will be, but for this fix, I'm content if it fixes the problem and doesn't break anything else :-)

OK, so the patch seems to fix the problem for me, and the try build seems OK... so I've flagged it for landing.
But it'd be nice to have a few more people who experienced the issue try it out and confirm it's an improvement (and doesn't obviously break anything else!)...

Keywords: checkin-needed-tb

(In reply to Ben Campbell from comment #41)

Brian and Josh: Yep, the issues you describe there do sound like the breakage I caused.

I think I've figured it out. It's not the filters which are the issue, but the copying of multiple messages from non-local folders to local ones.

For copying from non-local to local, the copy routines pass the message through a message parser (nsParseMailMessageState).
The changes I made in Bug 1728924 assumed that nsParseMailMessageState is only used for a single message at a time. This is usually the case... but here the local folder copy code just streams all the messages through the parser. And it gets confused where it's up to and sets the wrong sizes on the messages.
(there is a derived class, nsMsgMailboxParser, which does handle parsing multiple messages, but that's not being used here, for whatever reason).

I'm not sure why it uses the single parser for multiple messages. Probably it should be creating a new parser for each message. But the copy code is so brittle (Bug 1731177) that I think it's safer to just hack it for now and fiddle the required fields in the parser when each new message is started, so the size calculation is correct.

I'm just testing a patch now to do just that.

The real solution is that the parser shouldn't care about the size of the incoming message anyway - the size should be set by the code handling the writing (the mail store). Much more accurate, and better able to handle "From " escaping and all those icky details.
I'm working on it (Bug 1719121, Bug 1733849 and friends) but it's a major job. Assumptions about mbox are all over the place :-(

Once done, is this a done deal? In other words, if the bug gets fixed, will messages be back to normal?

If not, this needs to be a TOP PRIORITY over all others, due to its DATALOSS condition.

(In reply to Ben Campbell from comment #46)

(In reply to ISHIKAWA, Chiaki from comment #43)

　... omission ...

There are a few listeners involved in message copying, but they seem to be used rather inconsistently. Some paths don't seem to invoke all the callbacks you'd expect, and there are lots of corner cases where stuff gets initialised in multiple different places, depending on the exact details of the messages being copied...
Message copying has just evolved over time into a brittle lump of concerns, made worse by the way message streaming, mbox handling and nsIMsDBHdr population is all tangled up together (not to mention move/copy/move-to-trash complications and undo support).
It needs some real effort to refactor and simplify all these moving parts, and that's more or less what my focus is at the moment, starting with mbox (with the side effect outcome of getting maildir support solid!).

But it's really hard not to break stuff along the way - in fact this whole bug was a result of a relatively modest refactoring attempt by me! So it's obvious I need to be waaay more careful as I continue.

All of which is to say: yes, I totally agree with you that the error reporting needs to be much more robust! It will be, but for this fix, I'm content if it fixes the problem and doesn't break anything else :-)

I will keep my fingers crossed. :-)

BTW, I am trying to update my local patch set to enable buffered-write in TB (bug Bug 1242030)
Copying many messages from a folder to the other takes too long IMHO.
Also, along the way, I noticed so many ignored cases of error values from low-level I/O routines, which I needed to fix. pop3 error handling scheme may not work due to the programming errors I noticed there.

Right now, basically, I am trying to accommodate the download to temporary file patch and others which I could not accommodate due to other patches until several weeks ago.

I have put in many sanity checks in my local patch. So if I notice anything strange from my local log, I will report it to bugzilla. But I am afraid it would take me a few more weeks to that stage. I am trying classify bugs in to serious and not-so-serious categories right now and comparing other people's jobs on try-comm-central to figure out which ones are caused by my local patches for now.
Logs from mochitest and xpcshell test of DEBUG version of TB contain so many noises, I am trying to cut down the noise, too.

TIA

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/91332386d3dd
Fix message size calculations when copying multiple messages to local folder. r=mkmelin

Status: REOPENED → RESOLVED
Closed: 3 years ago → 3 years ago
Keywords: checkin-needed-tb
Resolution: --- → FIXED

(In reply to Worcester12345 from comment #48)

Once done, is this a done deal? In other words, if the bug gets fixed, will messages be back to normal?

For affected folders you'll need to do "Repair Folder". The actual messages are not damaged, but the .msf file of the folder has wrong data.

I raised defect 1736917 which has been marked as a duplicate of this one, for Thunderbird 95.0 appearing to corrupt mail messages when moved to a local folder by filters, but only for mails from two specific mail lists, all the rest of the 43 filters I have to move threads to local folders don't cause the issue.
About a week or so ago V95.0 of Thunderbird on my Linux VM image was upgraded to V96.0a1 and now the threads from the Fedora mail list that are sitting in my Imap mailbox are exhibiting the issue before the moves happen. So it looks like V96 has further compounded the issue.

regards,
Steve

For affected folders, you'll need to use the Repair Folder to get them working properly again. Did you do that?

That seems unrelated to this bug.

We should get this uplifted to beta.

Comment on attachment 9250105 [details]
Bug 1734847 - Fix message size calculations when copying multiple messages to local folder. r=mkmelin

[Approval Request Comment]
Regression caused by (bug #): bug 1728924
User impact if declined: .msf corruptions
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): continued .msf corruptions on beta

Attachment #9250105 - Flags: approval-comm-beta?

(In reply to Magnus Melin [:mkmelin] from comment #58)

Comment on attachment 9250105 [details]
...
Regression caused by (bug #): bug 1728924
User impact if declined: .msf corruptions
...
Risk to taking this patch (and alternatives if risky): continued .msf corruptions on beta

Is this considered "data loss"?

(In reply to Worcester12345 from comment #59)

Is this considered "data loss"?

Sorry, seeing this in key words just now.

Comment on attachment 9250105 [details]
Bug 1734847 - Fix message size calculations when copying multiple messages to local folder. r=mkmelin

[Triage Comment]
Approved for beta

Attachment #9250105 - Flags: approval-comm-beta? → approval-comm-beta+

So I just installed the testing build of 95.0b4 and so far so good. At least bug 1740486 seems gone now. I'm presently doing a repair on my Inboxes and I'll report back if recent mangled messages are repaired.

So I was testing 95.0b4 on my home PC and doing Folder Repairs on affected Inboxes and something seems to be worse now. Folder Repair seems to have run all day and night long but never seems to have actually stopped. Activity Mon isn't showing any more activity but Task Manager still shows TB as doing....something. And now, AFAICT, we have bug 1742590 to contend with as a byproduct.

I think we ultimately have to Mozgression this one because it appears that whatever fix included in b4 didn't quite do it and caused fallout.

Sorry but I have experienced the same issue also today in TB95.0b4 in a new fresh profile.

See Also: → 1742975

(In reply to gene smith from comment #44)

(In reply to ISHIKAWA, Chiaki from comment #43)

This sounds interesting.
Many moons ago, I asked in one of the bugzilla, how TB reports back an error during the copying of multiple messages.
Imagine that you want to copy 100 messages, but the transfer of 50th message failed due to filled up file system, temporary network outage of the remote file system, etc. Irrespective of what one should do afterward, the error ought to be returned to the caller and eventually to the
UI.
Back when I asked that someone answered he will check. But I heard nothing afterward.

You may want to make sure the error is returned properly to the caller in your local hack now. (Back when I asked the question above, it seems to me that TB kickstarted the copying process and waited for the completion via listener or whatever it is called, but there was no clear path for error information propagation. Some functions involved in the copying process were declared void and obviously there was no way to return an error value from such a function. I looked at the potential pathways including output parameter, but the listener only seemed to care for the termination (error or success).
I thought that WAS BAD.

This sounds a lot like the "copy lots of messages to another server bug" which I tried to summarize here: bug 538375 comment 194. This is something that has been a problem for many years as you can see by the long list of comments.

Oh, thank you for the heads-up.. This has been my nagging issue at the back of my mind.
I will put this on my radar when I work on my patch set for buffered-output performance improvement

Is anyone checking the crash reports? I assume they are being sent? Having them left and right. Also attaching corrupted .eml file. Sorry, doesn't look like there is any capability to do so in Bugzzzzz

(In reply to ISHIKAWA, Chiaki from comment #74)

Oh, thank you for the heads-up.. This has been my nagging issue at the back of my mind.
I will put this on my radar when I work on my patch set for buffered-output performance improvement

Are these open bugs right now? Could you give the bug numbers?

(In reply to Worcester12345 from comment #76)

(In reply to ISHIKAWA, Chiaki from comment #74)

Oh, thank you for the heads-up.. This has been my nagging issue at the back of my mind.
I will put this on my radar when I work on my patch set for buffered-output performance improvement
Are these open bugs right now? Could you give the bug numbers?

Are you interested in the buffered-output bugzilla?

(In reply to ISHIKAWA, Chiaki from comment #77)
...

Are you interested in the buffered-output bugzilla?

No idea what that is, so no.

Just noticing some emails that are crazy mixed up. This is on the newest nightly "beta" Thunderbird. So it apparently does not fix the issue. Or is it an issue where once the data (email) is messed up, it stays that way and won't be brought back or fixed?

Everyone affected must do Repair Folder, since the .msf file has incorrect data.

Since bugs like this happen from time to time, can a code for an asynchronous integrity check be added to TB so it would automatically run "Repair Folder" command on affected folders in the background, without any actions required from users?

(Obviously users that visit here can live with that but the corruption may affect people who are way removed from ever going to properties and manually starting the procedure.)

@Magnus Melin, I see recurring this issue in 96.0b1 (32-bit). After Repair Folder message is “composed” in another way. What should be done?

I'm not sure what you mean by that.
Possible further issue is tracked in bug 1742975, but it's yet unclear what it's about.

Ah, sorry my bad. Bad issue.

(In reply to uzivatel919 from comment #86)

Ah, sorry my bad. Bad issue.

No, it is not. I mean:

  1. I opened a message and saw it again merged with some.
  2. I run folder repait that seemed to did nothing.
  3. I repeat step 2 few times.
  4. Then message was merged another was.
  5. Restart, repair folder again, now message appears blank.

*repeated, another way

Saved .eml message has size about 192 MB.

Crashes upon message opening.

Folder repair continuously fails to do remedy.

It's possible there is some bug wrt compact, see bug 1742975 comment 13.

Can you try to delete the .msf file for the folder instead?

I deleted whole account (after previous more gentle attempts). Then I got immediately big message again. This time 125 MB. I can provide original message. Maybe it has some structural speciality?

(In reply to Magnus Melin [:mkmelin] from comment #82)

Everyone affected must do Repair Folder, since the .msf file has incorrect data.

Is there an easy way to do "repair all folders" at once?

If not, can there be?

(In reply to Magnus Melin [:mkmelin] from comment #92)

It's possible there is some bug wrt compact, see bug 1742975 comment 13.

Can you try to delete the .msf file for the folder instead?

Maybe they should come out with a "Thunderbird Repair Tool" which deletes all these files at once, and does other maintenance items. Kind of like the Mozbackup utility, but for repairs instead. Kind of a "Thunderbird Tuneup".

I'm currently on Thunderbird 98.0a1 in Linux and this issue is still occurring and has been occurring non-stop since Thunderbird 94.0a1. Repairing the inbox folder on both my imap accounts rectifies the issue but it keeps coming back all the time. The situation seems to have gotten worse recently in that when previewing emails that are potentially subject to the issue I get random graphic images (artifacts) flash on the screen, and with on particular email when I click on it, it seems to hang Thunderbird and eventually Thunderbird crashes, and this happens every time I tried to preview the mail. I don't have the email anymore as I deleted it to prevent this issue and to continue to use Thunderbird.
I also had an issue on one mail in my gmail imap inbox where instead of the mail showing what looked like binary data from the "cross linked" email it showed the html contents of the "cross linked" email.
I once did a reply on one of these "cross linked" emails back to the mail list it originated from without deleting the "binary data", and my reply was held for moderator intervention because the reply exceeded the mail size allowed by the mail list.

Hello,
since upgrading to 91.4 we have been experiencing these problem with various intensity on dozens of workstations:

  • Missing headers
  • Invalid email text shown
  • Other email shown instead
  • Half of an email shown as raw text (with headers MTA-provided headers removed, event content-type)
  • fragments of other emails shown

The only way to temporarily remedy the issue (in our case) is "repair folder" action (we're using IMAP).
Some observations:

  • Some users seem to be far more affected than others.
  • The issue impacts emails at random, it can just as well wreck an email in "Archive" from 2020, as a one from 2 hours ago.
  • Disabling/adding exceptions in AV / Firewall does not help
  • Disabling global indexing helped somewhat
  • Disabling time-based synchronization (i.e only last 90 days) helps somewhat
  • Disabling "automatic folder reorganization" and not re-organizing by hand also helps a bit

Will downgrading to 91.3.2 fix this?
We don't mind re-downloading whole mailboxes / folder trees, we just want get away from this issue.

Hard to believe developers cannot reproduce this issue.

From comment 97:

since upgrading to 91.4 we have been experiencing these problem with various intensity on dozens of workstations:

I thought this was only a problem with post-91 alphas and betas? Are you seeing on the 91 release?

From comment 98:

Hard to believe developers cannot reproduce this issue.

I haven't been directly involved with fixing this and haven't seen it myself. But can you suggest some steps to reproduce what you see?
Here are a few things that might be relevant:

Are the problem folders IMAP or POP?
If IMAP, do you save full messages locally (the default) or just on the server?
What is the mail server type, e.g., dovecot, gmail, exchange etc.?
Are you using using default mbox (default and one file per folder) or maildir (one file per message) as local storage?
How many messages are in the problem folders? Do many have really large attachments?
Any more details might be helpful.

Here's something to try if IMAP and storing messages locally. This is just to see if the problem is coming from tb trying to read the mbox or maildir file to display a message. This will causes messages in a single folder to only be stored on server.
Right-click folder and choose Properties
Go to "synchronization" tab
Uncheck "select this folder for offline use"
Click OK
Right-click same folder and again choose Properties
Then choose "repair folder". Emails will seem to disappear and then come back when repair finished. May take a while if lots of messages in folder.

Now when you open an email in that folder it will be fetched from the server instead of from the local mbox or maildir file. Other folder will still read mbox or maildir to open existing messages.
Does this make a difference?
If this helps it might be considered a temporary workaround but mostly just to see where the problem is coming from.

This bug is/was only about tb 94.
There is no general problem with 91 in this regard. If you have a problem there, open a new bug about it, with details as per previous comment.

(In reply to gene smith from comment #99)

From comment 97:

since upgrading to 91.4 we have been experiencing these problem with various intensity on dozens of workstations:

I thought this was only a problem with post-91 alphas and betas? Are you seeing on the 91 release?
Yes, definitely yes.

I haven't been directly involved with fixing this and haven't seen it myself. But can you suggest some steps to reproduce what you see?
Here are a few things that might be relevant:

Are the problem folders IMAP or POP?

IMAP

If IMAP, do you save full messages locally (the default) or just on the server?

Locally, or semi-locally. Due to the sheer volume of emails, for some users we have long time ago decided to use "time-based" synchronization. This is presented in the UI as "store locally only messages no older than X"

What is the mail server type, e.g., dovecot, gmail, exchange etc.?

Cyrus, it has not been upgraded recently, it's settings were not changed for half a year - at least.

Are you using using default mbox (default and one file per folder) or maildir (one file per message) as local storage?

mbox, I'm not crazy enough to try maildir :)

How many messages are in the problem folders? Do many have really large attachments?

Thousands of messages, sometimes dozens of thousands, some with attachments, max size 30mb.
However, issues occur with very small emails too. Hard to judge the chance of a problem vs email size - fewer big ones, more small ones, hence more chance for the problem to occur with small msg.

Any more details might be helpful.

This issue has come in waves.
On 17th of December two users in our remote office in Germany have experienced it.
It was solved by deleting the "ImapMail" folder in the profile dir.
We were not notified of subsequent issues with those users. However, it's unlikely that they continue to delete "ImapMail" on a regular basis.
There is 5% chance, that these two users were accidentally set with "beta" channel.
(due to sheer volume of new year's problems, including Firefox unable to connect to internet, and MS update breaking L2TP VPN, we are yet to verify those installations)

The storm came with 91.5.0 release. All hell broke loose. Some users had to delete "ImapMail" on a daily basis and they reported this.
We've decided to "guess" and introduce mitigations, which I've listed in my original comment.
This has helped greatly (far fewer corrupted emails), however I'm not able to pinpoint any specific mitigation that did the most of helping. Our goal was to enable our employees to work and have fewer support issues to solve.
As far as I can tell, there's no way to reliably reproduce this.

Here's something to try if IMAP and storing messages locally. This is just to see if the problem is coming from tb trying to read the mbox or maildir file to display a message. This will causes messages in a single folder to only be stored on server.
Right-click folder and choose Properties
Go to "synchronization" tab
Uncheck "select this folder for offline use"
Click OK

I know this procedure. We've used it, especially for folders under "Archives" tree, or shared mailboxes.
In general, this works, however yesterday we've had an instance of a corrupted email despite this setting. To complicate matters even more - upon inspection of "ImapMail", we've discovered that the folder for which we've disabled sync, had still it's "mbox" (non .mst) file present, along with .mst file. After deleting the "mbox" file, the issue was gone.

Right-click same folder and again choose Properties
Then choose "repair folder". Emails will seem to disappear and then come back when repair finished. May take a while if lots of messages in folder.

Yes, we know this procedure too and we're presently using it instead of deleting whole "ImapMail"

Now when you open an email in that folder it will be fetched from the server instead of from the local mbox or maildir file. Other folder will still read mbox or maildir to open existing messages.
Does this make a difference?
If this helps it might be considered a temporary workaround but mostly just to see where the problem is coming from.

Yes, we know of this procedure, we've immediately jumped to it when the problem revealed itself to be more widespread.

Thank You for Your help. I sincerely hope that this is fixed, otherwise we'd need to downgrade to 91.3.2, which will also involve deleting compatiblity.ini and widespread testing.

(In reply to Magnus Melin [:mkmelin] from comment #100)

This bug is/was only about tb 94.
There is no general problem with 91 in this regard. If you have a problem there, open a new bug about it, with details as per previous comment.

There was 91.5.1 recently, we'll wait for it to deploy and re-evaluate.
If our in-house mitigations prove to work, I'll not push this matter.
Shall the problems reappear, I will create a separate ticket. I'm also willing to submit the impacted mbox+mst file. However, I'd need to consult with legal due to GDPR & NDAs.

Thank You for Your hard work on Thunderbird.
Sorry, if my comments caused any chaos.

PS: Please delete comment 101 - it was badly formatted.

(In reply to filip.paczynski from comment #97)

since upgrading to 91.4 we have been experiencing these problem with various intensity on dozens of workstations:

That's when bug 163964 shipped.

As Magnus said, this bug report is explicitly for beta users only. Beta users only, who have ongoing issues, should see Bug 1742975 - Bug 1734847 (.msf corruption) NOT FIXED on beta. (NOT VERSION 91.x)

Magnus also indicated there are no reports of this type of issue for version 91. I concur, there are no matching issues for version 91 in this bug query https://mzl.la/3rTx3st (though I don't rule out my query could be inadequate)

since upgrading to 91.4 we have been experiencing these problem with various intensity on dozens of workstations:

There is also nothing in the list of patches applied to 91.4 and there after that would likely cause such a problem.

If you can reproduce these issues with Thunderbird started Help > Troubleshoot Mode, and Windows started in safe mode, then please file a new bug report or support request

Summary: TB (beta) 94.0b1 occasionally mis-indexing mail "subject" with previous inbox msg → TB (beta regression) 94.0b1 occasionally mis-indexing mail "subject" with previous inbox msg (NOT VERSION 91.x)

(In reply to Wayne Mery (:wsmwk) from comment #105)
OK I will act as I've outlined in comment #103.

However, I see no way for reproducing this. Maybe if I was able to record all CPU, memory, and network activity and be able to "replay" it, but I'm not.

Thank You for Your hard work on Thunderbird.

(In reply to Wayne Mery (:wsmwk) from comment #105)

There is also nothing in the list of patches applied to 91.4 and there after that would likely cause such a problem.

The entire IMAP startup was changed: Bug 163964 comment #83, it's even written in the release notes.

(In reply to newsfan from comment #107)

(In reply to Wayne Mery (:wsmwk) from comment #105)

There is also nothing in the list of patches applied to 91.4 and there after that would likely cause such a problem.

The entire IMAP startup was changed: Bug 163964 comment #83, it's even written in the release notes.

I wouldn't think that (my) change of Bug 163964 would cause internal folder corruption but who knows. I haven't seen it myself, but I mostly use "keep messages only on server". Will ask Richard Leger who requested the change to do folder discovery earlier and see if he sees problems. He has tons of folders that could potentially go bad -- but, if I remember right he also keeps messages on server...

Richard, I'm referring to comments above by filip.paczynski. Thanks.

Note: Bug 163964 title says "Skipping discovery". What it really does is just run the discover folder URL in it's own imap thread. It doesn't "skip" discovery.

Flags: needinfo?(richard.leger)

From comment 102:

To complicate matters even more - upon inspection of "ImapMail", we've discovered that the folder for which we've disabled sync, had still it's "mbox" (non .mst) file present, along with .mst file. After deleting the "mbox" file, the issue was gone.

I think if you un-tick/disable sync (offline storage) for a folder the mbox file is not deleted automatically and it continues to be used for existing emails. Just new emails aren't added to the mbox file. But if you also do a repair after the un-tick, the mbox file goes away and all message fetches go to the server. This is based on observation. So manually deleting the mbox file probably fixed the issue by causing all accesses to go to the server.

(In reply to gene smith from comment #109)

From comment 102:
But if you also do a repair after the un-tick, the mbox file goes away and all message fetches go to the server. This is based on observation. So manually deleting the mbox file probably fixed the issue by causing all accesses to go to the server.

This "stale" mbox file is not always deleted after "repair".
It's seems random, in about ~70% cases it is deleted, but sometimes it's not.
I think TB should delete this file immediately after disabling sync for a give folder.

Hi Gene,

(In reply to gene smith from comment #108)

I wouldn't think that (my) change of Bug 163964 would cause internal folder corruption but who knows. I haven't seen it myself, but I mostly use "keep messages only on server". Will ask Richard Leger who requested the change to do folder discovery earlier and see if he sees problems.

I can confirm I have encountered this Bug 1734847 myself few times in the past 6 months but not recently (in the past month or so) in beta. I could not tell in which version of TB it occurs as I use both beta and ESR in parallel, it has been too long... and it so random and not reproducible that it is very hard to troubleshoot. It happened to me mostly in Inbox folder.

It is not impossible also that I have changed my offline sync options during that period of time, up to few days ago I was set to sync and cache locally only 3 days worth of email for Inbox folder but I have now disabled that feature to work online since.

I noticed in beta that disabling offline mode for Inbox was not always immediately effective the first couple of time it was disabled... the option seem to re-tick itself after closing/re-opening options or TB. Not sure why... I would expect TB to immediately apply the change and delete cache repair the folder as needed. I think you are onto something with your Comment 109...

Like you I do believe that changing sync (offline options) does not always cause Thunderbird to behave as expected and that the local cache is not "cleared" properly (mbox or else deleted) properly when disabling offline mode. Though I haven't properly troubleshoot nor tested that theory... Disabling offline mode for a specific folder shall cause all cache files to be deleted and folder repaired automatically so only headers of message would remain.

Like you I do not believe that Bug 163964 is the root cause because I seems to recall it was happening before it was fixed and published (for quite a long time maybe past bug report may confirm it). As you may recall, the fix Bug 163964 was also delayed upon my request at the time because end-users were already having issues with beta version keeping re-downloading emails... which is different behaviour from this bug... but touch to the same similar mechanism/files and was possibly due to corrupted mbox/msf file issues or else that even a repair folder would not resolve... some had to manually clear the file on disks to fix if I recall correctly. A specific set of beta version were somehow corrupting the file wihtout TB being able to repair properly in successive updates...

Coming back to this Bug 1734847 it is likely linked to sync feature and changed in it... or corruption of cache files/indexing somehow... or access to them...

I seem to have noticed, when encountering it, that if you click on other emails or switch folder before coming back to the email which content was showing the previous email content, it sometime sort the issue. So it could well be a UI view issue... but I cannot know for sure as random in a blue moon and not reproducible at will...

filip.paczynski seems to suggest that deleting the ImapMail folder fix the issue and that end-users encountering the issue may have used some beta version... which maybe at some point corrupted the locally cached data file in a way that even a repair folder could not fix. Only deleting manually the cached file via File Explorer fixed it. At least for a while...

It may also be happening only after a certain number of email/data is cached in the file... but why so randomly is also very strange...

My suggestion would be to troubleshoot and run tests on what happens every-time the sync options are changed especially via the Option tab and if TB behave like it should. When offline mode is disabled on a folder end-users shall be prompted and asked to for clearing the locally cached data and related files (mbox/msf) or not... or TB do it automatically. If cache data remain as breadcrumbs, TB shall stop using it in online mode.

Nothing definite of course...

Regards,

Flags: needinfo?(richard.leger)

(In reply to Dan Howard from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36

Steps to reproduce:

Restarted TB (beta 94.0B1) after TB automatically updated, checked for new mail.

Actual results:

When retreving mail TB 94.0b1 now occasionally displays "subject" of email correctly, but when opening mail it shows occasionally shows content of next previous email in "inbox" (duplicates the the content of the previous email) and apparently discards (or at least does not display) the actual contents of the newer email. Restatrting Thunderbird and then restarting PC had no effect. Help!

Expected results:

Correct mail subject and content should be displayed

This is still happening in 98.0b1 (64-bit) and I've lost many, many, many messages. It's also only happening in one Gmail IMAP account and no others. I have done everything I can possibly think of but nothing has worked.

When you say "lost many, many messages" do you mean they are gone from the gmail server? I.e., can you see them in webmail? Are you using the default storage of the messages locally? Do you see this on old messages that have been around a while or just on newly arrived messages? Is gmail setup with default imap settings at the gmail site? Is the problem only seen on Inbox or on all folders. Do you have a lot of message in "All Mail"? Is "All Mail" subscribed in tb? Is tb setup to move messages to trash when deleted (default)?
Sorry for the random collection of questions. Haven't been able to duplicate the problem here.

(In reply to gene smith from comment #113)

When you say "lost many, many messages" do you mean they are gone from the gmail server? I.e., can you see them in webmail? Are you using the default storage of the messages locally? Do you see this on old messages that have been around a while or just on newly arrived messages? Is gmail setup with default imap settings at the gmail site? Is the problem only seen on Inbox or on all folders. Do you have a lot of message in "All Mail"? Is "All Mail" subscribed in tb? Is tb setup to move messages to trash when deleted (default)?
Sorry for the random collection of questions. Haven't been able to duplicate the problem here.

So many questions!

"When you say "lost many, many messages" do you mean they are gone from the gmail server?"

They're still on the server as I've been using an IMAP account but the local messages, the ones I've downloaded and stored on my machine were corrupted at some point. I first noticed while looking through an archive and found dozens of the same headers but with either nothing in the message window or some other message in the window. The suggestion of a fix might have worked with a few messages but with the thousands I had it wasn't possible.

"Do you see this on old messages that have been around a while or just on newly arrived messages? "

For the past few weeks it's been happening with new emails coming from one particular gmail account.

"Is tb setup to move messages to trash when deleted (default)?"

Yes. However, it turns out that none of them were deleted from the server and I found that out after creating a POP account (see below) and it downloaded thousands of deleted messages.

The other day I created a new POP account tapping that gmail account and there's been no problems with incoming messages though old ones in the local IMAP folders are still often garbage. When I started moving messages from the old folders to the new TB would hang... seeming to have problems when accessing the Master FIle Table and Pagefile.sys. And hang. And bring my machine down with it. Using the resource monitor I'd kill the process and watch it take ten, fifteen minutes to clear itself.

TB also seems to have a habit of scanning and reading my entire fonts directory and will hang on that as well for a very long time and then crash again accessing the pagefile and the master file table. (I'm on Win10, by the way).

(as for the machine, I've cleaned the hard drive, checked for updates, run chkdsk and sfc and DISM and you name it thinking that my machine might be at fault though only TB has this problem. Interestingly, when it does, Firefox balks and will sometimes freeze as well, also accessing the page file and the MFT, I suppose waiting for TB to finish what it's doing - which according to the monitor is - nothing.

As I was moving old files and watching TB crash I decided to move them two or three at a time and found the new local folder claiming 757mb of messages(!) from only 6 and those without attachments. I got that cleaned up - somehow - but the old folder still claims 58MB of messages with only a couple hundred text messages, none with images and none with attachments. But I dare not touch them(!) at least for now. I have repaired the folder and compacted it but neither has made a difference.

I've gone through every posted 'fix' that I could find for getting TB back to normal and none have worked.

Now, this morning, and trying not to have a kinnehorah here, it's been fine. But I'm wondering if during any of the updates some old DLL file or XUL or other program files were corrupted or got left behind and is sending faulty information to my system, thus the hangs. Is there a way to delete the TB program executables (and attendant files) and reinstall a fresh copy without having to recreate the dozens or so accounts I've got here?

Thanks for the attention to this. Iv'e been a TB user since I switched from Eudora and it's never had these problems until a few months back when all this began.

This is still happening in 98.0b1

I assume you are on the beta channel for updates? Do/did you see problems with the release version (91 is latest I think)?

I'm not sure what happens if you try to completely uninstall tb in win10. From what I read it does remove your emails stored in the profile too. So if you want to try that, I would recommend first saving the complete profile content somewhere so you can restore it back after you reinstall. You should find the profile at a location like this: c:/Users/<your-name>/AppData/Roaming/Thunderbird/Profiles/

Also, I would recommend first just installing the stable release 91 to make sure that works instead of running a beta. If 91 release works, then you might reinstall again the latest beta if you want to stay on the (b)leading edge.

There may be another issue if you try to run 91 with a profile that was touched by a newer version like 98b. If you get a message that a new profile is required to be setup (which you are trying to avoid) you will have to run tb 91 with command parameter --allow-downgrade. So maybe just staying with the beta channel would be OK, not sure.

Ignore this if you've already tried exaactly this: Another thing to try, instead of a re-install, is just completely remove the problem gmail account (including the mail content) and re-setup just that account. When setting it up go into "advanced" mode (you will be prompted to verify it's OK) and before clicking on Inbox to start bring in folders and messages, I would suggest disabling local full email storage by going to account settings and choosing "Synchronization and storage" and then uncheck "keep messages in all folders for this account on this computer" (you will be prompted again to save this setting). Then click on Inbox of the gmail account to bring in just the message headers. With this setup, when you click an existing message it will be fetched from the server and is not permanently stored locally (only the header is stored in a database).
If this helps and you don't see corrupted messages, you can still right click a folder and select it for "offline use" which will then download all the messages. So maybe you only see problems after you have downloaded the messages? In any case, I would recommend keeping at least the "All Mail" folder with no local storage and maybe even unsubscribe it.

(In reply to gene smith from comment #115)

Does this mean anything to you:

thunderbird.exe!TargetNtUnmapViewofSection+0cxd10

If I use Process Explorer that's the thing that TB gets gets hung up trying to do.

And if I look at the stack I get:

XUL.DLL!SoundTouch:operator+0xb1e74a

J

I'll also note that while poking around the troubled IMAP folder the sent mail file was invisible and at 47gig it was taking up a lot of space! The inbox was 25 gig! with two of the storage files being 12.9 and 12.1 gig in size but there are only a few messages in each folder. This is what was happening.

What I've done is, as noted above, create a POP account for the same gmail address and then moved the IMAP folder to a safe place for storage in case there's anything in the old archives (I am sure there will be) that I'll need in the future. Now all I need to do is to download the emails from gmail that I couldn't get to in the IMAP folder and I'll be good to go. But man, someone should take a look into why this happened.

Re comment 116: No idea what that means.

Re comment 117: So are you saying you are just moving to POP instead of accessing gmail with IMAP? I haven't used gmail with POP but I would think you can only "download" emails that are in Inbox. Maybe you will have to move emails in various other folders (actually labels) to gmail INbox at gmail.com webmail to access them with POP? Any folder on tb you set up with POP, except inbox, will actually just be local storage and won't reflect anything at gmail (unless you manually keep them in sync somehow).

But man, someone should take a look into why this happened.

Don't see a problem here so a bit hard to fix something I can't duplicate. :) But not sure, other tb people may be looking at this too.

See Also: → 628646

(In reply to gene smith from comment #118)

Re comment 116: No idea what that means.

Re comment 117: So are you saying you are just moving to POP instead of accessing gmail with IMAP? I haven't used gmail with POP but I would think you can only "download" emails that are in Inbox. Maybe you will have to move emails in various other folders (actually labels) to gmail INbox at gmail.com webmail to access them with POP? Any folder on tb you set up with POP, except inbox, will actually just be local storage and won't reflect anything at gmail (unless you manually keep them in sync somehow).

But man, someone should take a look into why this happened.

Don't see a problem here so a bit hard to fix something I can't duplicate. :) But not sure, other tb people may be looking at this too.

I understand.

What I do know is that right now there are 13 brief drafts in the folder for that account and each short message is somehow 11.9 meg in size. This holds true for every folder in that account.

So yeah, I've deleted what I could, moved the rest to an off-drive location and rebuilt as a POP3 account and so far TB has behaved like it should, Quick, reliable and sure.

Thanks for your attention and your help. It is much appreciated.

JmG

See Also: → 1741517
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: