Closed Bug 1924950 Opened 28 days ago Closed 24 days ago

The message list is there; the message content is blank before 26/7/24

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Version:
Thunderbird 132
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1911076

People

(Reporter: sandifordworld, Unassigned)

Details

Steps to reproduce:

My thunderbird version is 132.0b4 (64-bit) and 'up to date'.

This problem only occurs for my Local Folders Inbox. My email supplier is Telstra Bigpond in Australia. I download all the emails and do not leave them on the Bigpond Server.

When I searched the Inbox for a 'sender', the list of emails appeared. Clicking the latest in the list caused the email to appear. Clicking earlier ones left the message content as blank.

I then looked at the list of all my emails (in date order - latest first). The email contents appeared for all emails down to 26/7/24. There are many earlier emails.

Actual results:

This problem happened to me several weeks ago, but it appeared to disappear with an automatic software update of the beta. On the earlier occasion, all email contents were blank except for the latest email downloaded.

On that occasion, I copied my emails profile directory onto another computer not running the beta version. Fortunately, all the emails worked as normal.

I can't remember what I did then, but I put the profile back onto my beta-version computer after a new beta updater and everything worked normally.

Expected results:

The beta version should not be failing to display email content.

I can't go on like this. If you cannot fix the problem with an update, I need to abandon the beta version. I'm only using it because it seemed a good idea at the time.

What should happen now is a beta update that fixes the bug.

What is the size of your Inbox, both in GB and number of messages?

Flags: needinfo?(sandifordworld)

The profile directory is34.5Gb, containing 855 files in 93 folders. There are 75,703 messages in the Inbox.

Flags: needinfo?(sandifordworld)

(In reply to Wayne Mery (:wsmwk) from comment #2)

What is the size of your Inbox, both in GB and number of messages?

The profile directory is 34.5Gb, containing 855 files in 93 folders. There are 75,703 messages in the Inbox.

Size the pop Inbox please.

How can I find the size of the pop Inbox? The "Mail" directory in the profile directory is 25.0Gb. Is that the one?(In reply to Wayne Mery (:wsmwk) from comment #5)

Size the pop Inbox please.

How can I find the size of the pop Inbox? The "Mail" directory in the profile directory is 25.0Gb. Is that the one?

(In reply to Wayne Mery (:wsmwk) from comment #5)

Size the pop Inbox please.

Ah, I right-clicked on Inbox and looked at Properties. It says the Size on Disk is 4.1Gb.

I notice the Properties box gives an option to Repair Folder. Should I try that now?

See Also: → 1890230

I notice the Properties box gives an option to Repair Folder. Should I try that now?

Yes.

(In reply to Wayne Mery (:wsmwk) from comment #8)

I notice the Properties box gives an option to Repair Folder. Should I try that now?

Yes.

OK, I tried Repair Folder and the result was that all emails were deleted from 26/7/24 back down to 27/5/21 (three years' worth).

Earlier than 27/5/21, the emails are OK. I reloaded the original, unrepaired Profiles and found the emails are OK too earlier than 27/5/21.

So, emails are scrambled between those two dates. 'Repair Folder' does not repair the emails; it just deletes ones it thinks are damaged.

The actual email data is there; it's not lost. It ought to be possible to write a better Repair program that can work out what is still good, recover that and just leave unrecoverable ones scrambled. the Repair program should not delete any emails.

Also, note that it is the email content that is scrambled. The email Subject that appears in the top window is not scrambled. The Subject (and date and Sender, etc) has some value and is another reason not to delete the scrambled-content emails.

Since I have had this problem twice in less than two months, I think it's a software bug. My guess is that it's the Compress function that is causing the damage - just a guess.

Version: unspecified → Thunderbird 132

As stated in 'Steps to reproduce' above -

My thunderbird version is 132.0b4 (64-bit) and 'up to date'.

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

(In reply to Francesco from comment #11)

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

Thank you,
I will try what you suggest tomorrow and let you know what happens.

Did you always use the 64-bit version of Thunderbird with this profile?

Keywords: dataloss
Flags: needinfo?(sandifordworld)
Keywords: dataloss

(In reply to Francesco from comment #11)

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

  • I installed version 115.16.2
  • I did not configure an account
  • In Account Settings I was able to 'add a Feed Account'
  • A new Profile was created
  • I looked in the "Mail" subdirectory of the new Profile
  • It only contained a "Feeds" directory; BUT it did NOT contain a "Local Folders" directory
  • I created a "Mail\Local Folders" directory anyway
  • In my 132.0b5 (now auto-updated from version 132.ob4) Profile, I looked in "Mail\Local Folders" and found a file called "Inbox" of size 4+GB
  • I copied the "Inbox" file from 132.0b5 into the "Mail\Local Folders" directory of 115
  • I started TB 115
  • TB 115 only shows "Feed Account". There is no "Inbox"

It looks like I almost made it. Where do I go from here?

Flags: needinfo?(sandifordworld)

(In reply to Magnus Melin [:mkmelin] from comment #13)

Did you always use the 64-bit version of Thunderbird with this profile?

I have used TB for many years. I don't know why I installed the Beta version on my desktop computer. It seemed a good idea at the time. I think it must have been over a year ago that I changed to the Beta. When I downloaded and installed the Beta, I chose the 64-bit version. Since first install, it has always automatically updated. When I started with the Beta, it must have automatically used the Profile that was already there from the non-Beta version. Non-Beta versions may have been 32-bit in the past, but I don't think so.

Everything went fine with the Beta for many months. Then around 8th August 2024 I started TB as usual and the latest emails in the Inbox were blank. New emails being read in were OK, but there was obviously a major problem. I can't remember what I did, but I think I recovered the Profiles directory from a previous day's daily backup. Also, I seem to remember that the day before, TB had automatically run a Compact Folders. That did not cause problems at the time. The problems occurred when I started TB the following day. Anyway, after recovering the previous day's Profile successfully, I ran Compact Folders from "File" in the Main Menu. Everything carried on normally for days.

Problems occurred again around 6th September 2024. Again, I started TB and the latest emails in the Inbox were blank or corrupted as before. While I was trying to sort things out, I noticed there was a big update to the Beta version of TB. Anyway, I managed to get things going again on the desktop computer. Again, I had a feeling there had been an automatic "Compact Folders" just before the problem.

Also, at 6th Sept. I wondered about going to a non-Beta version of TB. The TB version on my laptop is 128.3.1esr. I tried to copy my Beta Profile to the laptop and hit the "later version not allowed" problem. However, I think I overcame that by editing the "profiles.ini" file and changing the default profile in "Troubleshooting Information". I think, at some point, I could read the 132 Beta emails in the 128 version on my laptop. Anyway, that is just in passing.

What has happened now is that I tried to look at an old email on my desktop computer (running 132 Beta) and I saw it was blank. The emails around it were blank or scrambled text. Then I discovered there was a problem with all the emails from July 2024 down to May 2021.

This is scary because I only found there was a problem by chance. Those emails may have been corrupted on 6th Sept, but I don't think so. I ran Mailstore Home after recovering from 6th Sept. and the emails are all there as normal.

Unfortunately, my daily backups only go back to 26th Sept and that earliest profile is corrupted. So, the latest problem occurred between 6th Sept. and 26th Sept. But I didn't find out until a few days ago.

I do have a desktop backup dated 11th May 2024 and all the emails are good for all dates. Ideally, I should like to replace the corrupt emails with those good one, but I don't know how to copy a chunk of emails from one profile to another.

Also, I can't go on like this and I need to get to a stable version of TB.

Do you have any suggestions?

(In reply to Magnus Melin [:mkmelin] from comment #13)

Did you always use the 64-bit version of Thunderbird with this profile?

Now, I just checked with the laptop and it has a 32-bit version of 128.3.1esr. So, it is possible that when I was trying to fix the problem on 6th Sept., I may have got a working Profile on the laptop and put that back on the desktop. However, as I mentioned, I ran "Mailstore Home" on the desktop after I had the emails restored. I can now search Mailstore Home for an email in the "corrupted" time-range and emails appear correctly However, having just check again today, this only works for some emails.

I am probably better off trying to use the Profile backup of 11th May 2024 to replace corrupted emails up to that date.

(In reply to Francesco from comment #11)

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

... and now version 115 vas updated itself to version 128.3.2esr !!

There is still no Inbox.

(In reply to David S from comment #17)

(In reply to Francesco from comment #11)

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

... and now version 115 vas updated itself to version 128.3.2esr !!

There is still no Inbox.

I could set up a small, existing Gmail account to create a "Local Folders" directory to experiment with. Would that help?

Or I can set up my main Bigpond account?

(In reply to David S from comment #18)

(In reply to David S from comment #17)

(In reply to Francesco from comment #11)

Please try this:
Install TB 115 to a different location, start it, don't configure an account. In Account settings, add a "Feed Account", that will give you "Local Folders". From you backup, copy the POP Inbox (not the .msf) to "Mail\Local Folders" in the new profile for TB 115. Then start TB 115, navigate to "Local Folders - Inbox". You need to wait a bit while TB 115 rebuilds the .msf from the 4+ GB Inbox. Are the messages between 2024 and 2021 OK?

... and now version 115 vas updated itself to version 128.3.2esr !!

There is still no Inbox.

I could set up a small, existing Gmail account to create a "Local Folders" directory to experiment with. Would that help?

Or I can set up my main Bigpond account?

OK, I used the "Import from another Thunderbird profile" option in 128.

  • That did create a "Local Folders" directory.
  • I copied across my "Inbox" file from 132.
  • It did build the ".msf" file, as you said.
  • Unfortunately, the emails are missing in the corrupted range from May 2021 to July 2024. The process effectively repeated the earlier "Repair"

I think I will have to try to recover from the good backup I have that goes up to 11/5/24. But how can I do that?

Sorry for the wrong advice of creating a Feed Account. It doesn't give you Local Folders, my mistake. Instead, create a News Account with some dummy values. It gives you "Mail\Local Folders" and if you copy an inbox into it, it will be picked up on the next start. I've tried it now.

The whole exercise it about providing a "testing area" where you can use TB 115 to repair your Inbox to see what you've got. Of course you can also set up any other account (Gmail, Bigpond) which will give you local folders for testing, but it's not necessary.

Yes, while working on the profile, TB will update from 115 to 128, which is undesired. You can prevent auto-update in three ways:

  1. Disconnect the PC from the network or block its access in the Windows firewall (tricky)
  2. Use the DisableAppUpdate policy (somewhat tricky)
  3. Create this Windows registry entry: DWORD DisableAppUpdate (vale 1) under HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mozilla\Thunderbird. If you're familiar with the Windows registry, this one is easy.

Magnus asked about the 32bit version since that had a bug which truncated large mailboxes (> 4 GB) to 4 GB losing messages which were stored in the mailbox beyond the 4 GB border. That's typically the latest messages, since new messages get appended at the end, in your case the mail from 2024 back to 2021. It that's the case, no repair will bring back those messages. You need to restore from backup.

Thanks, Fran(In reply to Francesco from comment #20)

Sorry for the wrong advice of creating a Feed Account. It doesn't give you Local Folders, my mistake. Instead, create a News Account with some dummy values. It gives you "Mail\Local Folders" and if you copy an inbox into it, it will be picked up on the next start. I've tried it now.

The whole exercise it about providing a "testing area" where you can use TB 115 to repair your Inbox to see what you've got. Of course you can also set up any other account (Gmail, Bigpond) which will give you local folders for testing, but it's not necessary.

Yes, while working on the profile, TB will update from 115 to 128, which is undesired. You can prevent auto-update in three ways:

  1. Disconnect the PC from the network or block its access in the Windows firewall (tricky)
  2. Use the DisableAppUpdate policy (somewhat tricky)
  3. Create this Windows registry entry: DWORD DisableAppUpdate (vale 1) under HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Mozilla\Thunderbird. If you're familiar with the Windows registry, this one is easy.

Magnus asked about the 32bit version since that had a bug which truncated large mailboxes (> 4 GB) to 4 GB losing messages which were stored in the mailbox beyond the 4 GB border. That's typically the latest messages, since new messages get appended at the end, in your case the mail from 2024 back to 2021. It that's the case, no repair will bring back those messages. You need to restore from backup.

Thanks, Francesco.
I do have a backup with good emails all the way down from 11/5/2024 to 26/8/2014.
I should like to bring those emails into my 132 version so that I can delete the corrupted emails and replace them with the backup ones.
How can I do that?

You can actually add mailbox files to your profile manually.

So where you have Inbox and Inbox.msf, place older inboxes restored from backup, say, Inbox2 and Inbox2.msf. If Inbox2 is your backup, you can then move messages from Inbox into Inbox2 until you have a complete set. Then you can delete Inbox and Inbox.msf in the Windows folder and rename Inbox2 and Inbox2.msf. Of course you can add Inbox3, etc. as you deem fit to compare.

I'll close the bug a duplicate of the bug for the issue in the 32bit version. Feel free to ask more questions here.

Also to summarise the issue: The loss of message occurred when you used a 32bit version of TB after version 122. The issue was only fixed in TB 130 beta and TB 128 ESR.

Status: UNCONFIRMED → RESOLVED
Closed: 24 days ago
Duplicate of bug: 1911076
Resolution: --- → DUPLICATE
See Also: 1890230 →

(In reply to Francesco from comment #22)

You can actually add mailbox files to your profile manually.

So where you have Inbox and Inbox.msf, place older inboxes restored from backup, say, Inbox2 and Inbox2.msf. If Inbox2 is your backup, you can then move messages from Inbox into Inbox2 until you have a complete set. Then you can delete Inbox and Inbox.msf in the Windows folder and rename Inbox2 and Inbox2.msf. Of course you can add Inbox3, etc. as you deem fit to compare.

I'll close the bug a duplicate of the bug for the issue in the 32bit version. Feel free to ask more questions here.

Also to summarise the issue: The loss of message occurred when you used a 32bit version of TB after version 122. The issue was only fixed in TB 130 beta and TB 128 ESR.

*** This bug has been marked as a duplicate of bug 1911076 ***

It's interesting it turned out to be the 32-bit/4Gb-limit issue. I can remember 64-bit computers becoming available at the end of the 1990s, solving that problem.

It's good there is a simple way to restore emails from a backup. That's useful to know.

Thank you for your help and patience.

You need to log in before you can comment on or make changes to this bug.