1773511 - sent message does not copy to save in folder after update to 102 beta

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect)

Description

[David McDivitt]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36

Steps to reproduce:

THIS IS A SEVERE PROBLEM. PLEASE SHARE ANY IMMEDIATE SOLUTIONS.

I upgraded to version 102 beta yesterday with overwrite.
Sent my first email with new version today.
Gasped in horror when sent email was not in sent folder.
Sent a test email to my work address.
Email was received at my work address but did not appear in sent folder.
Used thunderbird search on all folders and could not find messages sent.
Closed thunderbird and did binary file search finding no files with the text.

Actual results:

Emails just sent did not show in my sent folder, or any sent folder for any of my email accounts.

Expected results:

The message should have appeared in my sent folder as usual.

Comment 1

[David McDivitt]

I am looking through options and configurations trying to see if a "save sent" setting exists. If I find that and set it, we still have a huge issue because the setting did not persist when updating the application. I have also sent a few more emails with none copying to my sent folder, but I've been cc'ing myself.

Comment 2

[Arthur K. (he/him)]

I updated from 101 to all the recent 102 betas via installing to C:\Program Files\Mozilla Thunderbird (instead of C:\Program Files\Mozilla Thunderbird Beta) and anything I've sent is in my [Gmail] > Sent Mail folder. The generic Sent folder for one of my GMail account has one last mail sent from 101 and after updating to 102, I no longer see any headers indicating sent from 102. My other GMail's Sent folder does have messages in it sent from 102 as recently as a few minutes ago.

By chance is the mail folder with messages NOT showing in the Sent folder not a regular GMail account. Meaning, a corporate / org Google Mail account?

Comment 3

[Arthur K. (he/him)]

Attachment: settings.png

Double check to make sure that under your Account Settings > Copies & Folder that it didn't set or change "Place a Copy In" to (*) Other > Local Folder > Sent. It should look like the image to be correct.

I noticed on my org account it somehow got set to (*) Other > Local Folder > Sent and, sure enough, those messages sent via 102 were sitting there in a sub-folder in Local Folder > Sent.

Comment 4

[David McDivitt]

This was strange. First off I did have the issue reported. My sent emails were not copied to the sent folder of the account used to send the message. I have four POP3 accounts and one gmail account. I rarely if ever use the gmail account. The POP3 accounts go to my own domains and are not IMAP.

So, I looked at the image posted by Arthur K. For the account where sent emails were not copied to the sent folder, "place a copy in" was checked, "sent folder on" was selected, but in the text box it had "local folder". I did not change it. I looked at each of my accounts and each was correct except that one. This was weird. I closed settings but not the application, right clicked on the account again and chose settings. This time, without changing anything, it had the correct value in the text box instead of "local folder". I sent a test email and it copied to the sent folder of the account.

After updating to 102 beta I opened and closed the application several times and sent emails. None of the sent emails appeared in the sent folder. But, displaying account settings, without changing anything, appears to have resolved the problem. There is no "save" button. Stuff saves when changed. Obviously, clicking on configurations for the account induced a recalculation for the problem value, and since now different, and fortunately correct, it saved that way.

This will be hard to duplicate. I save my entire email folder structure each night. I leave emails on the server ninety days or so. On occasion I restore and pull all current emails again. I could restore my folder, uninstall, reinstall whatever current version of the app, pull emails, then upgrade to 102 beta again, but that would be a lot of work. I will play with it tomorrow and see if I can make the same thing happen again.

Comment 5

[David McDivitt]

Reading Arthur K. again, whereby he mentioned "local folder", I looked and found a sent sub folder there. Yes, it did have the missing messages. However, not all missing messages were enumerated in the message list. If I double-clicked one to view it, two messages would open, and the second messages was one I was missing. This is very strange. One message was embedded in the other message when looking at the list. So with comments by Arthur K. we have a second instance of the problem I reported.

If I do a binary search of the email folder, no file contains the missing email, but there it is in Thunderbird in local/sent. Very strange.

Comment 6

[Ping Chen [:rnons] (away)]

You must have changed username/hostname before, when you upgraded to 102, the fcc_folder pref still pointed to the old username/hostname. Then with the changes in bug 1483485, the old username/hostname got removed, so fcc_folder can't be found.

fcc_folder and some other prefs are associated with identity, not server, I forgot to migrate them. Let me think about a fix.

Comment 7

[Arthur K. (he/him)]

(In reply to Ping Chen (:rnons) from comment #6)

You must have changed username/hostname before, when you upgraded to 102, the fcc_folder pref still pointed to the old username/hostname. Then with the changes in bug 1483485, the old username/hostname got removed, so fcc_folder can't be found.

fcc_folder and some other prefs are associated with identity, not server, I forgot to migrate them. Let me think about a fix.

Could be. The messages I had in my Local Folder > Sent dated from 2017 through April 2020 and then begin right at June 1st 2022 when I bumped up to 102. I for sure didn't mess with any account settings. Gremlins. =)

Comment 8

[Arthur K. (he/him)]

Interesting that the Copies & Folders setting on my work PC's org account is pointing Sent, Drafts and Templates messages to folders in my Local Folder. Seeing as how I re-did from scratch my entire setup and accounts this February, I know for sure I did not touch this. And starting with 102, Sent messages are being put in Local Folder > Sent. Ping I don't know if your theory in comment 6 is 100 possible.

Comment 9

[Ping Chen [:rnons] (away)]

Attachment: Bug 1773511 - Save a single page in Account Settings to prevent resetting prefs unintentionally. r=mkmelin

Comment 10

[Ping Chen [:rnons] (away)]

Attachment: Bug 1773511 - Migrate identity folders when changing username/hostname. r=mkmelin

Depends on D148972.

Comment 11

[Rachel Martin]

Ping, please note the last paragraph in bug 1773822 comment #0.

Comment 12

[Ping Chen [:rnons] (away)]

I don't think corrupted or missing .msf file affects fcc_folder, I just tried removing .msf file manually and fcc_folder is intact.

Comment 13

[Rachel Martin]

Agreed, after the removing the MSF file a sent message even gets appended to the Sent folder. We noticed that on many accounts the fcc_folder was changed to local folder, so hopefully your fix will address this for upgrading users. When correcting the fcc_folder we always got this message:
The Junk settings on account "xxx" have a possible problem. Would you like to review them before saving Account Settings?
Maybe that's related. We weren't able to work out what the problem was. Potentially related to bug 1773618.

Comment 14

[Rachel Martin]

One more thing we noticed. (Re-)setting the fcc_folder doesn't appear to immediately apply the "Sent" folder flag to the new target folder. The flag gets set after a restart.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/dba6cfe14e25
Save a single page in Account Settings to prevent resetting prefs unintentionally. r=mkmelin
https://hg.mozilla.org/comm-central/rev/809ef03f835c
Migrate identity folders when changing username/hostname. r=mkmelin

Comment 16

[Arthur K. (he/him)]

I bumped up to 102.0b5 via the full installer and a few of my accounts switched their setting back to the bullet option for "Other" under Copies & Folders. And even after fixing the setting, a restart of TB will put some of them back to the "Other" setting again.

Comment 17

[Ping Chen [:rnons] (away)]

The patches have not be uplifted yet, and will only work if you still have realusername/realhostname in your prefs.js.

(In reply to Arthur K. [He/Him] from comment #16)

And even after fixing the setting, a restart of TB will put some of them back to the "Other" setting again.

I don't know, maybe you're seeing a different bug. Can you check the fcc_folder before and after restarting TB?

Comment 18

[Ping Chen [:rnons] (away)]

Comment on attachment 9280796 [details]
Bug 1773511 - Migrate identity folders when changing username/hostname. r=mkmelin

Request for two patches.
[Approval Request Comment]
Regression caused by (bug #): bug 1483485
User impact if declined: if have changed username/hostname before, Copy/Draft/Template folders maybe reset to Local Folders after upgrading
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): account settings page is also changed a bit, but should be fine

Comment 19

[Wayne Mery (:wsmwk)]

Comment on attachment 9280796 [details]
Bug 1773511 - Migrate identity folders when changing username/hostname. r=mkmelin

[Triage Comment]
Approved for beta

Comment 20

[Arthur K. (he/him)]

(In reply to Ping Chen (:rnons) from comment #17)

I don't know, maybe you're seeing a different bug. Can you check the fcc_folder before and after restarting TB?

At the risk of sounding stupid.....the fcc_folder? Where is that?

Comment 21

[Rob Lemley [:rjl]]

Thunderbird 102.0beta6:
https://hg.mozilla.org/releases/comm-beta/rev/dc2221434b07
https://hg.mozilla.org/releases/comm-beta/rev/6f105a87b6c3

Comment 22

[Ping Chen [:rnons] (away)]

(In reply to Arthur K. [He/Him] from comment #20)

(In reply to Ping Chen (:rnons) from comment #17)

I don't know, maybe you're seeing a different bug. Can you check the fcc_folder before and after restarting TB?

At the risk of sounding stupid.....the fcc_folder? Where is that?

• Find your profile folder on disk first (From Help > More Troubleshooting Information, search for Profiles. Open about:profiles, search for This is the profile in use)
• Open prefs.js in your profile folder, search for fcc_folder. An example is user_pref("mail.identity.id2.fcc_folder", "imap://[e-mail]/Sent");, which points to [e-mail].

The bug I fixed was username/hostname had been changed before, so [e-mail] didn't match any account.

Comment 23

[Arthur K. (he/him)]

(In reply to Ping Chen (:rnons) from comment #22)

(In reply to Arthur K. [He/Him] from comment #20)

(In reply to Ping Chen (:rnons) from comment #17)

I don't know, maybe you're seeing a different bug. Can you check the fcc_folder before and after restarting TB?

At the risk of sounding stupid.....the fcc_folder? Where is that?

• Find your profile folder on disk first (From Help > More Troubleshooting Information, search for Profiles. Open about:profiles, search for This is the profile in use)
• Open prefs.js in your profile folder, search for fcc_folder. An example is user_pref("mail.identity.id2.fcc_folder", "imap://[e-mail]/Sent");, which points to [e-mail].

The bug I fixed was username/hostname had been changed before, so [e-mail] didn't match any account.

So I was able to check in my prefs.js. Anything specific I am supposed to see there?

I saw that 102.0b7 got built so I bumped up to it. 1 of my 3 gmail accounts still wants to use the "(O) Other" setting for Sent and Drafts even after I set them to "Sent Folder On:" and "Drafts Folder On:" It just reverts back to "(O) Other" on TB restart.

Comment 24

[Ping Chen [:rnons] (away)]

Check the fcc_folder pref before and after restarting TB. As in the example, user_pref("mail.identity.id2.fcc_folder", "imap://[e-mail]/Sent"), fcc_folder contains server username and hostname, search for '.userName' and '.hostname', you should find something like

user_pref("mail.server.server2.userName", "test");
user_pref("mail.server.server2.hostname", "imap.localhost");

You can even construct and edit the fcc_folder string manually with the username/hostname. If the fcc_folder pref matches any configured server, I don't see why it will be reverted to Local Folders.

Comment 25

[Wayne Mery (:wsmwk)]

Perhaps this resolved an issue I was seeing in an account where I renamed [e-mail] to [e-mail] and account settings have "Place a copy in .... Inbox".

My replies to certain messages resulted in two copies of the reply in Inbox.

With 102.0b7 it works correctly, a reply results in only one message is saved in Inbox.

Comment 26

[David McDivitt]

I originally reported this bug. Today I sent an email and it did not appear in the sent folder of the account used to send it. My emails had been copying there after reading Arthur K's reply and changing the account configuration, which was spoiled with the 102 beta update with overwrite.

When sending email from the account I use four different from addresses. These all come into the same POP3 mailbox. I use one from address mainly with occasional use of the others. Today was the first time I used one of the others, after upgrading to 102 beta and after reporting this bug.

I looked in the local folders / sent folder, found it there, then moved where it was supposed to go.

There is no configuration change I can make to affect this. The only difference is, I used I different, previously stored, identity, sent an email with the same account, and it did not go into the sent folder for the account.

Comment 27

[Magnus Melin [:mkmelin]]

David, since you ran Thunderbird and the migration already happened for your profile (without the now fixed part) it would not run the (corrected) migration ever again, but would have to be manually corrected.