1958781 - If the folder for deleted messages contains a German umlaut (e.g. mail.server.server1.trash_folder_name Gelöscht), it will be removed in Server Settings. Emails are irrevocably deleted when you delete them

Status: VERIFIED FIXED

(MailNews Core :: Networking: IMAP, defect, P1)

Description

[Salvo]

Attachment: Screenshot Thunderbird.png

Steps to reproduce:

Assumption: You already have an IMAP email account configured in Thunderbird.

1. Start Thunderbird
2. Create a new folder named "Gelöscht" (which means deleted in German) (also works with any folder name, which includes either ä, ö, ü, ß)
3. Click on the name of your email account and then on "Account Settings" on the top right
4. Click on "Server Settings"
5. In the Server Settings area, set the option "When I delete a message" to "move it to this folder" then choose the name of the new folder "Gelöscht"
6. Click on the tab of your email account to get back to the folder overview
7. Click on the down arrow icon next to the account name to collapse the folders
8. Click on the changed right arrow icon next to the account name to expand the folders
9. Click on the "Account Settings" tab to get back to the account settings overview
10. Click on the left on e.g. Copys & Folders or any other option
11. Click on "Server Settings"
12. Check the setting "When I delete a message" and "move it to this folder", the folder is unset and "Choose folder..." is set

Actual results:

You see, that in the Server Settings area, the option "When I delete a message" and "move it to this folder" is no more set to your folder "Gelöscht", but it is unset to "Choose folder".

When you delete an email e.g. in your inbox, the email is immediately and irrevocably deleted. It is not moved to your folder named "Gelöscht.

Expected results:

The folder in "When I delete a message" and "move it to this folder" should still remain to "Gelöscht" and not removed. When you delete an email it should still be moved to this folder.

Comment 1

[Magnus Melin [:mkmelin]]

I can confirm that. It does work properly in v128.
Can we get a regression range?

Comment 2

[Alice0775 White]

Seems works for me on Daily139.0a1 (2025-04-07) (64-bit) en-US build, Windows11 Home.
Tested gmail and hotmail.

Comment 3

[Magnus Melin [:mkmelin]]

My tests were on a dovecot server.

Comment 4

[Salvo]

I tested it again. Thunderbird ESR 128.9.1esr (64-Bit) is unaffected. But Thunderbird Release 137.0.1 (64-bit), Thunderbird Beta 138.0b1 (64-bit) and Thunderbird Daily 139.0a1 (2025-04-08) are affected. Tested with gmx and my own domain. I always used the en-US build and for Thunderbird Release also the German build, Windows 11 24H2 (Build 26100.3476).

Comment 5

[Wayne Mery (:wsmwk)]

Thanks for the update.

Comment 7

[poncho]

The result from:
mozregression --app thunderbird --profile /home/testing/profile --profile-persistence clone-first

INFO: No more integration revisions, bisection finished.
INFO: Last good revision: 2e4be0842605799703d929a9b39e2163f04d90cf
INFO: First bad revision: bf984d603dcc40f64379dd02ec02abbb1bd5f726

Comment 8

[poncho]

https://hg-edge.mozilla.org/comm-central/pushloghtml?fromchange=2e4be0842605799703d929a9b39e2163f04d90cf&tochange=bf984d603dcc40f64379dd02ec02abbb1bd5f726

My guess would be that it is related to: Bug 1946011 - Use UTF-8 strings for server names and folder names.

Comment 9

[Magnus Melin [:mkmelin]]

Thanks, yes regressed by bug 1946011.

I think these need to use CopyMUTF7toUTF16 instead of NS_ConvertUTF8toUTF16
https://hg-edge.mozilla.org/comm-central/rev/e49801f1255a#l47.99
https://hg-edge.mozilla.org/comm-central/rev/e49801f1255a#l47.482

Eleanor, something to get your feet wet?

Comment 10

[Eleanor Dicharry]

Magnus, I tried to reproduce this using the steps in the report using 138.0b3, 139.0a1 (2025-04-07), and with my local debug build, but I couldn't reproduce. I'm wondering if it's platform related? It looks like the OP was on Windows. I tested on Mac and Linux. Were you testing on Linux?

Comment 11

[poncho]

I'm encountering the issue on linux. The symptoms are slightly different (see bug 1960546) than what what the OP describes here.

While regression testing I was looking whether the right click menu of the Gelöscht folder shows an "empty trash" item or just the regular menu for a generic folder. For me, this corresponds to whether the folder Gelöscht is working or not.

Comment 12

[Eleanor Dicharry]

It looks like this bug does not happen for all IMAP servers. I does not happen on my gmail hosted accounts, but I created a GMX account, and I can observe this behavior now when using that account.

Comment 13

[Eleanor Dicharry]

Attachment: Bug 1958781 - Fix MUTF-7/UTF-8 comparison of trash folder name. r=mkmelin,#thunderbird-back-end-reviewers

Comment 14

[poncho]

Patch fixes the issue for me. Tested on top of 137.0.2 running on linux

Comment 15

[Eleanor Dicharry]

Try Run: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=1b6263624df72298e297887e406fb7df3b4fa28a

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/28ebb8877b66
Fix MUTF-7/UTF-8 comparison of trash folder name. r=mkmelin

Comment 17

[Corey Bryant]

We should consider uplifting this to 138.0.1

Comment 18

[Magnus Melin [:mkmelin]]

Yes we should. Eleanor, I sent you some instructions on matrix.

Comment 20

[Eleanor Dicharry]

Comment on attachment 9480087 [details]
Bug 1958781 - Fix MUTF-7/UTF-8 comparison of trash folder name. r=mkmelin,#thunderbird-back-end-reviewers

Uplift Approval Request

• Please state case for uplift consideration and ensure bug severity is set: This is a fix for an issue that results in a permanent data loss under certain circumstances and when the user's mail server uses standard IMAP MUTF-7 encoding.
• User impact if declined: The issue results in a data loss when a user is expecting deleted items to be moved to their trash folder in the case where a user's trash folder contains non-ascii unicode characters.
• Is this code covered by automated tests?: No
• Has the fix been verified in Daily?: Yes
• Has the fix been verified in Beta?: N/A
• Needs manual test from QA?: Yes
• If yes, steps to reproduce: 0. This bug is reproducible only on servers that use IMAP MUTF-7 encoding. In particular, it is not reproducible on GMail or Hotmail. It is, however, reproducible on at least accounts hosted at least on GMX and Dovecot, and any others that are using standard IMAP MUTF-7.

1. Create a new folder called "Gelöscht" (German for "deleted"). Words containing other characters, including ä, ö, ü, ß can cause it to fail as well.
2. Go to account settings -> server settings and set the "When I delete a message:" option to move deleted items to the newly created folder.
3. Go back to the email account.
4. Go back to account settings.
5. Click away from Server Settings to any other settings item (such as Copies & folders).
6. Go back to Server Settings.
7. Observe the folder name under Move it to this folder: has changed back to the default Trash name.
8. Delete a message.
9. Observe that it does not appear in either Trash or Gelöscht .

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The fix queries the server's character encoding before deciding what to do. In the case of UTF-8, the trash folder name is stored without modification. In the case of MUTF-7, the folder name is converted from UTF-8 -> UTF-16 -> MUTF-7, so the conversion is not lossy.
• String changes made/needed: N/A

Comment 21

[Toby Pilling [:tobyp]]

Could you please take a look at this once it is merged into beta (on Tuesday)?

Comment 22

[Ramona Gavrilescu]

Hello!

I have tested this issue on TB 139.0b1(20250429124621) on Ubuntu 24.04.1 LTS.
I have created a GMX account, I created a new folder using the characters specified on the bug report, after executing the exact STR from comment 20 (step 1 to 5) going back to Server Settings, the “Move it to this folder” setting did not change to the default Trash name folder, it remained the Folder(Gelöscht) I created in previous steps. I then went to inbox folder and deleted some emails. The emails deleted appeared in the folder created.

I have tested this issue also on Win 11 and macOs 14.7.5 and the same behaviour occurs.

Comment 23

[Magnus Melin [:mkmelin]]

-> VERIFIED then (please set that when you do QA and can verify it works)

Comment 24

[Corey Bryant]

Comment on attachment 9480087 [details]
Bug 1958781 - Fix MUTF-7/UTF-8 comparison of trash folder name. r=mkmelin,#thunderbird-back-end-reviewers

[Triage Comment]
Approved for release

Comment 25

[Daniel Darnell [:dandarnell]]

Thunderbird 138.0.1:
https://hg.mozilla.org/releases/comm-release/rev/8a4253a84594