1777683 - Reply to all excludes initial Cc: recipients if only S/MIME is used

Status: RESOLVED FIXED

(Thunderbird :: Message Compose Window, defect, P1)

Description

[Laurent PELLISSIER]

Steps to reproduce:

When I reply to all to a mail with both To: and Cc: recipients compose Windows forget originals Cc: recipients. Only To: initial recipients are copied by the compose message windows.
This problem occurs for the first time on version 102 with Linux and Windows versions.

Comment 1

[Ping Chen [:rnons] (away)]

Seems to work for me. Can you upload a .eml file so we can try to reproduce? Remove any sensitive information from the .eml file before uploading.

Comment 2

[[e-mail]]

Same problem for me. Reply All only replies to the original TO: but not the CC: recipients.
Reply All on an email with hundreds of CC: recipients caused TB 102 to freeze.

Comment 3

[Thomas D. (:thomas8)]

wfm on 102.0 (64-bit), Win10 (tested with two recipients in To and CC, they correctly remain in those fields for the reply-all).

Comment 4

[Wayne Mery (:wsmwk)]

Also https://www.reddit.com/r/Thunderbird/comments/vrauch/reply_all_doesnt_seem_to_work_with_v102/ and bug 1778084. And I'm sure more.

recent bugs https://mzl.la/3yKjTmu

Comment 5

[[e-mail]]

Also, right-clicking on an address and selecting Compose Message To does not work.

In a clean TB profile, Reply All works for me.

Comment 6

[Laurent PELLISSIER]

My two instances of TB (on Linux and Windows) have a long history of several years of updates. For Linux, starting with a tgz initial install (not RPM) .

On Linux, I've created a new profile (thunderbird --ProfileManager) and start with the new profile. Cc: list from initial mail is copied to Cc: field when Reply to all. I then get back to default profile and Cc: recipients are still forgotten when Reply to all !
Same steps on Windows 10 produces the same result.

Version : Thunderbird 102.0.

Comment 7

[Robert]

Confirm, Reply All (a massive unattended upgrade today to TB v102 (x64 , Win Pro 8.1 - 10 (21H2) ) and lot of user start report that issue...

Comment 9

[[e-mail]]

This bug only happens with my primary identity. When Replying All to an email sent to a secondary identity, the CC fields are correctly populated. Both identities have identical settings -- except for the email address.

Other related failures:

1. Setting an automatic CC for emails does not work for my primary identity. The CC field is shown, but empty.
2. Plugin Nostalgy++: option "Always show CC line in compose" does not work anymore.

Version: TB 102.0.1

Comment 10

[Markus Burkert]

Same for me with Version 102.0.1 EN-US language
But this does not happen to all messages.
I have several email accounts (all IMAP)
It leaves out all recipients which were in CC: for example in my primary account (outlook.com) and in my 2nd account (GMX.NET)
But in my 3rd account (Gmail) the recipients which were in CC: are put into the to: field.

An other important point:
If I press CTRL-E in any accounts send folder to edit and re-send any email all fields (to: cc: and BCC:) are empty.

Comment 11

[Adriano Santoni]

I have the same problem as reported by Laurent.

Comment 12

[Rosario Esposito]

Same problem on 102.0.1. Very annoying.

Comment 13

[Thomas D. (:thomas8)]

Bug 1778048 also reports same issue.

Comment 14

[Hallon]

I have the same problem on 102.01.

Comment 15

[ykyri]

I have the same problem on windows 11 thunderbird version 102 & 102.01

Comment 16

[Thomas D. (:thomas8)]

Confirming from plenty reports.

Comment 17

[Richard Shewmaker]

Same problem, only To address copies to Reply All emails. I'm on Windows 11, Thunderbird 102.0.1

Comment 18

[newsfan]

Here Cc: addresses are maintained in the reply, apart from your own identity. Example:
From: [e-mail]
To: [e-mail]
Cc: [e-mail], [e-mail].
If you reply, only from, to and Cc other are included in the reply. There's no point sending yourself the mail. So original e-mail with 4 participants, reply with 3 recipients, plus yourself ([e-mail]) as from.

From comment #10:

If I press CTRL-E in any accounts send folder to edit and re-send any email all fields (to: cc: and BCC:) are empty.

Bug 1774991. You've set up OpenPGP?

Comment 19

[Thomas D. (:thomas8)]

I've tried harder to reproduce this, incl. on gmx and on default identity, and using different entry points to Reply-all (header button, message list context menu, main toolbar customization), but to no avail (102.0.1 (64-bit), Win10).
Note that if the original message was received on your own account via cc your-email, and you do reply-all in that account, your-email will not appear in CC because that address of yours is already the sender. I think that's by design, but otoh there's bug 1211512.

For those who can reproduce: Does this bug also occur in ≡ > Help > Troubleshoot Mode…?

Comment 20

[ykyri]

(In reply to Thomas D. (:thomas8) from comment #19)

I've tried harder to reproduce this, incl. on gmx and on default identity, and using different entry points to Reply-all (header button, message list context menu, main toolbar customization), but to no avail (102.0.1 (64-bit), Win10).
Note that if the original message was received on your own account via cc your-email, and you do reply-all in that account, your-email will not appear in CC because that address of yours is already the sender. I think that's by design, but otoh there's bug 1211512.

For those who can reproduce: Does this bug also occur in ≡ > Help > Troubleshoot Mode…?

Yes the problem remain in Troubleshoot Mode

Comment 21

[Richard Shewmaker]

Yes, it occurs in Troubleshoot Mode. Using the toolbar icon to Reply All. TB 102.0.1. Windows 11.

Comment 22

[newsfan]

Attachment: test.eml

What happens when you download this message and import it into a folder by dragging it in. What does "Reply All" do? And how about attaching a message where the function doesn't work. It just needs to be a simple message and you can obfuscate/remove personal details.

Comment 23

[Thomas D. (:thomas8)]

I'll collect what we know in the user story on top of this bug.
Please note the following false positives:

Caveats

• if the only missing CC address is one of your own addresses, this may be by design if that address is also the "From" address of your reply (I think that's bug 1211512).
• if the only missing CC address is one of your Auto-CC addresses, and it's also the Auto-CC address of another one of your accounts, and you're switching sender identities (From dropdown), you might be seeing bug 1699159.

Comment 24

[Thomas D. (:thomas8)]

Alice, could you try find the regression range?

• use an existing TB 91 or older profile which was updated to 102
• Preferably do not use the default sender account of your installation, but some secondary account

Comment 25

[Alice0775 White]

(In reply to Thomas D. (:thomas8) from comment #24)

Alice, could you try find the regression range?

• use an existing account which was updated to 102
• Preferably do not use the default sender account of your installation, but some secondary account

Sorry, I do not understand str.

Comment 26

[Thomas D. (:thomas8)]

No dataloss or stability issue in the strict sense, but Wayne and I tend to agree that this looks like a high-impact deal-breaker for lots of users -> P1 (blocking).

• highly error-prone (not noticing missing CC's in Reply-all may have serious real-life repercussions, especially for enterprise)
• very cumbersome to work around (sadly we don't support selecting and copying multiple recipients from message header in message reader, that's popular bug 167010)

Still trying to figure out reproducable steps. Sancus suspected a connection with bug 1777990.

Comment 28

[duckzill0r]

(In reply to newsfan from comment #22)

Created attachment 9284764 [details]
test.eml

What happens when you download this message and import it into a folder by dragging it in. What does "Reply All" do? And how about attaching a message where the function doesn't work. It just needs to be a simple message and you can obfuscate/remove personal details.

Not the original reporter, but: my Thunderbird is also failing at adding CCs correctly so - yes, the bug happens also with this test message being imported, at least on my machine.

Comment 29

[Paul Menzel]

Any hints, what function should be stepped through with a debugger to see where it fails?

Comment 30

[Andrei Hajdukewycz [:sancus]]

For those comfortable using it, it would be helpful to have a regression range from mozregression.
Download: https://github.com/mozilla/mozregression/releases
Tutorial: https://www.youtube.com/watch?v=IwrWot3jVFI

You can set it to clone one of your profiles to a TEMP file(use the clone-first setting, clone makes a new copy for every version run, which will be slow for large profiles) that is experiencing the bug.

Set the version range to something like 91 to current Daily: https://i.imgur.com/gy4vyll.png

And then you should be able to narrow down the version by telling mozregression which version it loads is Good or Bad.

To simplify the reproduction step, I would stick an email that you know triggers the bug in a folder by itself, so you can quickly open it and check after mozregression loads each version.

We will have a more detailed, Thunderbird-specific tutorial made in the future. To use this successfully you'll need to be comfortable manually finding your profile folder and using a tool that is a bit technical.

Comment 31

[newsfan]

(In reply to Andrei Hajdukewycz [:sancus] from comment #30)

For those comfortable using it, it would be helpful to have a regression range from mozregression.

Exactly. Comment #28 is from a user who can reproduce the issue with the test message in attachment 9284764 [details] or their own messages. This message, or any other causing the issue, could be imported into a copy of a profile showing the issue, and then mozregression could be used on this fixed profile (which is one of the options of mozregression). It's not really advisable to use mozregression on a production profile, hence the suggestion to copy it.

Comment 32

[duckzill0r]

I just ran mozregression GUI and tried to narrow down the problem.
I'll upload a screenshot with the progress view. Please let me know if you need me to provide any detailed logs of mozregression.
(I am just unsure how to attach the file directly into comments, so I'll upload after this comment. sorry)

Comment 33

[duckzill0r]

Attachment: Screenshot_mozreg_tb.png

Comment 34

[newsfan]

Thanks for running that, but usually the right side of the panel will point to the exact changeset that caused the regression. From the screenshot we can see that 2022-06-12 was still good and 2022-06-13 was bad.

Comment 35

[duckzill0r]

app_name: thunderbird
build_date: 2022-06-13
build_file: C:\Users\.........\.mozilla\mozregression\persist\2022-06-13--comm-central--thunderbird-103.0a1.en-US.win64.zip
build_type: nightly
build_url: https://archive.mozilla.org/pub/thunderbird/nightly/2022/06/2022-06-13-09-59-54-comm-central/thunderbird-103.0a1.en-US.win64.zip
changeset: 1a62e2afc99432552fd022db030584a988a4411b
pushlog_url: https://hg.mozilla.org/comm-central/pushloghtml?fromchange=c17365dc7e4e34f901c18b54c115e3f788051db3&tochange=1a62e2afc99432552fd022db030584a988a4411b
repo_name: comm-central
repo_url: https://hg.mozilla.org/comm-central

the output for the 2022-06-13 build.

Comment 36

[newsfan]

https://hg.mozilla.org/comm-central/pushloghtml?startdate=2022-06-11&enddate=2022-06-14
It's likely another side effect of bug 1771122 which landed before the Daily on 2022-06-13.
Between c17365dc7e4e34f901c18b54c115e3f788051db3 and 1a62e2afc99432552fd022db030584a988a4411b is what your screenshot is suggesting.
Not a surprise since bug 1771122 has already caused bug 1774991. The latter one is fixed in 103.0b3 and 102.0.2. So does the issue still happen with those versions? Download them here:
Beta: https://www.thunderbird.net/en-US/thunderbird/beta/all/
ESR: https://archive.mozilla.org/pub/thunderbird/candidates/102.0.2-candidates/build1/ - release candidates.

Comment 37

[newsfan]

There was a "mid-air" collision of comments, anyway, you just confirmed the range:
https://hg.mozilla.org/comm-central/pushloghtml?fromchange=c17365dc7e4e34f901c18b54c115e3f788051db3&tochange=1a62e2afc99432552fd022db030584a988a4411b
pointing to bug 1771122. Please try the said beta or ERS version to see whether this is fixed now or needs further work.

Comment 38

[duckzill0r]

(In reply to newsfan from comment #37)

There was a "mid-air" collision of comments, anyway, you just confirmed the range:
https://hg.mozilla.org/comm-central/pushloghtml?fromchange=c17365dc7e4e34f901c18b54c115e3f788051db3&tochange=1a62e2afc99432552fd022db030584a988a4411b
pointing to bug 1771122. Please try the said beta or ERS version to see whether this is fixed now or needs further work.

I tested against 103.0b4 : problem still persists there.

Comment 39

[duckzill0r]

Looking at the code from this changeset (which I am not familiar with) I am starting to think it's somehow connected with E2E-encryption (S/MIME and/or OpenPGP) enabled on those accounts/identities affected. On all accounts I could reproduce the issue E2E-encryption was enabled.

So I disabled E2E on those accounts affected and the bug disappeared. So it seems like it somehow discarded the information while checking if E2E is enabled for some addresses.
Also disabling E2E fixes the mailto-link problem for my accounts affected from bug 1777990 which is (assumingly) caused by that same change.

Comment 40

[newsfan]

Excellent work, duckzill0r, here and in bug 1777990. Yes, related to the OpenPGP changes in bug 1771122 and apparently only affecting users with OpenPGP set up.

Comment 41

[Andrei Hajdukewycz [:sancus]]

Great work guys. It definitely seems like a regression caused by bug 1771122. So I'm going to assign this one to Kai.

Comment 42

[Marcin Szczesny]

It also fixes bug 1778898.

Comment 43

[ykyri]

Maybe the problem is also related with the bug 1778727

Comment 44

[ykyri]

It also fixes bug 1778727

Comment 47

[Wayne Mery (:wsmwk)]

Thanks all for working this out on the weekend!

Will this come with automated tests?

Comment 48

[[e-mail]]

I can confirm that removing my name for both S/MIME certificates in the E2E settings fixes this issue. I had not set up E2E encryption in any way.

Comment 50

[Kai Engert [:KaiE:]]

I cannot reproduce yet, maybe I misunderstood what I need to do?

I'm using 102.0.1
I have an email account with OpenPGP configured.
I have a received email, which has 5 recipients on TO, and other 5 recipients on CC.
I click "reply all" on that email.

A compose window opens, and I see the expected 5 entries in TO and the expected 5 other entries in CC.

Comment 51

[Kai Engert [:KaiE:]]

(In reply to [e-mail] from comment #48)

I can confirm that removing my name for both S/MIME certificates in the E2E settings fixes this issue. I had not set up E2E encryption in any way.

I don't understand, can you please explain the meaning of "removing my name for both S/MIME certificates"

Comment 52

[Marcin Szczesny]

(In reply to Kai Engert (:KaiE:) from comment #51)

(In reply to [e-mail] from comment #48)

I can confirm that removing my name for both S/MIME certificates in the E2E settings fixes this issue. I had not set up E2E encryption in any way.

I don't understand, can you please explain the meaning of "removing my name for both S/MIME certificates"

Hi Kai,

in my case, when we use s/mime certificates (values assigned in fields: personal certificate for digital signing and personal certificate for encryption) we face following issues:

1. we send data from financial system via mailto protocol. TB open new window for composing email. Attachment and subject fields are correctly included in new email window but the field "to" is not being populated with recipient address.
2. we have Replay-to Address setup in account configuration. This value is not being transferred to new email window. It doesn't matter if we use create new message or use mailto protocol.

My comment: it seams that there is an issue with email header fields transferred to new message window when s/mime or PGP functionality is active. When I remove s/mime certificates above issues are fixed. Of course digital signing functionality is missing then.

Please let me know if I can provide any additional info.

Br,
Marcin

Comment 53

[Kai Engert [:KaiE:]]

Thanks for clarifying that you fixed it by clearing the s/mime configuration.

In the meantime, I'm able to reproduce the bug myself.
The bug is seen, if S/MIME is the ONLY e2ee technology configured in account settings.

If both OpenPGP and S/MIME are configured -> works fine
If only OpenPGP is configured -> works fine
If only S/MIME is configured -> this bug

Comment 54

[Marcin Szczesny]

Please also be informed that we experience different behaviour of the issues depending on OS.
For hosts with Windows 11 issues 1 & 2 applies.
For hosts with Windows 10 only issue 2 apply.

Comment 55

[Kai Engert [:KaiE:]]

This has a similar cause as bug 1774991, however, the fix for bug 1774991 was insufficient to fix this one here, too.

Originally in bug 1774991, I had suggested a broader fix - I had suggested to completey skip processing of function checkRecipientKeys() while the composer window is still loading. (We didn't do that, but had opted for a different fix.)

Now I'm testing this bug here, and the fix I had originally suggested fixes this bug.

That original fix had already been r+ed by darktrojan:
https://phabricator.services.mozilla.com/D149833#4925141

I'm re-requesting approval for that change, because I'm asking for an additional change on top:
Once loading is complete, we must perform the check.

I've tested the various described scenarios in which attributes in the composer window are missing, and they all appear to be fixed by the patch that I'll attach.

(I didn't test yet clicking a mailto link.)

Comment 56

[Kai Engert [:KaiE:]]

Attachment: Bug 1777683 - While still loading composer, don't check encryption recipient status. r=darktrojan

Comment 57

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/bf171cfffec9
While still loading composer, don't check encryption recipient status. r=darktrojan

Comment 58

[Martin Pecka]

Thanks for fixing this annoying bug! I see it has received thunderbird-102 wontfix. I don't really know the release process of TB. I hope it doesn't mean that if I'm using release channel, the fix won't appear in the 102 stable release channel versions. Or does it?

Comment 59

[Andrei Hajdukewycz [:sancus]]

(In reply to Martin Pecka from comment #58)

Thanks for fixing this annoying bug! I see it has received thunderbird-102 wontfix. I don't really know the release process of TB. I hope it doesn't mean that if I'm using release channel, the fix won't appear in the 102 stable release channel versions. Or does it?

thunderbird_esr102 is the release branch. This fix will get there after it's been tested on daily and beta for a bit.

Comment 60

[Andrei Hajdukewycz [:sancus]]

Comment on attachment 9284920 [details]
Bug 1777683 - While still loading composer, don't check encryption recipient status. r=darktrojan

[Approval Request Comment]
Regression caused by (bug #): 1771122

User impact if declined:
Users configured with SMIME only will see Compose windows fail to fill various fields that they should any time a pre-filled Compose window is created.

Testing completed (on c-c, etc.): c-c

Risk to taking this patch (and alternatives if risky):
The patch seems relatively small and targeteed. However there is some possibility it causes some new issue with the Compose window for users with E2E configured. It seems very unlikely to cause any new issue for other users.

Comment 61

[Paul Menzel]

Can you provide an easy way for distributions currently shipping Thunderbird 102.x to pick this up?

Comment 62

[Andrei Hajdukewycz [:sancus]]

(In reply to Paul Menzel from comment #61)

Can you provide an easy way for distributions currently shipping Thunderbird 102.x to pick this up?

The easy way is for them to update to 102.0.3 or the following release, whichever this ends up going into. There really isn't any other mechanism. We're already moving major 102 fixes along as quickly as is safely possible.

Comment 63

[Kai Engert [:KaiE:]]

Here's a 102.x try build with the patch:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=25666e4f79bd52fa03d01bb396ea7607aff571d8

Comment 64

[Wayne Mery (:wsmwk)]

Comment on attachment 9284920 [details]
Bug 1777683 - While still loading composer, don't check encryption recipient status. r=darktrojan

[Triage Comment]
Approved for beta

Comment 65

[Rob Lemley [:rjl]]

Thunderbird 103.0b5:
https://hg.mozilla.org/releases/comm-beta/rev/76d3459c0163

Comment 66

[Farid Wagner]

After initial testing of 103.0b5 on Win 10 21H2 64 (S/MIME) I can confirm that the issue no longer exists for us. Is there any ETA on when we will have the official 103 release?

Comment 67

[Kai Engert [:KaiE:]]

We will fix the bug in an upcoming 102.x release, soon.

There will be no new major release until summer 2023. Versions 103+ will only remain as beta versions.

Comment 68

[Farid Wagner]

I understand. Thanks for the quick response and hard work.

Comment 69

[Andrei Hajdukewycz [:sancus]]

Unless there is some unforeseen new related regression on Beta this week, this one will go into 102.0.3 which should build early next week.

Comment 70

[Kai Engert [:KaiE:]]

I didn't write a new approval comment, I agree with the one Andrei had written for beta.

Comment 74

[Wayne Mery (:wsmwk)]

Comment on attachment 9284920 [details]
Bug 1777683 - While still loading composer, don't check encryption recipient status. r=darktrojan

[Triage Comment]
Approved for esr102

Comment 75

[Rob Lemley [:rjl]]

Thunderbird 102.0.3:
https://hg.mozilla.org/releases/comm-esr102/rev/608f04d334e7