1800759 - In-Reply-To and References header missing in reply to mail since upgrading to 102.4.2 if they were set in mail.compose.other.header

Status: RESOLVED FIXED

(Thunderbird :: Message Compose Window, defect)

Description

[Johannes]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:104.0) Gecko/20100101 Firefox/104.0

Steps to reproduce:

On reply to emails, Thunderbird didn't set References and In-Reply-To header in the answer anymore since I upgraded from Thunderbird 78.14.0 to 102.4.2. This results in breakage of the mail-threading.

I also checked with all AddOns deactivated, same happens.

Actual results:

The sent mail doesn't contain References and In-Reply-To header.

Expected results:

The sent mail should contain References and In-Reply-To header.

Comment 1

[Magnus Melin [:mkmelin]]

Can't reproduce, those are normally set. Try Help | Troubleshoot mode

Comment 2

[Johannes]

I tried Help | Troubleshoot mode with no success.

Comment 3

[Magnus Melin [:mkmelin]]

Are you checking the copy in the Sent folder. Can you attach a sample as .eml?

Comment 4

[Pascal Hambourg]

I observe the same behaviour with 102.5.0 on Debian 11 if mail.compose.other.header contains "References".
Also, when displaying a message, headers listed in mail.compose.other.header are shown with empty value if present.
It did not happen with 102.4.0.

Comment 5

[Johannes]

I can confirm that mail.compose.other.header is causing this. I had

In-Reply-To,References

as value for mail.compose.other.header.
After resetting this pref In-Reply-To and References headers are added again when replying to messages.
Thanks Pascal for figuring that out!

After a restart of Thunderbird, the values of these headers are also shown again and not staying empty anymore - I have both mailnews.headers.showReferences and mailnews.headers.showMessageId set to true.

@Magnus : Yes, I checked the copy of the sent email. I guess, a sample eml is now not needed anymore?

Comment 6

[Magnus Melin [:mkmelin]]

Oh I see. No sample needed.
Why would you have those headers in mail.compose.other.header? That pref is not supposed to contain any standard header.

Comment 7

[Magnus Melin [:mkmelin]]

Attachment: Bug 1800759 - Don't allow mail.compose.other.header to contain References/In-Reply-To and other standard fields. r=#thunderbird-reviewers

Comment 8

[Magnus Melin [:mkmelin]]

I spent some time looking at this, and the patch will ignore it if you set some of the headers we normally show by other means.
But, ultimately setting standard headers in that pref is not allowed.

Comment 9

[Pascal Hambourg]

@Magnus : I set mail.compose.other.header="References,In-Reply-To,Message-ID,Sender" for the following reasons:
I need to set References: and In-Reply-To: values manually when making up a reply to a message found in mailing list or newsgroup web archives.
I need to set Message-ID: and Sender: values manually when cancelling spam or other abuse on Usenet newsgroups.
Please do not ignore these headers.

Comment 10

[Alessandro Vesely]

I too had mail.compose.other.header="References,In-Reply-To". Cannot remember why; was it hinted somewhere?

Comment 11

[max]

(In reply to Magnus Melin [:mkmelin] from comment #6)

Why would you have those headers in mail.compose.other.header? That pref is not supposed to contain any standard header.

I was trying to repair a thread in a mailing list. Some people do not care, some are affected by issues similar to the Bug #1755668. The result is threads split into several parts. I added References to mail.compose.other.header, copied Message-IDs from the References header of another message to my reply. It worked. Unfortunately my next messages were sent with missed references and so out of threads.

Notice that unlike email addresses for To and Co headers, <Message-ID> entries for References should not be separated by comma. It would be great if current value were available for edit when such header is "added" in compose window.

Comment 12

[sebastian]

This hit me too: People were complaining to me that my mails started starting new threads without need or intention to. Adjusting mail.compose.other.header was the workaround that helped, but it was working fine previously.

(In reply to Alessandro Vesely from comment #10)

I too had mail.compose.other.header="References,In-Reply-To". Cannot remember why; was it hinted somewhere?

I blogged about this near 12 years ago: https://blog.hartwork.org/posts/howto-late-join-a-mail-thread-seamlessly-technically/ . Maybe it was a blog post like that one that made you add it.

Comment 13

[sebastian]

PS: Fix https://phabricator.services.mozilla.com/D162680 does not seem like a good direction, because it kills the feature where you want to manually join a mailing list thread were you don't have a copy of the mail to reply to in Thunderbird, i.e. what the blog post above is describing. The fix will break that feature, if I read it correctly.

Comment 14

[max]

(In reply to sebastian from comment #12)

I blogged about this near 12 years ago: https://blog.hartwork.org/posts/howto-late-join-a-mail-thread-seamlessly-technically/ . Maybe it was a blog post like that one that made you add it.

For joining a mail list thread there is a workaround: create a mailto: link having Reply-To and References parameters. Anyway you have to find somewhere Message-ID to respond. It should work for messages available in mailboxes as well, but it is much less natural, to extract values of headers from message source to generate a link to be opened in thunderbird again. Perhaps it is possible to create an add-on to overcome issues described in this report.

Comment 15

[Jochen Bern]

FWIW, in my Sent folder, I can pinpoint the sudden lack of these two headers in my replies to the upgrade from 102.4.0 to 102.4.1 (from Fedora repos, in Nov-2022).

(In reply to max from comment #14)

(In reply to sebastian from comment #12)

I blogged about this near 12 years ago: https://blog.hartwork.org/posts/howto-late-join-a-mail-thread-seamlessly-technically/ . Maybe it was a blog post like that one that made you add it.

For joining a mail list thread there is a workaround: create a mailto: link having Reply-To and References parameters.

I initially added these two headers to the config so as to fix replies - yes, replies to actual e-mails present in TB - where I knew beforehand that they'll need fixing, though I forget the exact circumstances. Making these headers editable is the natural way how TB users will handle a need to fix headers that won't get autogenerated correctly. If we can agree that such needs may legitimately happen, please do not try to force those users through hoops a la "hmmm, gotta write myself a HTML page now, and what was the exact format for that URL again ... ?", that's not fixing the problem but rather taking aim at the messenger.

Comment 16

[Jochen Bern]

I now remember what my use case for manually entering these headers were: Replying to a specific post on a mailinglist that I'm only subscribed to in digest mode (so that replying to the digest would put my reply into a thread with the digest, rather than the mail(s)/subject I'm actually replying to).

Comment 17

[Magnus Melin [:mkmelin]]

Attachment: Bug 1800759 - Fix issues when mail.compose.other.header include standard headers. r=john.bieling

Comment 18

[Axel Grude]

We also need the Received headers for some FiltaQuilla search terms, see FiltaQuilla Issue 243. (It also appears that some x- headers are missing in the filter properties array).

Alternatively, we could also use the web mail extensions to get at all raw headers, but since web extensions are async this is currently not possible to call from the synchronous search terms of a filter. A bug future improvement would be turning both search terms and filter actions async so that we that can start incorporating asynchronous functions like the ones from the wx APIs.

Comment 19

[Magnus Melin [:mkmelin]]

I don't think that has anything to do with this bug.
This bug is just handling the case where someone includes the relevant standard headers in mail.compose.other.header

Comment 20

[Magnus Melin [:mkmelin]]

And like the pref name implies, it's about composition only.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/d8025a360c3a
Fix issues when mail.compose.other.header include standard headers. r=john.bieling