528039 - Reply to List not correctly identifying lists

Status: VERIFIED FIXED

(Thunderbird :: Mail Window Front End, defect)

Description

[Matthew Knowlton]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b3pre) Gecko/20091111 Namoroka/3.6b3pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090915 Thunderbird/3.0b4

When I hit reply to list on a Google Groups mailing list that I am a member of, it is not correctly identifying the list and is instead mailing the person that it is from. It used to correctly do this.

Reproducible: Always

Steps to Reproduce:
1. Join a Google Groups Mailing List
2. Have someone send an e-mail to it
3. Try and reply to their e-mail using "Reply To List"

Actual Results:  
instead of sending to the list it sends to the person.

Expected Results:  
Replies to the list's address.

Comment 1

[Nathan Tuggy (:tuggyne)]

What's more, Reply [to Sender] replies to the Google Groups list.

Reproduced on Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091112 Lightning/1.0pre Thunderbird/3.0 ID:20091112084939

Comment 2

[John Wilcock]

This change in behaviour occurs because Google Groups mailing list messages no longer include a Sender: header.

In the absence of a Sender:, Reply to list goes to the From: 
address (i.e. the original sender), whereas ordinary Reply goes to the 
Reply-To: address (i.e. the list)!

Does anyone have a contact at Google who could be asked to explain why they are ignoring RFC 5322 Section 3.6.2, which states: 
If the originator of the message can be indicated by a single mailbox and the author and transmitter are identical, the "Sender:" field SHOULD NOT be used. Otherwise, both fields SHOULD appear.

Notwithstanding, TB could be a bit more intelligent and, for example, compare the From and Reply-To with the Mailing-list: header in order to help detect poorly configured mailing list servers

Comment 3

[Martin Häcker]

I can confirm this on Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.5) Gecko/20091123 Lightning/1.0pre Shredder/3.0.1pre.

Exact same behaviour (also there if I disable all plugins).

Comment 4

[John Wilcock]

Feedback to google groups about this via their feedback forum appears to have been ignored. Anyone have a suitable direct contact address?

Google groups being one of the largest mailing list hosts out there, it would be nice to see TB3 behaving as expected - preferably by google restoring RFC-compliant Sender: headers. 

In an ideal world this would be a WONTFIX, but setting as NEW anyway, in that Thunderbird's behaviour could be improved in the absence of the Sender: header (perhaps by looking at the Mailing-List: or List-Post: headers).

Comment 5

[Matthew Knowlton]

Yeah, in an ideal world all mailing lists would comply with standards, but this will never be the case, so we shouldn't rely on just the Sender header being there, there are a couple other headers I've noticed in messages from Google mailing lists that identify the address of the mailing list properly.

Comment 6

[Martin Häcker]

TB3 is now final and this is one big issue for me. How to proceed from here?

Could somebody build a plugin that acts as a temporary fix till the next release of Thunderbird?

Comment 7

[John Wilcock]

FWIW, Google Groups seem to have restored the Sender: header on their mailing lists, removing the biggest source of problems covered by this bug. There are other cases too, of course, such as that mentioned in bug 524277.

Comment 8

[Patrick Reiter Horn]

TB 3.0 is still replying to the original author on google groups, even with the Sender header present and pointing to the mailing list.

The Sender, Reply-To and List-Post headers are all set to the mailing list, yet "reply list" appears to bypass all of those and go directly to what is specified in "From", or something (I don't know where else it would be getting the address from...)

I understand that Google's software is broken, but I would appreciate graying out or removing the "reply-list" button if Thunderbird doesn't have enough information, since I keep making the mistake of clicking the broken button.

Unless I am misunderstanding the RFC, the "From" and "Return-Path" headers are never correct for this purpose (unless there is some widely used mailing list software that does this). If you want to keep the "reply" button, maybe you could rename the "reply list" button to say "reply author", since it seems to work for that purpose.

Comment 9

[Mark Sapiro]

I disagree that Google is broken. Reply-list should have nothing to do with a Sender: header and in fact doesn't seem to.

Mailman lists have a Sender: of LISTNAME-bounces@..., yet reply-list to a Mailman list post is correctly addressed to LISTNAME@...

Reply-list should be offered only if a List-Post: header is present in the message and that appears to be the case. Reply-list should be directed as specified in the List-Post: header. This appears to be the source of the problem here.

The issue appears to be Google's List-Post: header which is of the following form:

List-Post: <http://groups.google.com/group/GROUPNAME/post?hl=>, 
	<mailto:[e-mail]>

It appears that TB sees the http: URL and ignores it and falls back to an ordinary reply to author.

RFC 2369 says

   A list of multiple, alternate, URLs MAY be specified by a comma-
   separated list of angle-bracket enclosed URLs. The URLs have order of
   preference from left to right. The client application should use the
   left most protocol that it supports, or knows how to access by a
   separate application.

TB should either ignore the http: URL and use the mailto: URL or invoke a web browser with the http: URL.

As an aside, I would much prefer TB to use the mailto: since the http: URL includes no subject or threading information and makes quoting more difficult. In this sense, I agree that Google is broken in that the mailto: URL is preferred and should be first.

Comment 12

[Patrick Reiter Horn]

Attachment: Example of a mailing list message with faulty "reply list"

Clicking "reply" goes to the list, and "reply list" goes to the original sender.

This occurs for any Google Groups messages after around 2009-11-17, 14:00 PST. Can anyone else who is on Google Groups confirm this?

Comment 13

[Patrick Reiter Horn]

Attachment: Working Google Groups email before November 17

From the same list as the broken email, but before the change on Nov. 17.

In this email, both "reply" and "reply list" go to the mailing list. Not the best behavior but much better then having the buttons swapped.

This mailing list has "Reply-To" set to go to the mailing list, so in my opinion, this is the appropriate way to handle mailing lists. Would be better if thunderbird could detect that both email addresses are equal, and *only* present a "reply list" button in that case.

Comment 14

[David Ascher (:davida)]

Google Groups recently changed their .sig separator -- maybe they also changed their headers?

Comment 15

[Bryan Clark (DevTools PM) [:clarkbw]]

We saw this in raindrop as well, Google Groups went from a single item (mailto) for most of their list headers into multiple items (mailto and http).  I'm not sure how we're processing these headers in Thunderbird but we could probably be looking for the mailto header and just drop the rest.

The RFC allows for multiple values for most of the list headers with descriptions.

e.g. List-X: <mailto:address> (description), <http:address> (other description), <ftp:address> (ftp description)

In Raindrop we create an array based off the comma and pass it into an email address utils function i.e. headers.get("list-unsubscribe").split(",")

In general however none of these headers are actually required, only suggested, and it should be noted that List-Post can have a possible value of "NO" which means posting is not allowed.

Comment 16

[Blake Winton (:bwinton) (:☕️)]

Attachment: A patch to fix the reply-to-list functionality.

Okay, here's what I've found.

For the listPost value:
<http://groups.google.com/group/ucosp/post?hl=>, 
	<mailto:[e-mail]>
The calls to:
PRInt32 startPos = listPost.Find("<mailto:");
PRInt32 endPos = listPost.Find(">", startPos);
were returning:
50,46

Looking up the signature for nsAutoString.Find, we see that it takes:
PRInt32 Find( const nsCString& aString, PRBool aIgnoreCase=PR_FALSE, PRInt32 aOffset=0, PRInt32 aCount=-1 );
Sadly, our startPos is being cast to a PRBool

Comment 17

[Blake Winton (:bwinton) (:☕️)]

(Dammit, wrong key.)

Anyways, after we add the PR_FALSE there, we get a start and end pos of:
50,80
which looks way better to me, and it also makes it all seem to work.

Thanks,
Blake.

Comment 18

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 420863 [details] [diff] [review]
A patch to fix the reply-to-list functionality.

Yeah, I know, no unit test.  If you wanted to get this in as is for 3.0.1, I would promise to write a unit test soon.

Thanks,
Blake.

Comment 19

[Phil Ringnalda (:philor)]

Comment on attachment 420863 [details] [diff] [review]
A patch to fix the reply-to-list functionality.

Even though I have the feeling I probably reviewed that and didn't catch it, I'm really not a mailnews/ peer.

Comment 20

[David :Bienvenu]

Comment on attachment 420863 [details] [diff] [review]
A patch to fix the reply-to-list functionality.

stepped through the code; it works for one of my google groups.

Comment 21

[David :Bienvenu]

I've landed this on the trunk

Comment 22

[Blake Winton (:bwinton) (:☕️)]

Landed <http://hg.mozilla.org/comm-central/rev/dae7751b6fa1> on 1.9.1 as <http://hg.mozilla.org/releases/comm-1.9.1/rev/cdd93c2db780>.

Comment 23

[Ludovic Hirlimann [:Usul]]

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.7) Gecko/20100110 Shredder/3.0.1pre

Comment 24

[mcanonic]

Does the patch work also for yahoo groups? Conversely to googlegroups.com (where TB recognizes that it is a list), with yahoogroups.com my TB3.0 seems does provide "reply list" option.

M

P.S.: I'm new in this debug system. "VERIFIED FIXED" means that this fix will be included in the next release?

Comment 25

[Mark Sapiro]

(In reply to comment #24)
> Does the patch work also for yahoo groups? Conversely to googlegroups.com
> (where TB recognizes that it is a list), with yahoogroups.com my TB3.0 seems
> does provide "reply list" option.


Do you mean "does NOT provide"?

Mail I receive from various Yahoo Groups lists does not include a List-Post: header. The only RFC 2369 headers included are List-Id: and List-Unsubscribe:. Therefore, TB cannot provide a Reply List because it doesn't have unequivocal knowledge of the list posting address.


> P.S.: I'm new in this debug system. "VERIFIED FIXED" means that this fix will
> be included in the next release?


The target milestone is 3.1a1. Does that answer your question?

Comment 26

[Blake Winton (:bwinton) (:☕️)]

(In reply to comment #25)
> (In reply to comment #24)
> > Does the patch work also for yahoo groups?
> Mail I receive from various Yahoo Groups lists does not include a List-Post:
> header. The only RFC 2369 headers included are List-Id: and List-Unsubscribe:.
> Therefore, TB cannot provide a Reply List because it doesn't have unequivocal
> knowledge of the list posting address.

I've just emailed Yahoo Groups support, to ask them to add the List-Post header.  If they do so, then the Reply-List button will be enabled for future messages from Yahoo Groups.

> > P.S.: I'm new in this debug system. "VERIFIED FIXED" means that this fix will
> > be included in the next release?
> The target milestone is 3.1a1. Does that answer your question?

Along a different tack, the "status-thunderbird3.0:" flag is set to ".1-fixed", which means that this fix will appear in Thunderbird 3.0.1.  (Which I know, because I checked it in to the Thunderbird 3.0 branch a couple of days ago.  ;)

Thanks,
Blake.

Comment 27

[mcanonic]

Mmmm, it seems that we have problem again. I received email from googlegroups mailing list and both "reply" and "reply list" always use googlegroups email address and never the sender address.

What about you?

Cheers,
 Massimo

Comment 28

[Mark Sapiro]

(In reply to comment #27)
> Mmmm, it seems that we have problem again. I received email from googlegroups
> mailing list and both "reply" and "reply list" always use googlegroups email
> address and never the sender address.


That is not this bug and not a Tbird bug at all. Google groups can be configured to direct replies as follows:

Replies to messages
   Replies are sent to the whole group.
   Replies are sent to the author of the message.
   Replies are sent to the owners of the group.
   Users decide where their replies are sent.

The first three options add a Reply-To: with the corresponding address to the outgoing email. The group from which you received the message is set for "Replies are sent to the whole group." so there is a Reply-To: with the group address. Therefore, "reply" goes to the group because of the Reply-To: and "reply list" goes to the group as it should because of the List-Post: header.