Multiple To and CC headers are not sent when using Long email addresses

VERIFIED FIXED in mozilla1.0.1

Status

--
critical
VERIFIED FIXED
17 years ago
10 years ago

People

(Reporter: ludovic, Assigned: bugzilla)

Tracking

Trunk
mozilla1.0.1

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [adt2 RTM] [ETA 06/25])

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

17 years ago
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415
BuildID:    2002041513

When long email addresses are used (Ludovic Dubost <ludovic@netvalue.com>)
instead of simple email addresses (ludovic@netvalue.com) with multiple To and/or
CC recipient, then the second one of the To and the CC are not added to the
headers.. However the email is sent to all people (so people that are not in the
To or CC receive the email like in a BCC)..

Example emails will follow.. The bug apparently appeared in Mozilla 0.9.8 and is
reproducible in 1.0rc1 on Linux and Windows... The problem does not appear in
Netscape 6.2.1

The email in the sent folder is correctly formated...

Reproducible: Always
Steps to Reproduce:
1. Write an email
2. Add To and CC looking like this:

To: Ludovic Dubost <ludovic@netvalue.com>
To: Ludovic Dubost <ludo@netvalue.com>
CC: Ludovic Dubost <ldubost@pobox.com>
CC: Ludovic Dubost <ludovic.dubost@netvalue.com>

3. Send the email..

Actual Results:  Message received looks like this:

Return-Path: <ludovic@netvalue.com>
Received: from mail-fr.netvalue.fr ([192.168.1.18]) by mail.netvalue.fr
          (Netscape Messaging Server 3.6)  with ESMTP id AAA4A33;
          Tue, 16 Apr 2002 14:04:02 +0200
Received: from netvalue.com ([192.168.1.102]) by
          mail-fr.netvalue.fr (Netscape Messaging Server 4.01) with ESMTP
          id GUNTIP00.OBV; Tue, 16 Apr 2002 14:04:01 +0200 
Message-ID: <3CBC1331.5010606@netvalue.com>
Date: Tue, 16 Apr 2002 14:04:01 +0200
From: "Ludovic Dubost" <ludovic@netvalue.com>
Organization: Netvalue ( http://www.netvalue.com )
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415
X-Accept-Language: fr, en
MIME-Version: 1.0
To: Ludovic Dubost <ludovic@netvalue.com>
CC: Ludovic Dubost <ldubost@pobox.com>
Subject: Test email to show missing headers
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit


Test email to show missing header



Expected Results:  Something like:

Message-ID: <3CBC1331.5010606@netvalue.com>
Date: Tue, 16 Apr 2002 14:04:01 +0200
From: Ludovic Dubost <ludovic@netvalue.com>
Organization: Netvalue ( http://www.netvalue.com )
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415
X-Accept-Language: fr, en
MIME-Version: 1.0
To: Ludovic Dubost <ludovic@netvalue.com>, Ludovic Dubost
 <ludo@netvalue.com>
CC: Ludovic Dubost <ldubost@pobox.com>, Ludovic Dubost
 <ludovic.dubost@netvalue.com>
Subject: Test email to show missing headers
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit


Test email to show missing header




All 4 people have actually received the email..
Look at the To and CC headers in the results:

To: Ludovic Dubost <ludovic@netvalue.com>
CC: Ludovic Dubost <ldubost@pobox.com>

It should be:

To: Ludovic Dubost <ludovic@netvalue.com>, Ludovic Dubost
 <ludo@netvalue.com>
CC: Ludovic Dubost <ldubost@pobox.com>, Ludovic Dubost
 <ludovic.dubost@netvalue.com>

This problem is a corporate stopper as it defeats all filtering and email
exchanged inside the company.. This is actually a problem for me and I will need
to downgrade my version of Mozilla if there is not a solution to this problem...

I consider this is loss of data as people don't receive the email that they
should...
(Reporter)

Comment 1

17 years ago
Correction:

The bug does not appear in Mozilla 0.9.8 and neither in 0.9.6
(Reporter)

Comment 2

17 years ago
I believe this bug should be fixed for 1.0.. So I'm adding the mozilla1.0 keyword..
Keywords: mozilla1.0

Comment 3

17 years ago
rc1 is not out yet.
you're running builds from the rc1 branch.

I tried reproducing this by sending an email adressed to 4 email addressed: 2 TO
and 2 CC and it came ok.

Are you sure this is not a MTA problem?
Can you send it in other clients and receive it properly formatter?
(Reporter)

Comment 4

17 years ago
It only happens with certain versions of Mozilla.. It does not happen with
Mozilla <= 0.9.8 with the same MTA.. The problem also happens using RC1..

It could be a combined pb.. Mozilla + certain MTA.. Our MTA is a Netscape
Messaging 3..

When you are trying to reproduce are you sure you are putting email addresses
and a full name (Ludovic Dubost <ludovic@netvalue.com>) .. Because the problem
only happens in this case.. not if the email address only contains a standard
email (ludovic@netvalue.com)

Comment 5

17 years ago
I confirm the problem.
Happens using Mozilla >= 0.9.9 with MTA = Netscape Messaging Server 3.66
Must be something related to To: and CC: fields composition which changed
between 0.9.8 and 0.9.9.

Seems only to happen with long names with spaces in them like
John Doe <john_doe@doecompany.com>
but not with long names without spaces like
Doe <john_doe@doecompany.com>

I'll check that it doesn't happen if you use another MTA and that reverting to
0.9.8 will solve the problem.

I suggest the bug is reopened so that I can be in the 1.0.0 scope...

Comment 6

17 years ago
confirming based on Thierry's comment.

please provide the information you mentioned when you can
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 7

17 years ago
Confirming this to be a regression : version 0.9.8 is working correctly.

A few of my test cases :

Sending with Mozilla >= 0.9.9 an email with :
To: John Doe <jdoe@tc.com>
To: Jane Doe <jane@dot.com>
-> message will be received by John and Jane
-> headers will just show John
-> every email client will just show John
-> some will show an incomplete To: line like "To: John Doe <jdoe@tc.com>,"
-> INCORRECT BEHAVIOUR

Sending with Mozilla = 0.9.8 the same email :
-> message will be received by John and Jane
-> headers will show John and Jane
-> CORRECT BEHAVIOUR

Sending with Mozilla >= 0.9.9 an email with :
To: John Doe <jdoe@tc.com>
To: Jane <jane@dot.com>
-> message will be received by John and Jane
-> headers will show John and Jane (!)
-> CORRECT BEHAVIOUR

So the problem seems to be conditionned by :
- using multiple To: or Cc: addresses
- using long names with spaces in them
- using the Netscape Messaging Server MTA (I have to check that one)

I will check that behaviour is correct with Exim instead of NMS, but I think it
will be : must be the reason why you don't manage to reproduce when you're not
using NMS.

Something must have changed in the way To: or Cc: addresses are submitted
between 0.9.8 and 0.9.9 that confuses NMS (but sending is correct, it's just the
headers that are incorrect...)

Comment 8

17 years ago
Informative network captures about differences between 0.9.8 and 1.0RC1 :

(1) Moz 0.9.8 writes the multiple To: header in the DATA like this :
To: John Doe <jdoe@tc.com>,[0D][0A][09]Jane Doe <jane@dot.com>[0D][0A]
and it works properly with NMS delivery

(2) Moz 1.0RC1 (and 0.9.9) writes it this way :
To: John Doe <jdoe@tc.com>, Jane Doe[0D][0A] <jane@dot.com>[0D][0A]
and it doesn't work properly.

(3) One-word names are written OK using 1.0RC1 :
To: John Doe <jdoe@tc.com>, Jane <jane@dot.com>[0D][0A]
and it works properly.

I think the 0.9.8 line is buggy but tolerated. It must have been corrected in
0.9.9 but there is still a problem with multiple-words in names (see trace (2)
above) which confuses NMS and might confuse others.

I think it should be corrected before the 1.0 final release since other mail
servers than NMS might be affected by the buggy To: line...

Comment 9

17 years ago
I confirm this is not a NMS-specific bug.

The "To:" and "Cc:" (and probably others like "Bcc:") headers are written
incorrectly since Moz 0.9.9 :
- when you use multiple addresses
- when one of the names used (but not the first) is a multiple-word name like
"Jane Doe <jane@dot.com>"

The part about NMS is that it rewrites the headers before delivering the
message, and the parsing of the wrongly-written headers fails, but this is
because there is a bug in Mozilla, not in NMS.

Reproduced on Linux and Win32.

Updated

17 years ago
QA Contact: gayatri → esther

Comment 10

17 years ago
I originally tried it with the conditions specified running qmail as the mta and
I did not see this problem. 
So as you said, some maybe more tolerant than others.

Comment 11

17 years ago
More info :

The problem is not due to having multiple words in the display name but is due
to having a long name. More precisely, Mozilla 0.9.9 and over can cut the To:
header line between the display name and the address :

To: John Doe <jdoe@tc.com>
To: JohnnieHasAVeryVeryLongName <johnnie@dot.com>

will be cut in two lines between JohnnieHasAVeryVeryLongName and
<johnnie@dot.com> (inserting space) :

To: John Doe <jdoe@tc.com>, JohnnieHasAVeryVeryLongName
 <johnnie@dot.com>

whereas with 0.9.8 and before, it wrote the display name and address on the same
line and will cut between addresses (inserting tab).

To: John Doe <jdoe@tc.com>,
     JohnnieHasAVeryVeryLongName <johnnie@dot.com>

The two behaviours should be accepted by the mail server, but NMS 3.X doesn't
like it and rewrites the headers (why?), dropping addresses from the incomplete
one to the end.

So this could be considered as an NMS bug, but there is no way to work around it
and Mozilla is the only client I know of with this behaviour.

Should this be corrected in Mozilla ? Are there any other mail servers having
problems with the Moz 1.0RC1-style recipient headers ?

Comment 12

17 years ago
A last note on this issue :

I think we should change the title of this bug to :

"New recipient headers folding confuses some mail servers"

because the problem can be reduced to where exactly you should fold the
recipient header lines (like To: or Cc:) in case of long header fields.

From RFC #822 :
 Note:  While the standard  permits  folding  wherever  linear-
 white-space is permitted, it is recommended that struc-
 tured fields, such as those containing addresses, limit
 folding  to higher-level syntactic breaks.  For address
 fields, it  is  recommended  that  such  folding  occur
 between addresses, after the separating comma.

Old Netcape Messaging Server versions (like 3.X) expect the "recommended" way of
folding long header lines, and maybe other mail servers as well. Mozilla 0.9.8
was implementing the "recommended" way. Header folding in Mozilla 1.0RC1 is
standard-compliant strictly-speaking, but not the recommended way.
(Reporter)

Comment 13

17 years ago
From our testing the problem also occurs using Netscape Messaging Server 4.x 
Does annybody know why the header folding has been changed ?

I think this should clearly be fixed because I'm not sure that even if patches
are created for the servers, corporation would not necessasrly deploy them easily..

This can dramatically reduce the adoption of Mozilla in corporations.. and this
is not what we all want...

Comment 14

17 years ago
This is a regression by a fix for #73403.
Change platform and OS to ALL.
OS: Linux → All
Hardware: PC → All

Comment 15

17 years ago
Change component to MIME, cc ducarroz, putterman.

So the current known problem is that address may be removed by NMS 3.X?
Component: Mail Back End → MIME
(Reporter)

Comment 16

17 years ago
Yes.. but only in the headers.. the email are actually sent to all people..

Comment 17

17 years ago
>Yes.. but only in the headers.. the email are actually sent to all people..
so there may be a problem when reply all 

cc ducarroz, putterman
(Reporter)

Comment 18

17 years ago
Yes.. the typical scenario of what happens is:

- somebody sends an email to multiple people in the company.. With the LDAP
server + rewritting of emails addresses by the server, the email comes in with
long names in the addresses

- reply to all using Mozilla 1.0 misses some addresses in the headers

- recepient receive the email with missing headers.. Filters might not put the
email in the right folder and reply to all will not include some users...

After a few weeks of usage this caused several communication problems for me,
and I had to revert to Mozilla 0.9.8 to avoid the problem...

Comment 19

17 years ago
Just tried with NMS 4.15 MTA and works fine with following To: line:

To: Takayuki Tei <taka@netscape.com>, Johnnie Has A Very Very Long Name Johnnie
 Has A Very Very Long Name <takayukitei@netscape.com>

Comment 20

17 years ago
I think this has to be evaluated by mail group, adding nsbeta1.
Keywords: nsbeta1

Comment 21

17 years ago
Remark on comment #19 :

Since the problem depends on where exactly the line folding occurs, it might
work with short names, not work with long names, work with very long names, etc...

But it is also possible that header parsing has been rewritten in NMS 4.15.

To correct this one, I think there is two possibilities : you can either
implement the RFC822 "recommended" way of folding or revert to the pre-0.9.9
recipient header encoding... Both should be compliant with older versions of NMS.

Like Ludovic, I had to delay Mozilla deployment in my company due to header
information lossage.

Comment 22

17 years ago
reassigning to ducarroz.  Jean-Francois, can you look at this?  Marking adt1
until more is known about this.
Assignee: mscott → ducarroz
Keywords: nsbeta1 → nsbeta1+
Whiteboard: [adt1]

Comment 23

17 years ago
>Since the problem depends on where exactly the line folding occurs, it might
>work with short names, not work with long names, work with very long names, etc...

Taka, is that also be applied to MIME encoded case which tends to be a long name?
Is that possible to enable the old folding behavior by option?

Comment 24

17 years ago
> But it is also possible that header parsing has been rewritten in NMS 4.15.

Which patch version are you using?  There have been bunch of header rewrtting
problem in earlier version.  You should upgrade to latest patch.


> Both should be compliant with older versions of NMS.

We have released patch to collect these problem on server.

Do you know any other server implementation which doesn't work correctly
with folded structure headers?


> Like Ludovic, I had to delay Mozilla deployment in my company due to header
> information lossage.

We recommend you to upgrade to latest patch of NMS.

Comment 25

17 years ago
> Taka, is that also be applied to MIME encoded case which tends to be a long name?
> Is that possible to enable the old folding behavior by option?

RFC 2047 "8 Examples" shows folded To: field body with multiple recipients.

RFC 822 "3.1.1.  LONG HEADER FIELDS" also shows folded To: field body with 
multiple recipients.  Since RFC 2047 is designed on top of RFC 822, it's 
impossible to go back to old broken MIME encoding style without seeing
bugs such as #73403.

Comment 26

17 years ago
In comment #24:
> We have released patch to collect these problem on server.

I meant "correct" not "collect". 

Comment 27

17 years ago
Created attachment 81004 [details] [diff] [review]
patch from taka

explanation about the patch copied from e-mail:

Although the attached patch should fix the bug 137742, this bug is
fundamentally caused by a NMS3.x bug.

RFC 822 explicitly shows how long headers can be folded, and RFC 2047 is
designed on top of that folding logic.	This patch works only when given
structured field body is in ASCII, therefore, those MTA which cannot unfold
folded To:/Cc: field body can be screwed up when user reply to multiple
receipients with RFC 2047 encoded names.

Comment 28

17 years ago
On comment #24 :

> Which patch version are you using?  There have been bunch of header rewrtting
> problem in earlier version.  You should upgrade to latest patch.

I'm using NMS version 3.62. If there is a patch which could correct the bug
appliable to this version, I would be interested to download and install it (I
cannot find anymore NMS 3.X patches on IPlanet site). However, if the only way
to get rid of this is to reinstall a complete NMS (or other mail server) I will
have to delay this (we have plans to change the mail server for Q4 2002) and
delay Mozilla deployment (planned for Q2 2002) accordingly.

> Do you know any other server implementation which doesn't work correctly
> with folded structure headers?

No. I tested with sendmail and exim without any problem.

On comment #27 :

> Although the attached patch should fix the bug 137742, this bug is
> fundamentally caused by a NMS3.x bug.

I agree completely. For Mozilla, this is more a compatibility feature than a
bug. But if NMS 3.X is the only mail server having problems, Mozilla is the only
mail client I know of that triggers the problem... 

I'm available and ready to test any win32 build to validate the problem
is gone. Just tell me in which nightly build it will appear. 

And thanks to you all for supporting all these very-specific demands and coping
with my bad English !

Comment 29

17 years ago
I have very simlar problem when I sent message to both NNTP and SMTP. It
sometimes fails. I cannot even see sent messages in Sent folder.
(I use 1.0rc1/Linux and didn't have problem with 0.9.9)

SMTP server is smtp.mail.yahoo.co.jp
NNTP server is news.php.net

I don't send much mails, so I cannot tell if I have the same problem or not. It
seems it's closely related.


Comment 30

17 years ago
There is a known bug 137742 which also causes message delivery problem.
Can you give us a couple of examples you couldn't send message as expected?

Comment 31

17 years ago
*** Bug 141703 has been marked as a duplicate of this bug. ***
(Assignee)

Updated

17 years ago
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.0.1

Updated

17 years ago
Blocks: 143047
(Assignee)

Comment 32

17 years ago
sofar, no luck, I am not able to reproduce this problem using various version of
Mozilla/Netscape, even most recent builds (branch and trunk)
(Assignee)

Comment 33

17 years ago
The only problem I see here is that when we reformat the headers to comply with
RFC2047, we can potentially fold an address in the middle between the long name
and the email address. Then some MTA might tried to reformat the headers and cut
some of the recipients by accident (bug?). I beleive this problem has been
introduced on February 20, 2002.

Taka proposed patch will fix that problem. Reassign to taka,
Assignee: ducarroz → taka
Status: ASSIGNED → NEW
(Assignee)

Comment 34

17 years ago
Comment on attachment 81004 [details] [diff] [review]
patch from taka

R=ducarroz
Attachment #81004 - Flags: review+

Comment 35

17 years ago
Comment on attachment 81004 [details] [diff] [review]
patch from taka

Please make len a PRInt32 instead of a raw int. We should be using the NSPR
types for integers unless the code is platform specific.

Updated

17 years ago
Whiteboard: [adt1] → [adt1 RTM]
(Assignee)

Comment 36

17 years ago
reassign to myself as I have some free cycle...
Assignee: taka → ducarroz
(Assignee)

Comment 37

17 years ago
Created attachment 87115 [details] [diff] [review]
Proposed fix, v2

Replaced int by PRInt32.
Attachment #81004 - Attachment is obsolete: true
(Assignee)

Comment 38

17 years ago
accepting
Status: NEW → ASSIGNED

Comment 39

17 years ago
Comment on attachment 87115 [details] [diff] [review]
Proposed fix, v2

sr=bienvenu
Attachment #87115 - Flags: superreview+
(Assignee)

Comment 40

17 years ago
Comment on attachment 87115 [details] [diff] [review]
Proposed fix, v2

R=ducarroz
Attachment #87115 - Flags: review+
(Assignee)

Comment 41

17 years ago
Fix checked in the trunk
Status: ASSIGNED → RESOLVED
Last Resolved: 17 years ago
Resolution: --- → FIXED

Updated

17 years ago
Keywords: adt1.0.1, mozilla1.0.1

Comment 42

17 years ago
Esther, can you verify this on the trunk?

Comment 43

17 years ago
Correction appears in 2002061208/Win32.
Thanks to everyone...

Comment 44

17 years ago
I'll give it a try, but I need to first reproduce it to be able to verify it. It
will take time.

Comment 45

17 years ago
I spent a gread deal of time trying to reproduce this and couldn't. I took:
linux mozilla build 0.9.9 
linux mozilla build rv:1.Orc1 with Gecko string 20020417 (couldn't find one with
20020415)
linux commercial branch 20020415 
win32 commercial branch 20020610 
All test conducted using Netscape Messaging Server 4.15 and testing addresses
with really long pretty names with spaces (50+characters),  multiple recipients,
mixing addressing using To: and CC:, I could not reproduce this.
Since reporter has tested this and commments that "correction appears in
2002061208/Win32", I will take a trunk build with the fix and do a spot check to
see if basic sending to mutiple recipients is still working. 

Comment 46

17 years ago
You need to test against early version of NMS3.x since this symptom appeared due
to a bug in NMS3.x, not Mozilla.
(Reporter)

Comment 47

17 years ago
I've checked it with build 2002061221 and the problem does not occur on my
computer anymore with a NMS 3.x

Great news.. that will probably allow deployment of the next release of mozilla
in our company..

Comment 48

17 years ago
Lowering impact since it only happens on one server.
Whiteboard: [adt1 RTM] → [adt2 RTM]

Comment 49

17 years ago
esther - can you pls verify this fix on the branch? thanks!
Keywords: verifyme

Comment 50

17 years ago
Esther will not verify that this works with Netscape Messaging Server 3.66 since
we do not actively support that server. However, Esther will verify that this
does not break anything with the servers we do test with, i.e. NMS 6.0, NMS 6.1
and NMS 4.15.

Updated

17 years ago
Whiteboard: [adt2 RTM] → [adt2 RTM] [ETA 06/25]

Comment 51

17 years ago
Has Esther verified that this fix does not cause any regressions, with servers
we currently support?

Comment 52

17 years ago
I have not done formal testing ye.   I'm testing other regressions that have
come up in the last 2 days that are now fixed on the trunk and waiting for
verification.  But I have been using the trunk builds for these regressions for
daily sending to email address of the same type mentioned in the original
scenario, there doesn't seem to be any problems.  Since the reporter is saying
it's fixed with the trunk build, if this hasn't gone into the branch yet I think
that it could.  I am confused by Jaime's question in comment #49 about verifying
it on the branch?  Is the fix in the branch or trunk or both now?

Comment 53

17 years ago
In the past week 6-24 through today 7-1, I have used the trunk builds (this bug
was fixed in trunk on 6-12) and branch builds (I don't see that it was ever
checked into the branch yet) with our NS server 4.15 & 6.0.  I have seen no new
problems with multiple addressees in messages with long names or short names. 
Since we never saw the problem on these servers all I can say is the fix doesn't
appear to have broken anything in the addressing field.  If this was checked
into the branch after 6-27-2002 then I will say this is verified for both trunk
and branch.  If it hasn't been checked into branch, then I will say verified on
trunk. 
(Assignee)

Comment 54

17 years ago
This fix has not been checked in the branch has it has never been approved by ADT.

Comment 55

17 years ago
adding adt1.0.1-.  Let's get this in the next release.
Keywords: adt1.0.1 → adt1.0.1-

Comment 56

16 years ago
I tested this scenario again with latest trunk builds using the 4.15 & 6.0
messaging servers, this works fine.  The fix that was checked in for a 4.33
messaging server bug, so I just verified the 4.15 and 6.0 servers still work. 
Since we will be basing future releases off the current trunk build I will
verify this as fixed.  
Status: RESOLVED → VERIFIED

Comment 57

16 years ago
Did this make it into Netscape 7.02?  I'm seeing this problem with Netscape 7.02
and Netscape Messenger 3.6X.
Product: MailNews → Core
Product: Core → MailNews Core

Updated

10 years ago
Keywords: verifyme
You need to log in before you can comment on or make changes to this bug.