284590 - When using SMTP, emails are not sent to the CC list associated with the flag type

Status: RESOLVED FIXED

(Bugzilla :: Email Notifications, defect)

Description

[tim]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041109 Firefox/1.0
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041109 Firefox/1.0

Using local logins (emailregexp ^[^@]+$) and an emailsuffix, which is not
supposed to be appended to CC List names, does not seem to send an email to the
cc list, instead the MTA comes up with the following error:-

###################################################
Your message

  Subject: chargeable_work requested: [Call 983] Test of charging flag
again
  Sent:    Thu, 3 Mar 2005 09:18:48 -0000

did not reach the following recipient(s):

unknown on Thu, 3 Mar 2005 09:18:48 -0000
    The e-mail system was unable to deliver the message, but did not
report a specific reason.  Check the address and try again.  If it still
fails, contact your system administrator.
    < espy.post #5.0.0 X-Postfix; No recipients specified>
#################################################

I have tried this with a local (to the system) user such as root, and a full
email address [e-mail] (entered by temporarily turning of the
local emailregexp and then turning it back on)

Reproducible: Always

Steps to Reproduce:
1. Set up a local user as described in details
2. Request a flag be set
3.
Actual Results:  
Undeliverable error from the MTA

Expected Results:  
sent an email to the address specified in the CC List field

Comment 1

[Frédéric Buclin]

Can anyone of you confirm?

Comment 2

[Marc Schumann [:Wurblzap]]

This WORKSFORME.

The flag type CC field doesn't use Bugzilla logins. It uses plain e-mail
addresses which need not be Bugzilla logins. Since bug 277389, this actually
works, and the page to edit flag types says so, too. Which version are you using?

Comment 3

[Frédéric Buclin]

so please close the bug

Comment 4

[Max Kanat-Alexander]

WFM per comment 2.

Comment 5

[tim]

I am using 2.18. I will revisit and see if I can get it to work, unless the fix
is in 2.19?

Comment 6

[Marc Schumann [:Wurblzap]]

The fix went into 2.18, too.

(Btw, see bug 278414 on the need to fiddle with emailregexp when adding e-mail
addresses to flag cc lists.)

Comment 7

[tim]

Bug 278141 describes the problem I was trying to work around, without sucess, as
my original comment indicates. The question is should I try 2.18.3 to fix it or
hang on as 2.20rc2 is now out!

Comment 8

[Marc Schumann [:Wurblzap]]

I'm not sure I understand exactly... Are you saying that your initial problem is
that of bug 278414?

Comment 9

[tim]

No! Even when I change the regexp to avoid the red warning as described in
278414 I still get the MTA failure!

Comment 10

[tim]

I have now upgraded to 2.18.3 and the problem remains!

In the flags set up I have active and requestabel ticked and a cc list
containing a single fully qualified email address ([e-mail]) but the
MTA still reports 
"X-Supplementary-Info: < espy.g211.post #5.0.0 X-Postfix; No recipients specified>"

Under other circumstances I do get emails sent as expected so its not a generic
email set up problem.

Comment 11

[tim]

I have now found the circumstances under which the problem will occur and when
it will not.
If a flag is requestable, and there is a cc entry then the error occours, an
email with no recipient is generated.

If a flag is requestable, *and* specifically requestable, and there is a CC
entry the error does not occur.

Comment 12

[Marc Schumann [:Wurblzap]]

Confirming.

Comment 13

[Dave Miller [:justdave]]

This was reported against 2.18, and it's a crasher.  Crashers get fixed.  Also flagging bmo-regressions as it fits the profile with a crash that's been getting reported frequently on bmo lately.

Comment 14

[Teemu Mannermaa (:wicked)]

Looks like this happens when the bug is restricted to one or more groups or the attachment is private AND none of the flag CC list emails are allowed to see the bug/attachment. Note that the code starting from line 950 on Bugzilla/Flag.pm specifically excludes all non-account emails.

Maybe we should add a check for no to/cc/bcc condition somewhere and just not try to sent the notification in this case? Or maybe we should reconsider this block about non-account emails never getting notifications about restricted bugs/attachments?

Comment 15

[Frédéric Buclin]

What you describe in comment 14 sounds like a dupe of bug 266147. Comment 11 seems to be a different problem.

Comment 16

[Marc Schumann [:Wurblzap]]

Comment 14 is the "other problem". This bug is about comment 11.

Comment 17

[Teemu Mannermaa (:wicked)]

(In reply to comment #16)
> Comment 14 is the "other problem". This bug is about comment 11.

What's this "other problem"? Bug 266147? I can trigger problem described in comment 11 when environment is how I described in comment 14.

(In reply to comment #15)
> What you describe in comment 14 sounds like a dupe of bug 266147. Comment 11

Hmm, this would mean there is another problem somewhere in the code with same symptoms as in comment 14. Could happen but I somehow doubt it.. Maybe we should fix bug 266147 first and then see if somebody can still reproduce this bug.

Comment 18

[Marc Schumann [:Wurblzap]]

(In reply to comment #17)
> Hmm, this would mean there is another problem somewhere in the code with same
> symptoms as in comment 14. Could happen but I somehow doubt it.. Maybe we
> should fix bug 266147 first and then see if somebody can still reproduce this
> bug.

That's what I thought might be the case... My intentions in comment 16 were to prevent this bug being resolved as a duplicate of bug 266147 before we're sure whether we're looking at two different issues or not.

Comment 19

[Teemu Mannermaa (:wicked)]

When flag is requested from no-one the flag notification has an empty To field. This happens when either no requestee was specified or the flag is not specifically requestable. There's only a CC field with the flag's CC list email.

I tested with sendmail and my flag mail was delivered succesfully in all test cases. But with SMTP mail delivery to same sendmail host resulted with no flag mail. Actually, SMTP doesn't even send an email to the CC list email but only to the requestee. So it seems the SMTP delivery basicly ignores the CC field of the message.

Uh oh, from the Mail::Mailer documentation: The smtp mailer does not handle "Cc" and "Bcc" lines, neither their "Resent-*" fellows. So this is probably what we are running into here. :( Do we use CC or BCC email fields anywhere else?

Other problem I noticed with these mails is that they contain ".. has asked  for test-cclist.." text. Note the two blanks where the requestee should be located. Shouldn't this say "from wind" or just have one blank?

Comment 20

[Marc Schumann [:Wurblzap]]

For the record, comment 12 is based on seeing the effect in a mail_delivery_method=testfile environment.

Comment 21

[Frédéric Buclin]

(In reply to comment #19)
> Other problem I noticed with these mails is that they contain ".. has asked 
> for test-cclist.." text. Note the two blanks where the requestee should be
> located. Shouldn't this say "from wind" or just have one blank?

Minor cleanup unrelated to this bug. I really don't care seeing two whitespaces instead of one. Much more important is the problem about SMTP ignoring the CC list.

If we haven't any other choice, then the fix is to loop over the CC list and add them in turn to the To: field in Bugzilla::Flag::notify(), which is a trivial fix, especially thanks to my fix for bug 266147, but is slower as we cannot send only one email (but generally, I don't expect the flag CC list to be very long).

Comment 22

[Max Kanat-Alexander]

I'd rather see people in the "To" field anyhow, instead of exposing their email address to everybody via "CC."

Comment 23

[Teemu Mannermaa (:wicked)]

I tested SMTP mailer and found following restrictions:
 1) Only one address per one To field is supported because comma separated addresses will not be processed. However, multiple To fields can be used.
 2) CC or BCC fields are not supported. It tries to but there's a bug between Mail::Header and Mail::Mailer, first one created all uppercase hash refs for CC/BCC and latter expects mixed case.

Maybe we should add something about this to release notes so that people editing mail parameters or templates knows to avoid these constructs?

Comment 24

[Teemu Mannermaa (:wicked)]

Attachment: Fix CC of flag notifications, V1

This patch changes notification mails by:
 1) No longer uses CC field
 2) Uses only one address per one To field
 3) Ignores blank addresses

Comment 25

[:glob ✱]

according to rfc2822 (3.6) you're only allowed one To: field

i have encountered issues with some systems not liking multiple to fields and had to work around them for other projects.

Comment 26

[Frédéric Buclin]

Comment on attachment 206953 [details] [diff] [review]
Fix CC of flag notifications, V1

I'm with glob. I don't like having multiple To: fields. I think the best solution is to do a loop in Flag::notify() over the addressee and the CC list, and adding a new $vars->{'to'} field which is used by the To: field only. This way, you can keep the addressee field as is.

Comment 27

[Teemu Mannermaa (:wicked)]

Attachment: Fix CC of flag notifications, V2

Moved loop to Flag.pm module to proces and send flag notification mail for each recipient.

Comment 28

[Frédéric Buclin]

Comment on attachment 207122 [details] [diff] [review]
Fix CC of flag notifications, V2

r=LpSolit for tip only. This patch needs to be backported.

Comment 29

[Teemu Mannermaa (:wicked)]

Attachment: Fix CC of flag notifications, V2 - 2.20 branch

Comment 30

[Teemu Mannermaa (:wicked)]

Hmm, looks like 2.18 branch is not affected by the problems fixed in the patches in this bug. It doesn't have the SMTP mail delivery method and sendmail should support CC mail fields correctly.

This MTA error in 2.18 is most likely effect of the bug 266147 like I originally suspected.

Comment 31

[Frédéric Buclin]

Comment on attachment 207130 [details] [diff] [review]
Fix CC of flag notifications, V2 - 2.20 branch

r=LpSolit

Comment 32

[Frédéric Buclin]

tip:

Checking in Bugzilla/Flag.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Flag.pm,v  <--  Flag.pm
new revision: 1.59; previous revision: 1.58
done
Checking in template/en/default/request/email.txt.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/request/email.txt.tmpl,v  <--  email.txt.tmpl
new revision: 1.11; previous revision: 1.10
done


2.20:

Checking in Bugzilla/Flag.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Flag.pm,v  <--  Flag.pm
new revision: 1.45.2.4; previous revision: 1.45.2.3
done
Checking in template/en/default/request/email.txt.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/request/email.txt.tmpl,v  <--  email.txt.tmpl
new revision: 1.9.4.2; previous revision: 1.9.4.1
done

Comment 33

[Max Kanat-Alexander]

There's no reason for this bug to have the relnote keyword on it anymore--these versions have been released long since.