1246228 - Email addresses must not be encoded

Status: RESOLVED FIXED

(Bugzilla :: Email Notifications, defect)

Description

[Stuart]

User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0
Build ID: 20151102093252

Steps to reproduce:

Upgraded perl Encode from 2.78 to 2.80 (using Fedora 23 RPMs)
Using email submission method of sendmail, with exim 4.86.


Actual results:

Emails from my bugzilla installation started bouncing. Examining the bounces the mails were being sent to =?UTF-8?Q?stuart=40gmail=2Ecom?=@example.com


Expected results:

It appears as if exim doesn't currently understand internationalized
email addresses using MIME encoded words. So not seeing a valid domain
in the address it appends the local domain before forwarding it.

The reason I'm seeing this now appears to be linked to the upgrade of the
perl Encode library. For example:
perl -E 'use Encode; print encode("MIME-Q", "stuart\@gmail.com") . "\n";'
using version 2.78 produces:
[e-mail]
while on 2.80:
=?UTF-8?Q?stuart=40gmail=2Ecom?=

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Considering this seems to impact several releases of the upstream module, we should have a work-around.

Comment 3

[Frédéric Buclin]

I can confirm the issue with Encode 2.82. Using MIME-B or MIME-Header doesn't help.

@dan: you are the maintainer of the Encode module on CPAN. Could you tell us if this is a regression or an intentional change? If it's intentional, is there a way to fall back to the old behavior?

Comment 4

[Frédéric Buclin]

http://search.cpan.org/diff?from=Encode-2.78&to=Encode-2.80&w=1#lib/Encode/MIME/Header.pm

I suspect that the removal of

  if (   $word =~ /[^\x00-\x7f]/o ...

is responsible for this problem. It looks like headers are encoded in all cases.

Comment 5

[Dylan Hardison [:dylan] (he/him)]

Looks related to this: https://rt.cpan.org/Ticket/Display.html?id=111853

Also, heh, See Also doesn't support RT?

Comment 6

[Frédéric Buclin]

(In reply to Dylan William Hardison [:dylan] from comment #5)
> Also, heh, See Also doesn't support RT?

It does, if the MoreBugUrl extension is enabled.

Comment 7

[Dylan Hardison [:dylan] (he/him)]

Based on https://github.com/dankogai/p5-encode/issues/28, we might consider using https://metacpan.org/pod/MIME::EncWords
to handle encoding. 

As an immediate stopgap, we can specify (in Makefile.PL) that we need Encode < 2.80 for master,
and use the previous blacklisting feature for 5.0 branch.

Comment 8

[Frédéric Buclin]

The problem is on our side, not on Encode side, see https://rt.cpan.org/Public/Bug/Display.html?id=111853#txn-1611993. As stated there, and as correctly done in Email::MIME::Encode already, only the phrase must be encoded, not the email address itself. But in Bugzilla::MIME, we encode the whole string, including the email address. So we are doing it wrong.

Comment 9

[Frédéric Buclin]

Attachment: patch for master, v1

header_str_set() has special code which correctly encodes what must be encoded, and leaves alone what must not be encoded, such as email addresses, see http://cpansearch.perl.org/src/RJBS/Email-MIME-1.937/lib/Email/MIME/Encode.pm

Comment 10

[Frédéric Buclin]

Attachment: patch for 5.0, v1

Comment 11

[Frédéric Buclin]

Attachment: patch for 5.0, v1.1

|use Encode::MIME::Header| is required by attachment.cgi to alter $Encode::Encoding{'MIME-Q'}->{'bpl'}. This missing dependency has never been caught before because Bugzilla::MIME was (luckily?) loaded before view() in attachment.cgi.

Comment 12

[Frédéric Buclin]

Attachment: patch for master, v1.1

Comment 13

[David Lawrence [:dkl]]

Comment on attachment 8736874 [details] [diff] [review]
patch for master, v1.1

Review of attachment 8736874 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl

Comment 14

[David Lawrence [:dkl]]

Comment on attachment 8736873 [details] [diff] [review]
patch for 5.0, v1.1

Review of attachment 8736873 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl

Comment 15

[Frédéric Buclin]

To ssh://[e-mail]/bugzilla/bugzilla.git
   6c14fac..60299ec  master -> master

To ssh://[e-mail]/bugzilla/bugzilla.git
   aa5f64e..652ee91  5.0 -> 5.0

Comment 17

[Dave Miller [:justdave]]

Dataloss (dropped email) qualifies for a backport in my opinion, unless there are architectural reasons that we can't.

Comment 18

[Frédéric Buclin]

Attachment: patch for 4.4, v1

Comment 19

[Emmanuel Seyman]

JFTR, I've tested the tip of the 4.4 branch and can confirm it suffers from the issue (email bounce). I can also confirm that LpSolit's patch fixes this.

Comment 20

[David Lawrence [:dkl]]

Comment on attachment 8749602 [details] [diff] [review]
patch for 4.4, v1

Review of attachment 8749602 [details] [diff] [review]:
-----------------------------------------------------------------

Confirmed fixed. r=dkl

Comment 21

[Frédéric Buclin]

To ssh://[e-mail]/bugzilla/bugzilla.git
   01ad7ac..f630337  4.4 -> 4.4

Comment 22

[Markus Wagner]

Installed Bugzilla 5.0.3 today and I could verify the fix.