Quoted-printable bugmail has a =0D at the end of every line

RESOLVED FIXED in Bugzilla 3.2

Status

()

Bugzilla
Email Notifications
RESOLVED FIXED
8 years ago
7 years ago

People

(Reporter: philor, Assigned: Max Kanat-Alexander)

Tracking

3.2.3
Bugzilla 3.2
Bug Flags:
approval +
approval3.4 +
blocking3.4 +
approval3.2 +
blocking3.2.4 +

Details

Attachments

(2 attachments, 4 obsolete attachments)

(Reporter)

Description

8 years ago
Starting today on b.m.o, and reproducible on http://landfill.bugzilla.org/bugzilla-3.2-branch/ but not http://landfill.bugzilla.org/bugzilla-tip/, any bugmail which needs to have the body quoted-printable encoded has every line ending with a =0D, which at least in Thunderbird trunk on Windows makes it look like it's double-spaced.

STR:
1. Change your landfill prefs to get mail when you make changes, and change your name to Frédéric.
2. CC yourself on any bug, in both tip and 3.2
3. Read the resulting bugspam in Thunderbird/Win, or view-source (Gmail's website won't show it, since they do whatever they please with message bodies, but their "original" view will at least show the =0D)

No, I'm not sure why Tb/Win shows those mails as double-spaced and Tb/Mac doesn't, but in any case I doubt that 3.2-only behavior is intended. From the timing I'd bet on bug 467920, but I lack a test install to be sure.
Flags: blocking3.2.3?
(Reporter)

Comment 1

8 years ago
Created attachment 370296 [details]
OK email from tip
(Reporter)

Comment 2

8 years ago
Created attachment 370297 [details]
Overencoded email from 3.2
(Assignee)

Comment 3

8 years ago
Well, that patch went into both 3.2 and tip, and all it's doing at the moment is making sure that email lines correctly end with CRLF per RFC2822 (which they actually didn't before). I'll take a look at the attached emails, though. This could be a problem with the particular versions of Perl modules installed on bmo or on landfill, too.
(Assignee)

Updated

8 years ago
Attachment #370296 - Attachment mime type: message/rfc822 → text/plain
(Assignee)

Updated

8 years ago
Attachment #370297 - Attachment mime type: message/rfc822 → text/plain
(Assignee)

Comment 4

8 years ago
Comment on attachment 370297 [details]
Overencoded email from 3.2

This looks like the translated source, not the raw source. Any way to get the raw source of emails that you're getting? Of course, I can probably generate them myself, too, but it'd be nice to have them from your end and also mine.
(Assignee)

Comment 5

8 years ago
We've actually released 3.2.3. We need a blocking3.2.4 flag.
Flags: blocking3.2.3? → blocking3.2.3-
(Reporter)

Comment 6

8 years ago
Comment on attachment 370297 [details]
Overencoded email from 3.2

Curse you, Thunderbird, and whoever thought that decoding qp while saving an attachment was a good idea!
Attachment #370297 - Attachment is obsolete: true
(Reporter)

Updated

8 years ago
Attachment #370296 - Attachment is obsolete: true
(Reporter)

Comment 7

8 years ago
Created attachment 370326 [details]
Overencoded (but not then decoded) email from 3.2
Flags: blocking3.2.4?
(Assignee)

Comment 8

8 years ago
Comment on attachment 370326 [details]
Overencoded (but not then decoded) email from 3.2

Okay, yeah, this is what I see Bugzilla actually sending.
(Assignee)

Comment 9

8 years ago
So the authoritative answer on this seems to be:

http://www.freesoft.org/CIE/RFC/1521/6.htm (See Rule #4)

Which indicates that lines should ALWAYS end in CRLF if you're encoding quoted-printable text. In actual fact we are doing that legally correctly. The fact that our encoder has chosen to encode one of them as =0D and one of them as a literal LF is strange, but in actual fact the MUA should be handling that correctly, if I am reading the spec correctly.

In other words, we may be the only people generating emails in this format, and that may be exposing a bug in Thunderbird's QP decoder.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → INVALID
(Assignee)

Comment 10

8 years ago
Oh wait. Actually, Rule #1:

      Rule #1: (General 8-bit representation) Any octet, except those
      indicating a line break according to the newline convention of the
      canonical (standard) form of the data being encoded, may be
      represented by an "=" followed by a two digit hexadecimal
      representation of the octet's value.

So in fact it is illegal for our QP encoder to be encoding those CRs as =0D.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
(Assignee)

Comment 11

8 years ago
Okay, so the problem is https://rt.cpan.org/Ticket/Display.html?id=44709 , which I just filed. I've looked into working around this in Bugzilla by bypassing Email::MIME::Modifier and just using MIME::QuotedPrint directly, but we would have to hack around dangerously with the internals of our Email::MIME object if I did that--something that I think would be a very bad idea.

(For those curious, we'd have to be manually setting $email->{body} and $email->{body_raw} and bypassing $email->body_set, and who knows what consequences that could have.)
Maybe the fix for this bug will fix Bug 365713 too.
(Assignee)

Comment 13

8 years ago
Created attachment 370336 [details] [diff] [review]
v1

Okay, this patch bypasses Email::MIME::Encodings and does the encoding directly, still resulting in valid emails. However, I'd rather see the bug fixed in Email::MIME::Encodings, so I'm going to wait to see if that happens before we actually check in this patch.
Assignee: email-notifications → mkanat
Status: REOPENED → ASSIGNED
Attachment #370336 - Flags: review?(bugzilla)
(Assignee)

Comment 14

8 years ago
I also posted a patch for Email::MIME::Encodings, so hopefully things will just get fixed upstream.
Assignee: mkanat → email-notifications
Flags: blocking3.2.3-
(Assignee)

Updated

8 years ago
Assignee: email-notifications → mkanat
OS: Windows XP → All
Hardware: x86 → All
Target Milestone: --- → Bugzilla 3.2
(Assignee)

Updated

8 years ago
Version: 3.2.2 → 3.2.3
(Assignee)

Comment 15

8 years ago
I'll block 3.2.4 on this. Either we'll require a newer version of Email::MIME::Encodings, or the above patch will go into Bugzilla.
Flags: blocking3.2.4? → blocking3.2.4+
(Assignee)

Comment 16

8 years ago
(In reply to comment #12)
> Maybe the fix for this bug will fix Bug 365713 too.

  Oh hey, yeah...in fact, I believe that the bugs are identical! :-) Though this one started affecting every email as of 3.2.3.
(Assignee)

Updated

8 years ago
Duplicate of this bug: 365713
Comment on attachment 370336 [details] [diff] [review]
v1

r=glob
Attachment #370336 - Flags: review?(bugzilla) → review+

Comment 19

8 years ago
I applied the patch in attachment 370336 [details] [diff] [review] on a local 3.2.3 installation, and can confirm that the issue (superfluous =0D) is resolved by that patch.
(Reporter)

Updated

8 years ago
Duplicate of this bug: 488356
max has chatted with the maintainer of the Email:: modules and this will be fixed upstream (just needs some test written).
(Assignee)

Comment 22

8 years ago
Created attachment 373252 [details] [diff] [review]
v2 - Up Email::MIME::Encodings requirement

rjbs is going to release a new version of Email::MIME::Encodings later tonight, 1.313, which fixes this bug. (I tested it on landfill by checking out his unreleased code from Git.) So this just bumps up the version number requirement.
Attachment #370336 - Attachment is obsolete: true
Attachment #373252 - Flags: review?(LpSolit)
(Assignee)

Comment 23

8 years ago
Created attachment 373253 [details] [diff] [review]
v3

This also fixes the release notes.
Attachment #373252 - Attachment is obsolete: true
Attachment #373253 - Flags: review?(LpSolit)
Attachment #373252 - Flags: review?(LpSolit)

Updated

8 years ago
Attachment #373253 - Flags: review?(LpSolit) → review+

Comment 24

8 years ago
Comment on attachment 373253 [details] [diff] [review]
v3

Looks good, but this shouldn't land before Email::MIME::Encodings 1.313 is available on CPAN as nobody can upgrade in that case. r=LpSolit

Comment 25

8 years ago
Oh, it's already available on CPAN, cool.
Flags: blocking3.4+
Flags: approval3.4+
Flags: approval3.2+
Flags: approval+
(Assignee)

Comment 26

8 years ago
tip:

Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.62; previous revision: 1.61
done
Checking in template/en/default/pages/release-notes.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/pages/release-notes.html.tmpl,v  <--  release-notes.html.tmpl
new revision: 1.35; previous revision: 1.34
done


3.4:

Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.60.2.1; previous revision: 1.60
done
cvs Checking in template/en/default/pages/release-notes.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/pages/release-notes.html.tmpl,v  <--  release-notes.html.tmpl
new revision: 1.33.2.2; previous revision: 1.33.2.1
done


3.2:

Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.47.2.7; previous revision: 1.47.2.6
done
Checking in template/en/default/pages/release-notes.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/pages/release-notes.html.tmpl,v  <--  release-notes.html.tmpl
new revision: 1.17.2.18; previous revision: 1.17.2.17
done
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Assignee)

Updated

8 years ago
Duplicate of this bug: 489064
Blocks: 489907
(Assignee)

Updated

8 years ago
Duplicate of this bug: 365790

Updated

7 years ago
Duplicate of this bug: 534607
You need to log in before you can comment on or make changes to this bug.