714488 - Outgoing bugmail containing non-ascii unicode is getting stuck in the queue

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, defect)

Description

[Dave Miller [:justdave]]

There's almost 16,000 emails stuck in the outgoing queue within Bugzilla on b.m.o right now.  They all show as erroring out with one of these errors:

Wide character in subroutine entry at lib/x86_64-linux-thread-multi/Encode.pm line 174.

ascii "\xE9" does not map to Unicode at lib/x86_64-linux-thread-multi/Encode.pm line 174.

ascii "\xFC" does not map to Unicode at lib/x86_64-linux-thread-multi/Encode.pm line 174.

etc.

Comment 1

[David Lawrence [:dkl]]

I would like to see a few examples with the full email text to get an idea of what to look for. Are these emails the new ones with patches as attachments by chance?

dkl

Comment 2

[Frédéric Buclin]

(In reply to David Lawrence [:dkl] from comment #1)
> I would like to see a few examples with the full email text to get an idea
> of what to look for. Are these emails the new ones with patches as
> attachments by chance?

I don't think that's related to attachments. One example could be bug 714472. We cannot send you a copy of these emails if they aren't sent. ;)

Comment 3

[Dave Miller [:justdave]]

They're still queued in the ts_jobs table.

Comment 4

[Dave Miller [:justdave]]

OK, I peeked at a few, they're normal bugmails, that do happen to have non-ascii characters in the comments that were being mailed about.

Based on the timestamps of what's in the queue, it looks like something got pushed on December 5th that broke it.  (We have mail dating back to 12/5 still in the queue continually getting that error).

Comment 5

[:glob ✱]

bug 696005 looks the likely candidate given that timeframe.
/me investigates

Comment 6

[:glob ✱]

trace:

/data/www/bugzilla.mozilla.org/lib/i386-linux-thread-multi/Encode.pm line 174
	Encode::decode('us-ascii', '...', 1) called at /data/www/bugzilla.mozilla.org/lib/Email/MIME.pm line 340
	Email::MIME::body_str('Email::MIME=HASH(0xd051d50)') called at /data/www/bugzilla.mozilla.org/extensions/Profanivore/Extension.pm line 76
	Bugzilla::Extension::Profanivore::mailer_before_send('Bugzilla::Extension::Profanivore=HASH(0xd2b973c)', 'HASH(0xd3e266c)') called at /data/www/bugzilla.mozilla.org/Bugzilla/Hook.pm line 33
	Bugzilla::Hook::process('mailer_before_send', 'HASH(0xd3e266c)') called at /data/www/bugzilla.mozilla.org/Bugzilla/Mailer.pm line 151
	Bugzilla::Mailer::MessageToMTA('...') called at /data/www/bugzilla.mozilla.org/Bugzilla/BugMail.pm line 596
	Bugzilla::BugMail::sendMail('HASH(0xd4e2428)') called at /data/www/bugzilla.mozilla.org/Bugzilla/BugMail.pm line 436
	Bugzilla::BugMail::Send(686499, 'HASH(0xba71590)', 'HASH(0xd3e38e4)') called at /data/www/bugzilla.mozilla.org/Bugzilla/Bug.pm line 1201
	Bugzilla::Bug::_send_bugmail('HASH(0xd3e38e4)', 'HASH(0xcf7a74c)') called at /data/www/bugzilla.mozilla.org/Bugzilla/Bug.pm line 1150
	Bugzilla::Bug::send_changes('Bugzilla::Bug=HASH(0xd6de5cc)', 'HASH(0xd44f75c)', 'HASH(0xcf7a74c)') called at /data/www/bugzilla.mozilla.org/process_bug.cgi line 393
	ModPerl::ROOT::Bugzilla::ModPerl::ResponseHandler::data_www_bugzilla_2emozilla_2eorg_process_bug_2ecgi::handler('Apache2::RequestRec=SCALAR(0xd04fc5c)') called at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/ModPerl/RegistryCooker.pm line 204
	eval {...} called at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/ModPerl/RegistryCooker.pm line 204
	ModPerl::RegistryCooker::run('Bugzilla::ModPerl::ResponseHandler=HASH(0xd5afc8c)') called at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/ModPerl/RegistryCooker.pm line 170
	ModPerl::RegistryCooker::default_handler('Bugzilla::ModPerl::ResponseHandler=HASH(0xd5afc8c)') called at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/ModPerl/Registry.pm line 31
	ModPerl::Registry::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0xd04fc5c)') called at /data/www/bugzilla.mozilla.org/mod_perl.pl line 138
	Bugzilla::ModPerl::ResponseHandler::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0xd04fc5c)') called at -e line 0
	eval {...} called at -e line 0

Comment 7

[:glob ✱]

what's happening is Email::MIME is treating the email body as us-ascii, and is calling:

> decode('us-ascii', '«tier3»', 1);

which is failing.


/me investigates setting some headers to give Email::MIME a better hint

Comment 8

[:glob ✱]

it looks like we're stuck between a rock and a hard place here -- you can't add an attachment with an extension if you set the encoding header before calling hooks, however you can't retrieve the email body from an extension if you set the encoding header after calling hooks.

however max's suggestion on bug 696005 of setting the transfer encoding to 8bit means that the email isn't double-encoded when modified by an extension, neatly fixing the issue bug 696005 tries to address.


it looks like the simplest thing to do is:

1. revert the patch from bug 696005
2. disable inline patches from bug 689601
3. fix bug 714724 and backport to bmo
4. re-enable inline patches

Comment 9

[:glob ✱]

Attachment: patch v1

Comment 10

[David Lawrence [:dkl]]

Comment on attachment 585365 [details] [diff] [review]
patch v1

Review of attachment 585365 [details] [diff] [review]:
-----------------------------------------------------------------

Normal email works as expected with the patch applied. r=dkl

Comment 11

[:glob ✱]

rollback:

Committing to: bzr+ssh://[e-mail]/bmo/4.0/
modified Bugzilla/Mailer.pm
modified extensions/BMO/Extension.pm
Committed revision 8004.